index problems after update

Fri Feb 15 11:03:14 EET 2019

El 15/02/2019 a las 8:47, Hajo Locke via dovecot escribió:
> Hello,
>
> Am 11.02.2019 um 09:12 schrieb Hajo Locke via dovecot:
>> Hello,
>> Am 08.02.2019 um 09:25 schrieb Hajo Locke via dovecot:
>>> Hello List,
>>>
>>> i have a problem with index-files which is separated in 2 
>>> subproblems. May be these problems are connected.
>>> Currently we use Ubuntu 18.04 LTS which is bundled with dovecot 
>>> 2.2.33.2
>>> These servers are fresh installed machines and users are added to 
>>> the system after, there was no upgrade.
>>> Sometimes it happens, that dovecot stops showing new mail. There is 
>>> no error in log, dovecot just seems to do his normal operations but 
>>> is not delivering new emails from /var/mail/myuser
>>> emails are delivered again if i delete index-files from .imap 
>>> folder. So index is recreated and everything works again. I cant 
>>> reproduce this problem, but every 2 days i have one user who is 
>>> reporting this problem.
>>>
>
> Iam surprised that no one has an opinion in this case. Nobody else 
> noticed this problem?
> Yesterday i had a customer who was affected twice. First time we saw 
> an errormessage, which leads to an index problem:
>
> Feb 13 15:46:58 hostname dovecot[30097]: imap(username): Error: 
> Corrupted record in index cache file 
> /home/popuser/username/mail/.imap/INBOX/dovecot.index.cache: UID 
> 26876: Broken physical size in mailbox INBOX: read(/var/mail/username) 
> failed: Cached message size smaller than expected (2118 < 8088, 
> box=INBOX, UID=26876)
> Feb 13 15:46:58 hostname dovecot[30097]: imap(username): Error: 
> read(/var/mail/username) failed: Cached message size smaller than 
> expected (2118 < 8088, box=INBOX, UID=26876) (FETCH BODY[] for mailbox 
> INBOX UID 26876)
>
> At this point no new incoming mail is delivered by imapd. We deleted 
> index files, dovecot reacreated them and all was working again.
> a day later same user was affected again. But like all other cases 
> before there is no error or unusual logline. i turned mail_debug on, 
> but no further hint.
> There are definitely new Mails in /var/mail/username but dovecot still 
> shows old state from cache. I deleted index-files again and 
> imap-client showed a lot of new unread mails.
> In log i can see that people are active in imap-clients and managing 
> mails. If a new mail is coming in this period it seems to harm 
> index-files some times. We never noticed same behaviour with long used 
> dovecot 2.2.22
>
> It seems that 2.2.33.2 which is bundled with Ubuntu18.04 LTS has a 
> special problem.
>
> Please help. What is your advice?
>
>
>>> Second problem is similiar. I Upgraded a server from Ubuntu 16.04 to 
>>> Ubuntu 18.04. This includes an upgrade from dovecot 2.2.22 to 
>>> 2.2.33.2. After that i installed an self packaged dovecot 2.2.36.1
>>> Now here is same problem, dovecot is not showing new mails, but the 
>>> difference is we have a Raw backtrace in Log just as expected:
>>>
>> i think this crash is a special problem of dovecot 2.2.36.1.
>> if i downgrade to 2.2.33.2 all is working well, upgrading again to 
>> 2.2.36.1 leeds to same crash.
>>> Feb  8 08:45:37 hostname dovecot[14882]: imap(myuser): Error: Raw 
>>> backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xa1ee2) 
>>> [0x7f78b3b2cee2] -> /usr/lib/dovecot/libdovecot.so.0(+0xa1fda) 
>>> [0x7f78b3b2cfda] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) 
>>> [0x7f78b3abb8a8] -> 
>>> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_parse_header+0x78a) 
>>> [0x7f78b3e7940a] -> /usr/lib/dovecot/libdovecot.so.0(+0x83592) 
>>> [0x7f78b3b0e592] -> 
>>> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x74) 
>>> [0x7f78b3b38814] -> 
>>> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) 
>>> [0x7f78b3b392fd] -> 
>>> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_get_header_stream+0x4a) 
>>> [0x7f78b3e7a2ea] -> 
>>> /usr/lib/dovecot/libdovecot-storage.so.0(mail_get_header_stream+0x4a) 
>>> [0x7f78b3dfd8da] -> 
>>> /usr/lib/dovecot/libdovecot-storage.so.0(imap_msgpart_open+0x24a) 
>>> [0x7f78b3ebb57a] -> dovecot/imap [myuser ip.ip.ip.ip 
>>> FETCH](+0x2081c) [0x5581ac76d81c] -> dovecot/imap [m038422 
>>> ip.ip.ip.ip FETCH](+0x1ea54) [0x5581ac76ba54] -> dovecot/imap 
>>> [myuser ip.ip.ip.ip FETCH](imap_fetch_more+0x39) [0x5581ac76cd69] -> 
>>> dovecot/imap [myuser ip.ip.ip.ip FETCH](cmd_fetch+0x31b) 
>>> [0x5581ac75e1bb] -> dovecot/imap [myuser ip.ip.ip.ip 
>>> FETCH](command_exec+0x5c) [0x5581ac76a09c] -> dovecot/imap [myuser 
>>> ip.ip.ip.ip FETCH](+0x1b612) [0x5581ac768612] -> dovecot/imap 
>>> [myuser ip.ip.ip.ip FETCH](+0x1b6ac) [0x5581ac7686ac] -> 
>>> dovecot/imap [myuser ip.ip.ip.ip FETCH](client_handle_input+0x18d) 
>>> [0x5581ac768a6d] -> dovecot/imap [myuser ip.ip.ip.ip 
>>> FETCH](client_input+0xac) [0x5581ac768fbc] -> 
>>> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x52) 
>>> [0x7f78b3b43482] -> 
>>> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x12e) 
>>> [0x7f78b3b44b9e] -> 
>>> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x36) 
>>> [0x7f78b3b43516] -> 
>>> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f78b3b436c8] 
>>> -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) 
>>> [0x7f78b3ac6aa3] -> dovecot/imap [myuser ip.ip.ip.ip 
>>> FETCH](main+0x329) [0x5581ac75b159] -> 
>>> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) 
>>> [0x7f78b36bbb97] -> dovecot/imap [myuser ip.ip.ip.ip 
>>> FETCH](_start+0x2a) [0x5581ac75b2fa]
>>> Feb  8 08:45:37 hostname dovecot[14882]: imap(myuser): Fatal: 
>>> master: service(imap): child 14135 killed with signal 6 (core dumped)
>>>
>>>
>>>
>> Thanks,
>> Hajo
>>
> Thanks,
> Hajo
>
Hi,

A few days ago I posted a similar issue. My environment has RHEL6 and 7 
servers. When I last updated the servers RHEL6 servers mantained 
2.2.10-1_14.el6.x86_64 version, while RHEL7 updated dovecot from 
2.2.10-8.el7.x86_64 to 2.2.36-3.el7.x86_64. When the RHEL7 servers (used 
for sympa) processed a message for a user, its indexes were corrupted, 
and the user could't access his inbox through webmail, so I had to 
delete dovecot.* files from the user mail path to get it working again. 
This is the error shown at the dovecot log:

Feb  5 09:33:25 buzon1 dovecot: imap(user at domain): Panic: file 
mail-index-sync-keywords.c: line 227 (keywords_update_records): 
assertion failed: (data_offset >= sizeof(struct mail_index_record))
Feb  5 09:33:25 buzon1 dovecot: imap(user at domain): Error: Raw backtrace: 
/usr/lib64/dovecot/libdovecot.so.0() [0x3713268b8a] -> 
/usr/lib64/dovecot/libdovecot.so.0() [0x3713268bf6] -> 
/usr/lib64/dovecot/libdovecot.so.0() [0x37132224aa] -> 
/usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_sync_keywords+0x7fd) 
[0x3713aca43d] -> 
/usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_sync_record+0xec) 
[0x3713acacac] -> 
/usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_sync_map+0x234) 
[0x3713acbae4] -> 
/usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_map+0x83) 
[0x3713abcce3] -> 
/usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_refresh+0xe) 
[0x3713ab793e] -> 
/usr/lib64/dovecot/libdovecot-storage.so.0(maildir_sync_header_refresh+0x10) 
[0x3713a4e1e0] -> /usr/lib64/dovecot/libdovecot-storage.so.0() 
[0x3713a4e330] -> /usr/lib64/dovecot/libdovecot-storage.so.0() 
[0x3713a4f3d4] -> /usr/lib64/dovecot/libdovecot-storage.so.0() 
[0x3713a4f7b3] -> 
/usr/lib64/dovecot/libdovecot-storage.so.0(maildir_storage_sync_init+0xd9) 
[0x3713a4fa59] -> 
/usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x31) 
[0x3713a7d731] -> 
/usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_sync+0x27) 
[0x3713a7e7b7] -> 
/usr/lib64/dovecot/libdovecot-storage.so.0(index_storage_get_status+0x62) 
[0x3713aa8fc2] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0xc3ec) 
[0x7fd272dfe3ec] -> 
/usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_get_status+0x5c) 
[0x3713a7f4fc] -> dovecot/imap(imap_status_get+0x7a) [0x41dbea] -> 
dovecot/imap(cmd_status+0x179) [0x413059] -> 
dovecot/imap(command_exec+0x3d) [0x4170bd] -> dovecot/imap() [0x416180] 
-> dovecot/imap() [0x41627a] -> dovecot/imap(client_handle_input+0x11d) 
[0x4164ed] -> dovecot/imap(client_input+0x6f) [0x41685f] -> 
/usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x3713278a56] 
-> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0xa7) 
[0x3713279b27] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) 
[0x37132789c8]
Feb  5 09:33:25 buzon1 dovecot: imap(user at domain): Fatal: master: 
service(imap): child 2057 killed with signal 6 (core dumped)

My solution was to downgrade dovecot and dovecot-pigeonhole back to 
2.2.10-8.el7.x86_64
Regards

-- 

*Gonzalo Palacios Goicolea*
U.T de Infraestructura de Equipos Centrales
/Tecnologías de la Información/
Universidad Autónoma de Madrid • Campus de Cantoblanco
<3D"http://www.uam.es/">

Antes de imprimir este correo piense si es necesario.Cuidemos el 
medioambiente.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://dovecot.org/pipermail/dovecot/attachments/20190215/47dedad2/attachment-0001.html>