index problems after update

Mon Feb 11 10:12:50 EET 2019

Hello,
Am 08.02.2019 um 09:25 schrieb Hajo Locke via dovecot:
> Hello List,
>
> i have a problem with index-files which is separated in 2 subproblems. 
> May be these problems are connected.
> Currently we use Ubuntu 18.04 LTS which is bundled with dovecot 2.2.33.2
> These servers are fresh installed machines and users are added to the 
> system after, there was no upgrade.
> Sometimes it happens, that dovecot stops showing new mail. There is no 
> error in log, dovecot just seems to do his normal operations but is 
> not delivering new emails from /var/mail/myuser
> emails are delivered again if i delete index-files from .imap folder. 
> So index is recreated and everything works again. I cant reproduce 
> this problem, but every 2 days i have one user who is reporting this 
> problem.
>
> Second problem is similiar. I Upgraded a server from Ubuntu 16.04 to 
> Ubuntu 18.04. This includes an upgrade from dovecot 2.2.22 to 
> 2.2.33.2. After that i installed an self packaged dovecot 2.2.36.1
> Now here is same problem, dovecot is not showing new mails, but the 
> difference is we have a Raw backtrace in Log just as expected:
>
i think this crash is a special problem of dovecot 2.2.36.1.
if i downgrade to 2.2.33.2 all is working well, upgrading again to 
2.2.36.1 leeds to same crash.
> Feb  8 08:45:37 hostname dovecot[14882]: imap(myuser): Error: Raw 
> backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xa1ee2) [0x7f78b3b2cee2] 
> -> /usr/lib/dovecot/libdovecot.so.0(+0xa1fda) [0x7f78b3b2cfda] -> 
> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f78b3abb8a8] -> 
> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_parse_header+0x78a) 
> [0x7f78b3e7940a] -> /usr/lib/dovecot/libdovecot.so.0(+0x83592) 
> [0x7f78b3b0e592] -> 
> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x74) [0x7f78b3b38814] 
> -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) 
> [0x7f78b3b392fd] -> 
> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_get_header_stream+0x4a) 
> [0x7f78b3e7a2ea] -> 
> /usr/lib/dovecot/libdovecot-storage.so.0(mail_get_header_stream+0x4a) 
> [0x7f78b3dfd8da] -> 
> /usr/lib/dovecot/libdovecot-storage.so.0(imap_msgpart_open+0x24a) 
> [0x7f78b3ebb57a] -> dovecot/imap [myuser ip.ip.ip.ip FETCH](+0x2081c) 
> [0x5581ac76d81c] -> dovecot/imap [m038422 ip.ip.ip.ip FETCH](+0x1ea54) 
> [0x5581ac76ba54] -> dovecot/imap [myuser ip.ip.ip.ip 
> FETCH](imap_fetch_more+0x39) [0x5581ac76cd69] -> dovecot/imap [myuser 
> ip.ip.ip.ip FETCH](cmd_fetch+0x31b) [0x5581ac75e1bb] -> dovecot/imap 
> [myuser ip.ip.ip.ip FETCH](command_exec+0x5c) [0x5581ac76a09c] -> 
> dovecot/imap [myuser ip.ip.ip.ip FETCH](+0x1b612) [0x5581ac768612] -> 
> dovecot/imap [myuser ip.ip.ip.ip FETCH](+0x1b6ac) [0x5581ac7686ac] -> 
> dovecot/imap [myuser ip.ip.ip.ip FETCH](client_handle_input+0x18d) 
> [0x5581ac768a6d] -> dovecot/imap [myuser ip.ip.ip.ip 
> FETCH](client_input+0xac) [0x5581ac768fbc] -> 
> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x52) 
> [0x7f78b3b43482] -> 
> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x12e) 
> [0x7f78b3b44b9e] -> 
> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x36) 
> [0x7f78b3b43516] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) 
> [0x7f78b3b436c8] -> 
> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) 
> [0x7f78b3ac6aa3] -> dovecot/imap [myuser ip.ip.ip.ip 
> FETCH](main+0x329) [0x5581ac75b159] -> 
> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) 
> [0x7f78b36bbb97] -> dovecot/imap [myuser ip.ip.ip.ip 
> FETCH](_start+0x2a) [0x5581ac75b2fa]
> Feb  8 08:45:37 hostname dovecot[14882]: imap(myuser): Fatal: master: 
> service(imap): child 14135 killed with signal 6 (core dumped)
>
> Over all dovecot and OS versions we use same old-fashioned setup. 
> Mails are delivered by procmail to classic mbox file.
> This is output of doveconf -n
>
> # 2.2.36.1 (5d621cf65): /etc/dovecot/dovecot.conf
> # Pigeonhole version 0.4.21 (92477967)
> # OS: Linux 4.18.0-14-generic x86_64 Ubuntu 18.04.1 LTS
> # Hostname: myhostname
> auth_mechanisms = plain login
> auth_username_chars = 
> abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@üÜöÖäÄ
> auth_worker_max_count = 60
> disable_plaintext_auth = no
> imap_capability = +XLIST
> lock_method = flock
> login_greeting = mailserver ready.
> login_log_format_elements = user=<%u> method=%m rip=%r lip=%l %c
> mail_cache_min_mail_count = 30
> mail_location = mbox:~/mail:INBOX=/var/mail/%u
> mail_max_userip_connections = 30
> mbox_read_locks = dotlock
> mbox_very_dirty_syncs = yes
> namespace inbox {
>   inbox = yes
>   location =
>   mailbox Archiv {
>     auto = no
>     special_use =
>   }
>   mailbox Drafts {
>     auto = no
>     special_use =
>   }
>   mailbox Junk {
>     auto = no
>     special_use =
>   }
>   mailbox Sent {
>     auto = no
>     special_use =
>   }
>   mailbox Trash {
>     auto = no
>     special_use =
>   }
>   prefix =
> }
> passdb {
>   args = /etc/dovecot/dovecot-sql.conf
>   driver = sql
> }
> protocols = imap pop3
> service auth-worker {
>   idle_kill = 1 mins
> }
> service auth {
>   idle_kill = 1 mins
>   unix_listener /var/spool/postfix/private/auth {
>     group = postfix
>     mode = 0660
>     user = postfix
>   }
>   user = root
> }
> service imap-login {
>   executable = /usr/lib/dovecot/imap-login
>   process_min_avail = 4
>   service_count = 0
>   user = dovecot
> }
> service imap {
>   process_limit = 1024
> }
> service pop3-login {
>   executable = /usr/lib/dovecot/pop3-login
>   process_min_avail = 4
>   service_count = 0
>   user = dovecot
> }
> service pop3 {
>   executable = /usr/lib/dovecot/pop3
>   process_limit = 1024
> }
> shutdown_clients = no
> ssl_ca = </path/cert.pem
> ssl_cert = </path/cert.pem
> ssl_cipher_list = 
> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:!DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA
> ssl_dh_parameters_length = 2048
> ssl_key =  # hidden, use -P to show it
> ssl_prefer_server_ciphers = yes
> userdb {
>   args = /etc/dovecot/dovecot-sql.conf
>   driver = sql
> }
> verbose_proctitle = yes
> protocol imap {
>   imap_client_workarounds = delay-newmail
>   mail_plugins = mail_log notify
>   ssl = yes
> }
> protocol pop3 {
>   pop3_client_workarounds = oe-ns-eoh outlook-no-nuls
>   pop3_no_flag_updates = no
>   pop3_uidl_format = %08Xu%08Xv
>   ssl = yes
> }
>
>
> Whats going wrong here? Main problem is first one. dovecot stops 
> showing new mail during service without telling a reason. reason looks 
> like a broken index file.
> What to do here? Please tell me your opinion.
>
> Thanks,
> Hajo
>
Thanks,
Hajo