- dovecot - dovecot.org

doveadm-server uses 100% cpu and never finishes, dsync backup also never finishes
by Patrick van Zweden 09 Dec '16

09 Dec '16

Hi Guys, I have a quite interesting problem. I have two servers that replicate to each-other. Suddenly one of the accounts is starting to eat a very large amount of disk space on, what basically is the replica for this account. doveadm-server seems to hang and keeps on filling the disk. I came across this message in the logs: Dec 8 07:36:40 <server> dovecot: dsync-server(<account>): Fatal: pool_system_malloc(6291528): Out of memory Dec 8 07:36:40 <server> dovecot: dsync-server(<account>): Fatal: master: service(doveadm): child 883 returned error 83 (Out of memory (service doveadm { vsz_limit=256 MB }, you may need to increase it) - set DEBUG_OUTOFMEM=1 environment to get core dump) I’ve increased the vsz_limit to 5G so it has stopped complaining. Only… the process never seems to finish. I’ve tried to export the mailbox using dsync: dsync -Dv backup -u 'xxxxx' sdbox:/tmp/patrick_export/ However this process never finishes either. It looks like it stays in one of the mailboxes and keeps on trying to copy messages. Format of the mailbox is sdbox Version of dovecot on all servers: 2.2.9 Any pointers for solving this problem? Kind regards, Patrick -- Patrick van Zweden

1 1

doveadm
by Eric Broch 09 Dec '16

09 Dec '16

Hello list, I'm interested in setting up dovecot replication but when I run the command doveadm user '*', which is necessary as indicated here: http://wiki2.dovecot.org/Replication, I get the following output: Error: User listing returned failure Fatal: user listing failed It also indicates the following on the wiki2: "Make sure that user listing is configured for your userdb, this is required by replication to find the list of users that are periodically replicated" I use a vpopmail backup so the question is how do I "Make sure that user listing is configured for...userdb"? This is my userdb settings for vpopmail: userdb { args = cache_key=%u quota_template=quota_rule=*:backend=%q driver = vpopmail } Thank-you Eric

1 1

Correct user for replication
by John Fawcett 08 Dec '16

08 Dec '16

Hi I've set up replication with the settings below. With these settings, aggregator runs as user dovecot and replicator as user root. Are those the recommended settings or should I change them, in particular for replicator which is running as a privileged user? service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = vmail } unix_listener replication-notify { mode = 0666 user = vmail } } service replicator { unix_listener replicator-doveadm { mode = 0660 group = mail } } service doveadm { inet_listener { port = 999 ssl = yes } } ssl_client_ca_file = /etc/pki/tls/cert.pem doveadm_password = secret plugin { mail_replica = tcps:hostname.domainname.tld:999 } thanks John

1 0

Migrate from dovecot 1.1 to 2.2 Maildir to 2.2 mdbox
by Daniel Betz 08 Dec '16

08 Dec '16

Hello list, im am trying to migrate customer mails via dsync from an old dovecot 1.1 setup to an new installed dovecot 2.2.25 setup with mdbox. Upgrading the old dovecot 1.1 to 2.2 with Maildir isnt an problem and the replication to the new setup works well too. The only problem now is, that the POP3 UIDL are completely wrong on the target setup. Im using for Maildir pop3_uidl_format = %f With mdbox i cant use it anymore. But what do i need for mdbox ? Or is there an way to migrate the POP3 uidl list to the mdbox ? I could live with it, when i were able to mark the mails older 14 days as read. Is there an way to do this on many mailboxes ? I have read in Release News from 2.2.26 that there are some fixes with POP3 UIDL .. what is it in detail and could it help me ? Hop you can point me to the right direction. Regards, Daniel Freundliche Grüße, Daniel Betz System Design Engineer / Senior Systemadministration ___________________________________ domainfactory GmbH Oskar-Messter-Str. 33 85737 Ismaning Germany Telefon: +49 (0)89 / 55266-364 Telefax: +49 (0)89 / 55266-222 E-Mail: dbetz(a)df.eu<mailto:dbetz@df.eu> Internet: www.df.eu<http://www.df.eu/> Registergericht: Amtsgericht München HRB-Nummer 150294, Geschäftsführer: Tobias Mohr, Stephan Wolfram

2 1

Dovecot: Mails flagged as read get flagged as unread
by plataleas 07 Dec '16

07 Dec '16

Hi all We experience some unexpected behavior with dovecot. It happens that emails marked as read get marked as unread (MUA is Thunderbird on port 143). Unfortunately this happens randomly, reproducing this issue is difficult. We could not find any pattern, it happens rarely. We are running dovecot version 2.2.24 on Debian Jessie (backports repository). /root@dovecot01:~# dovecot --version// //2.2.24 (a82c823)/ Our Setup: /ha-proxy01 -> dovecot-proxy01 (proxy=yes) -> dovecot01 (proxy=no) * IP is used for multiple services (ports). We use the haproxy to send traffic on different ports to different servers. / Below the configuration (IMAPC is active for migration purposes). Do you have any recommendations where we could focus for troubleshooting? I found some posts about mbox_lazy_writes causing similar issues. However we use mdbox in our setup. If we enable rawlog for what log patterns should we search? Your Inputs are highly appreciated! Thanks and Regards plataleas *----- haproxy configuration ------ *frontend frontend_mail.company.org-143 bind 10.100.189.34:143 mode tcp default_backend backend_mail.company.org-143 backend backend_mail.company.org-143 mode tcp balance source server dovecot-proxy01.company.org dovecot-proxy01.company.org:10143 check send-proxy-v2 server dovecot-proxy02.company.org dovecot-proxy02.company.org:10143 check send-proxy-v2 backup *----- config Dovecot Proxy -----* /root@dovecot-proxy01:~# dovecot -n # 2.2.24 (a82c823): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.13 (7b14904) # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.5 auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@* default_client_limit = 5000 default_process_limit = 500 disable_plaintext_auth = no haproxy_trusted_networks = 10.10.189.28,10.10.189.29 imap_capability = IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS listen = 10.10.189.25 mail_location = mdbox:%h/mdbox mail_max_userip_connections = 0 mail_plugins = zlib mdbox_rotate_size = 10 M namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX. separator = . } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { gzip_save_level = 6 pop3_migration_mailbox = POP3-MIGRATION-NS/INBOX sieve = file:~/sieve;active=~/.dovecot.sieve zlib_save = gz } protocols = imap pop3 lmtp service imap-login { inet_listener imap_haproxy { address = 10.10.189.25 haproxy = yes port = 10143 } inet_listener imaps_haproxy { address = 10.10.189.25 haproxy = yes port = 10993 ssl = yes } } service lmtp { inet_listener lmtp { address = 10.10.189.25 port = 24 } } service pop3-login { inet_listener pop3_haproxy { address = 10.10.189.25 haproxy = yes port = 10110 } inet_listener pop3s_haproxy { address = 10.10.189.25 haproxy = yes port = 10995 ssl = yes } } ssl_ca = </etc/pki/tls/certs/wildcard_company.org-bundle.pem ssl_cert = </etc/pki/tls/certs/wildcard_company.org-cert.pem ssl_key = </etc/pki/tls/private/wildcard_company.org.key userdb { args = /etc/dovecot/dovecot-ldap.conf.ext default_fields = home=/home/virtual/%u driver = ldap } verbose_proctitle = yes protocol imap { mail_plugins = zlib imap_zlib } / /root@dovecot-proxy01:~# grep -v '^ *$#.*$\?$' /etc/dovecot/dovecot-ldap.conf.ext// //uris = ldaps://ldap.comany.org// //dn = 'cn=ldap-reader,ou=customers,dc=company,dc=org'// //dnpass = upser_password// //sasl_bind = no// //tls = no// //auth_bind = yes// //base = ou=customers,dc=company,dc=org// //user_attrs = =home=/data/vmails/%Ld/%Ln,=uid=10000,=gid=10000,mailquota=quota_rule=*:storage=%$G// //user_filter = (&(uid=%u)(pfAccountStatus=active))// //pass_attrs = uid=user,userPassword=password,pfmailHost=host,=proxy_maybe=yes// //pass_filter = (&(uid=%u)(pfAccountStatus=active))// / *-------------config Dovecot LDA---------* * * /root@dovecot01:~# dovecot -n / # 2.2.24 (a82c823): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.13 (7b14904) # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.5 auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@* default_client_limit = 5000 default_process_limit = 500 disable_plaintext_auth = no haproxy_trusted_networks = 10.10.189.28,10.10.189.29 imapc_features = rfc822.size imapc_host = mail-direct.company.org imapc_list_prefix = INBOX imapc_user = %u listen = 10.10.189.42 mail_location = mdbox:%h/mdbox mail_max_userip_connections = 0 mail_plugins = zlib mail_prefetch_count = 20 mdbox_rotate_size = 10 M namespace { hidden = yes list = no location = pop3c: prefix = POP3-MIGRATION-NS/ } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX. separator = . } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { gzip_save_level = 6 pop3_migration_ignore_missing_uidls = yes pop3_migration_mailbox = POP3-MIGRATION-NS/INBOX sieve = file:~/sieve;active=~/.dovecot.sieve zlib_save = gz } pop3c_host = mail-direct.company.org pop3c_password = # hidden, use -P to show it protocols = imap pop3 lmtp service imap-login { inet_listener imap_haproxy { address = 10.10.189.42 haproxy = yes port = 10143 } inet_listener imaps_haproxy { address = 10.10.189.42 haproxy = yes port = 10993 ssl = yes } } service lmtp { inet_listener lmtp { address = 10.10.189.42 port = 24 } } service pop3-login { inet_listener pop3_haproxy { address = 10.10.189.42 haproxy = yes port = 10110 } inet_listener pop3s_haproxy { address = 10.10.189.42 haproxy = yes port = 10995 ssl = yes } } ssl_ca = </etc/ssl/certs/certs/wildcard_company.org-bundle.pem ssl_cert = </etc/ssl/certs/wildcard.company.org.crt ssl_key = </etc/ssl/private/wildcard.company.org.key userdb { args = /etc/dovecot/dovecot-ldap.conf.ext default_fields = home=/home/virtual/%u driver = ldap } verbose_proctitle = yes protocol imap { mail_plugins = zlib imap_zlib } protocol doveadm { mail_plugins = zlib pop3_migration }

4 5

Dovecot migration via dsync : mail_location problem (INBOX) ?
by Cédric ML 07 Dec '16

07 Dec '16

Hello, I'm trying to migrate mail accounts from an old dovecot 1.0.15 to a new dovecot 2.2.26. I've set up a master-user on old dovecot, and I run dsync on new server, like this : # doveadm -D -o mail_fsync=never sync -R -u user(a)domain.tld imapc: my user(a)domain.tld looks like this (on the new server): # doveadm user user(a)domain.tld field value uid 2000 gid 2000 home /home/vmail/domain.tld/user mail maildir:/home/vmail/domain.tld/user/mail quota_rule *:bytes=0 User mail folder on the old server looks like this : # ls -la /home/virtual/user(a)domain.tld/ total 64 drwx------ 11 vmail vmail 4096 2016-12-06 16:26 ./ drwxrwx--- 234 vmail vmail 12288 2016-12-05 12:57 ../ drwx------ 2 vmail vmail 89 2016-12-06 15:29 cur/ -rw------- 1 vmail vmail 352 2016-12-06 16:26 dovecot.index -rw------- 1 vmail vmail 19456 2016-12-06 16:26 dovecot.index.cache -rw------- 1 vmail vmail 2220 2016-12-06 16:26 dovecot.index.log -rw------- 1 vmail vmail 10 2016-12-06 15:14 dovecot-keywords -rw------- 1 vmail vmail 170 2016-12-06 16:21 dovecot-uidlist drwx------ 5 vmail vmail 148 2016-12-05 16:52 .Drafts/ drwx------ 5 vmail vmail 148 2016-12-05 16:52 .FauxIndesirable/ drwx------ 5 vmail vmail 148 2016-12-05 16:52 .Indesirable/ -rw-r----- 1 vmail vmail 293 2016-12-06 15:29 maildirsize drwx------ 2 vmail vmail 85 2016-12-06 15:29 new/ drwx------ 5 vmail vmail 148 2016-12-05 16:52 .Sent/ drwx------ 5 vmail vmail 148 2016-12-05 16:52 .Spam/ drwx------ 2 vmail vmail 6 2016-12-07 10:46 tmp/ drwx------ 5 vmail vmail 4096 2016-12-06 15:36 .Trash/ When I run the sync, with "doveadm -D -o mail_fsync=never sync -R -u user(a)domain.tld imapc:" on the new server, the new server synchronises the mail from the new server, and the file system on the new server looks like this : root@dovecot-pa3:/home/vmail/domain.tld# ls -la /home/vmail/domain.tld/user/mail/ total 84 drwx------ 15 mailbox mailbox 4096 Dec 7 11:02 . drwx------ 3 mailbox mailbox 4096 Dec 7 11:02 .. drwx------ 2 mailbox mailbox 4096 Dec 7 11:02 cur -rw------- 1 mailbox mailbox 556 Dec 7 11:02 dovecot.index.log -rw------- 1 mailbox mailbox 96 Dec 7 11:02 dovecot.mailbox.log -rw------- 1 mailbox mailbox 51 Dec 7 11:02 dovecot-uidlist -rw------- 1 mailbox mailbox 8 Dec 7 11:02 dovecot-uidvalidity -r--r--r-- 1 mailbox mailbox 0 Dec 7 11:02 dovecot-uidvalidity.5847de3b drwx------ 5 mailbox mailbox 4096 Dec 7 11:02 .Drafts drwx------ 5 mailbox mailbox 4096 Dec 7 11:02 .INBOX.Drafts drwx------ 5 mailbox mailbox 4096 Dec 7 11:02 .INBOX.FauxIndesirable drwx------ 5 mailbox mailbox 4096 Dec 7 11:02 .INBOX.Indesirable drwx------ 5 mailbox mailbox 4096 Dec 7 11:02 .INBOX.Sent drwx------ 5 mailbox mailbox 4096 Dec 7 11:02 .INBOX.Spam drwx------ 5 mailbox mailbox 4096 Dec 7 11:02 .INBOX.Trash drwx------ 5 mailbox mailbox 4096 Dec 7 11:02 .Junk -rw------- 1 mailbox mailbox 17 Dec 7 11:02 maildirsize drwx------ 2 mailbox mailbox 4096 Dec 7 11:02 new drwx------ 5 mailbox mailbox 4096 Dec 7 11:02 .Sent -rw------- 1 mailbox mailbox 23 Dec 7 11:02 subscriptions drwx------ 2 mailbox mailbox 4096 Dec 7 11:02 tmp drwx------ 5 mailbox mailbox 4096 Dec 7 11:02 .Trash But dovecot is condigured to have ".Trash", ".Junk" etc... folders, not ".INBOX.Trash" I've try multiple things : - use namespaces for compatibility and add "-n <namespace>" in sync command, without success - modify the mail_location for the user, tried to add LAYOUT=fs, INBOX=<folder>, ... without success. Can anybody help me with that please ? Here's the #doveconf -n output : ------------------------------------------------------- # 2.2.devel (f7f4b65): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.devel (53a3582) # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.6 auth_mechanisms = plain login doveadm_password = # hidden, use -P to show it doveadm_port = 12345 imapc_features = rfc822.size fetch-headers imapc_host = oldserver.domain.tld imapc_master_user = masteruser imapc_password = # hidden, use -P to show it imapc_user = %u mail_plugins = " quota notify replication" mail_prefetch_count = 20 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mbox_write_locks = fcntl dotlock namespace compat { hidden = no list = no location = maildir:~/mail/ prefix = INBOX separator = / } namespace inbox { hidden = no inbox = yes list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { mail_replica = tcp:192.168.159.9:12345 quota = maildir:User quota quota_grace = 10%% quota_rule = Trash:storage=+100M quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Boite de messagerie pleine. quota_status_success = DUNNO quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=85%% quota-warning 85 %u sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /var/lib/dovecot/sieve/ sieve_before2 = /home/vmail/%d/%n/sieve/Vacation.sieve sieve_default = /var/lib/dovecot/sieve/dovecot.sieve sieve_default_name = default sieve_global = /var/lib/dovecot/sieve/ } protocols = " imap sieve pop3 sieve" service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = mailbox } unix_listener replication-notify { mode = 0666 user = mailbox } } service auth { inet_listener { port = 12346 } unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = mailbox user = mailbox } } service config { unix_listener config { user = mailbox } } service doveadm { inet_listener { port = 12345 } user = mailbox } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 } } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 1 service_count = 1 vsz_limit = 64 M } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = mailbox } user = mailbox } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { group = mailbox mode = 0660 } } ssl = required ssl_cert = </etc/ssl/certs/crt-and-intermediate.pem ssl_key = # hidden, use -P to show it userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { postmaster_address = postmaster(a)domain.tld } protocol lda { mail_plugins = " quota notify replication sieve" postmaster_address = postmaster(a)domain.tld } protocol imap { mail_plugins = " quota notify replication imap_quota" } ------------------------------------------------------- Best regards, Cédric

2 2

Re: v2.2.27 released --- libressl
by Ruga 07 Dec '16

07 Dec '16

>openssl version Libressl 2.4.4 Patch for dovecot: perl -i -ple 's/^(#if OPENSSL_VERSION_NUMBER < 0x10100000L\s*)$/$1 || defined (LIBRESSL_VERSION_NUMBER)/' ./src/lib-dcrypt/dcrypt-openssl.c; perl -i -ple 's/^(#if OPENSSL_VERSION_NUMBER < 0x10100000L\s*)$/$1 || defined (LIBRESSL_VERSION_NUMBER)/' ./src/lib-ssl-iostream/dovecot-openssl-common.c; perl -i -ple 's/^(#if OPENSSL_VERSION_NUMBER >= 0x10100000L\s*)$/$1 && !defined (LIBRESSL_VERSION_NUMBER)/' ./src/lib-dcrypt/dcrypt-openssl.c; perl -i -ple 's/^(#if OPENSSL_VERSION_NUMBER >= 0x10100000L\s*)$/$1 && !defined (LIBRESSL_VERSION_NUMBER)/' ./src/lib-ssl-iostream/dovecot-openssl-common.c; Both configuration and compilation are OK. The test fails as follows: [...] test_load_v1_public_key .............................................. : ok Panic: file randgen.c: line 21 (random_fill): assertion failed: (init_refcount > 0) Error: Raw backtrace: 2 libdcrypt_openssl.so 0x0000000103413d24 default_fatal_finish + 36 -> 3 libdcrypt_openssl.so 0x0000000103413afd default_fatal_handler + 61 -> 4 libdcrypt_openssl.so 0x0000000103414069 i_panic + 169 -> 5 libdcrypt_openssl.so 0x000000010344110c random_fill + 220 -> 6 libdcrypt_openssl.so 0x000000010340a63d dcrypt_openssl_store_private_key + 1037 -> 7 test-crypto 0x0000000103387f54 test_load_v2_key + 580 -> 8 test-crypto 0x000000010338990e test_run + 142 -> 9 test-crypto 0x0000000103386921 main + 81 -> 10 libdyld.dylib 0x00007fff9da95255 start + 1 /bin/sh: line 1: 56954 Abort trap: 6 ./$bin make[2]: *** [check-test] Error 1 make[1]: *** [check-recursive] Error 1 make: *** [check-recursive] Error 1 -------- Original Message -------- Subject: v2.2.27 released Local Time: 3 December 2016 6:48 PM UTC Time: 3 December 2016 17:48 From: tss(a)iki.fi To: dovecot-news(a)dovecot.org, Dovecot Mailing List <dovecot(a)dovecot.org> https://dovecot.org/releases/2.2/dovecot-2.2.27.tar.gz https://dovecot.org/releases/2.2/dovecot-2.2.27.tar.gz.sig Note that the download URLs are now https with a certificate from Let's Encrypt. * dovecot.list.index.log rotation sizes/times were changed so that the .log file stays smaller and .log.2 is deleted sooner. + Added mail_crypt plugin that allows encryption of stored emails. See http://wiki2.dovecot.org/Plugins/MailCrypt + stats: Global stats can be sent to Carbon server by setting stats_carbon_server=ip:port + imap/pop3 proxy: If passdb returns proxy_not_trusted, don't send ID/XCLIENT + Added generic hash modifier for %variables: %{<hash algorithm>;rounds=<n>,truncate=<bits>,salt=s>:field} Hash algorithm is any of the supported ones, e.g. md5, sha1, sha256. Also "pkcs5" is supported using SHA256. For example: %{sha256:user} or %{md5;truncate=32:user}. + Added support for SHA3-256 and SHA3-512 hashes. + config: Support DNS wildcards in local_name, e.g. local_name *.example.com { .. } matches anything.example.com, but not multiple.anything.example.com. + config: Support multiple names in local_name, e.g. local_name "1.example.com 2.example.com" { .. } - Fixed crash in auth process when auth-policy was configured and authentication was aborted/failed without a username set. - director: If two users had different tags but the same hash, the users may have been redirected to the wrong tag's hosts. - Index files may have been thought incorrectly lost, causing "Missing middle file seq=.." to be logged and index rebuild. This happened more easily with IMAP hibernation enabled. - Various fixes to restoring state correctly in un-hibernation. - dovecot.index files were commonly 4 bytes per email too large. This is because 3 bytes per email were being wasted that could have been used for IMAP keywords. - Various fixes to handle dovecot.list.index corruption better. - lib-fts: Fixed assert-crash in address tokenizer with specific input. - Fixed assert-crash in HTML to text parsing with specific input (e.g. for FTS indexing or snippet generation) - doveadm sync -1: Fixed handling mailbox GUID conflicts. - sdbox, mdbox: Perform full index rebuild if corruption is detected inside lib-index, which runs index fsck. - quota: Don't skip quota checks when moving mails between different quota roots. - search: Multiple sequence sets or UID sets in search parameters weren't handled correctly. They were incorrectly merged together.

2 2

Segmentation fault in imap_bodystructure_is_plain_7bit
by Mihai Craiu 06 Dec '16

06 Dec '16

Hi, I have a lot of errors like this in my log: Fatal: master: service(imap): child 26049 killed with signal 11 (core dumped) Dovecot 2.2.18 build from sources ./configure --prefix=/opt/dovecot2 --with-mysql --with-sqlite --with-solr --with-ssl --disable-rpath --disable-static. Debian Wheezy 3.2.63-2 x86_64. Filesystem is ZFS. All the core files are similar: $ gdb /opt/dovecot2/libexec/dovecot/imap core GNU gdb (GDB) 7.4.1-debian Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html > This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /opt/dovecot2/libexec/dovecot/imap...done. [New LWP 11635] warning: Can't read pathname for load map: Input/output error. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/opt/dovecot2/libexec/dovecot/imap imap-postlogin'. Program terminated with signal 11, Segmentation fault. #0 imap_bodystructure_is_plain_7bit (part=part@entry=0x234cd50) at imap-bodystructure.c:458 458 if (data->content_subtype != NULL && (gdb) bt #0 imap_bodystructure_is_plain_7bit (part=part@entry=0x234cd50) at imap-bodystructure.c:458 #1 0x00007f920a0fbfaf in index_mail_body_parsed_cache_flags (mail=0x234be60) at index-mail.c:587 #2 index_mail_parse_body_finish (mail=mail@entry=0x234be60, field=field@entry=MAIL_CACHE_IMAP_BODYSTRUCTURE, success=success@entry=true) at index-mail.c:1007 #3 0x00007f920a0fc369 in index_mail_parse_body (mail=mail@entry=0x234be60, field=field@entry=MAIL_CACHE_IMAP_BODYSTRUCTURE) at index-mail.c:1081 #4 0x00007f920a0fc498 in index_mail_parse_bodystructure (mail=mail@entry=0x234be60, field=field@entry=MAIL_CACHE_IMAP_BODYSTRUCTURE) at index-mail.c:1222 #5 0x00007f920a0fcf1c in index_mail_get_special (_mail=_mail@entry=0x234be60, field=field@entry=MAIL_FETCH_IMAP_BODYSTRUCTURE, value_r=value_r@entry=0x7ffc1518fde8) at index-mail.c:1379 #6 0x00007f920a0b29cd in dbox_mail_get_special (_mail=_mail@entry=0x234be60, field=field@entry=MAIL_FETCH_IMAP_BODYSTRUCTURE, value_r=value_r@entry=0x7ffc1518fde8) at dbox-mail.c:229 #7 0x00007f920a0a83f0 in mdbox_mail_get_special (_mail=0x234be60, field=MAIL_FETCH_IMAP_BODYSTRUCTURE, value_r=0x7ffc1518fde8) at mdbox-mail.c:213 #8 0x00007f920a08f96d in mail_get_special (mail=<optimized out>, field=field@entry=MAIL_FETCH_IMAP_BODYSTRUCTURE, value_r=value_r@entry=0x7ffc1518fde8) at mail.c:317 #9 0x0000000000419a6a in fetch_bodystructure (ctx=0x232bbd8, mail=<optimized out>, context=<optimized out>) at imap-fetch.c:690 #10 0x0000000000419f9f in imap_fetch_more_int (ctx=ctx@entry=0x232bbd8, cancel=false) at imap-fetch.c:506 #11 0x000000000041af07 in imap_fetch_more (ctx=0x232bbd8, cmd=cmd@entry=0x232ba00) at imap-fetch.c:558 #12 0x000000000040f0c9 in cmd_fetch (cmd=0x232ba00) at cmd-fetch.c:286 #13 0x0000000000418aec in command_exec (cmd=cmd@entry=0x232ba00) at imap-commands.c:167 #14 0x0000000000417ad0 in client_command_input (cmd=cmd@entry=0x232ba00) at imap-client.c:814 #15 0x0000000000417b64 in client_command_input (cmd=0x232ba00) at imap-client.c:874 #16 0x0000000000417e45 in client_handle_next_command (remove_io_r=<synthetic pointer>, client=0x232ade0) at imap-client.c:912 #17 client_handle_input (client=client@entry=0x232ade0) at imap-client.c:924 #18 0x0000000000418212 in client_input (client=0x232ade0) at imap-client.c:966 #19 0x00007f9209df134b in io_loop_call_io (io=0x232b8f0) at ioloop.c:501 #20 0x00007f9209df1e5b in io_loop_handler_run_internal (ioloop=ioloop@entry=0x22ec790) at ioloop-epoll.c:220 #21 0x00007f9209df13d9 in io_loop_handler_run (ioloop=ioloop@entry=0x22ec790) at ioloop.c:548 #22 0x00007f9209df1458 in io_loop_run (ioloop=0x22ec790) at ioloop.c:525 #23 0x00007f9209d92993 in master_service_run (service=0x22ec620, callback=callback@entry=0x421140 <client_connected>) at master-service.c:581 #24 0x000000000040c60b in main (argc=2, argv=0x22ec390) at main.c:431 (gdb) q Thanks.

1 0

Panic: file dsync-mailbox-tree-sync.c: line 576 (node_mailbox_trees_cmp): assertion failed: (ret != 0)
by Gilles Chauvin 06 Dec '16

06 Dec '16

Hi, Here is a crash that's happening using the latest Dovecot version (v2.2.27 on CentOS7 x86_64): We are using replication. Judging by the second server's logs, I believe this has something to do with the fact that we're using the lazy_expunge plugin. Every night after midnight, we purge the lazy_expunge namespace by running a command similar to following one on the main server (server01 in the logs below): # doveadm expunge -A MAILBOX 'EXPUNGED/*' SAVEDBEFORE "$(date +'%F')" The lazy_expunge namespace is configured as follows: namespace expunged { disabled = no hidden = yes ignore_on_failure = no inbox = no list = no location = sdbox:/srv/mail/expunged/%1n/%n:SUBSCRIPTIONS=subscriptions-expunged order = 0 prefix = EXPUNGED/ separator = / subscriptions = no type = private } plugin { lazy_expunge = EXPUNGED/ } First server's logs: 2016-12-06T13:09:31.212620+01:00 server01 dovecot: dsync-server(user01): Panic: file dsync-mailbox-tree-sync.c: line 576 (node_mailbox_trees_cmp): assertion failed: (ret != 0) 2016-12-06T13:09:31.213473+01:00 server01 dovecot: dsync-server(user01): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x92d70) [0x7f57cc18ad70] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x92e4e) [0x7f57cc18ae4e] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f57cc1264e0] -> dovecot/doveadm-server [10.0.0.2 user01 recv_mailbox_tree_deletes]() [0x447584] -> dovecot/doveadm-server [10.0.0.2 user01 recv_mailbox_tree_deletes]() [0x447a57] -> dovecot/doveadm-server [10.0.0.2 user01 recv_mailbox_tree_deletes]() [0x447849] -> dovecot/doveadm-server [10.0.0.2 user01 recv_mailbox_tree_deletes]() [0x447849] -> dovecot/doveadm-server [10.0.0.2 user01 recv_mailbox_tree_deletes](dsync_mailbox_trees_sync_init+0x1f9) [0x447cc9] -> dovecot/doveadm-server [10.0.0.2 user01 recv_mailbox_tree_deletes](dsync_brain_recv_mailbox_tree_deletes+0x207) [0x43a477] -> dovecot/doveadm-server [10.0.0.2 user01 recv_mailbox_tree_deletes](dsync_brain_run+0x5fe) [0x436e4e] -> dovecot/doveadm-server [10.0.0.2 user01 recv_mailbox_tree_deletes]() [0x437280] -> dovecot/doveadm-server [10.0.0.2 user01 recv_mailbox_tree_deletes]() [0x44c5af] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x52) [0x7f57cc19e4d2] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xe7) [0x7f57cc19fa07] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x3c) [0x7f57cc19e56c] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f57cc19e728] -> dovecot/doveadm-server [10.0.0.2 user01 recv_mailbox_tree_deletes]() [0x41faae] -> dovecot/doveadm-server [10.0.0.2 user01 recv_mailbox_tree_deletes]() [0x421316] -> dovecot/doveadm-server [10.0.0.2 user01 recv_mailbox_tree_deletes]() [0x433b64] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x52) [0x7f57cc19e4d2] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xe7) [0x7f57cc19fa07] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x3c) [0x7f57cc19e56c] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f57cc19e728] -> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f57cc12c513] -> dovecot/doveadm-server [10.0.0.2 user01 recv_mailbox_tree_deletes](main+0x197) [0x413b57] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f57cbd57b15] -> dovecot/doveadm-server [10.0.0.2 user01 recv_mailbox_tree_deletes]() [0x413bf5] 2016-12-06T13:09:31.214076+01:00 server01 dovecot: dsync-server(user01): Fatal: master: service(doveadm): child 6837 killed with signal 6 (core dumps disabled) 2016-12-06T13:11:42.880516+01:00 server01 dovecot: dsync-local(user01): Error: read(server02.localdomain) failed: EOF (last sent=mailbox_delete, last recv=mailbox_tree_node) Second server's logs: 2016-12-06T12:09:36.220591+01:00 server02 dovecot: doveadm(user01): Error: Duplicate mailbox GUID eab35824e49b4658840001001776124f for mailboxes EXPUNGED/Trash/C2i - Coordination-temp-ac1fea3af7e1f2c29ce1c62442a06272/Demissions C2i and EXPUNGED/Trash/C2i - Coordination/Demissions C2i - giving a new GUID c4ffaf08709c465813ae0000a26dfa6e to EXPUNGED/Trash/C2i - Coordination-temp-ac1fea3af7e1f2c29ce1c62442a06272/Demissions C2i 2016-12-06T12:09:36.220859+01:00 server02 dovecot: doveadm(user01): Error: Duplicate mailbox GUID 2ab10a2bee9b4658840001001776124f for mailboxes EXPUNGED/Trash/C2i - Coordination-temp-ac1fea3af7e1f2c29ce1c62442a06272/2014-2015 and EXPUNGED/Trash/C2i - Coordination/2014-2015 - giving a new GUID c6ffaf08709c465813ae0000a26dfa6e to EXPUNGED/Trash/C2i - Coordination-temp-ac1fea3af7e1f2c29ce1c62442a06272/2014-2015 2016-12-06T12:09:36.221091+01:00 server02 dovecot: doveadm(user01): Error: Duplicate mailbox GUID eeb35824e49b4658840001001776124f for mailboxes EXPUNGED/Trash/C2i - Coordination-temp-ac1fea3af7e1f2c29ce1c62442a06272/2012-2013 and EXPUNGED/Trash/C2i - Coordination/2012-2013 - giving a new GUID c8ffaf08709c465813ae0000a26dfa6e to EXPUNGED/Trash/C2i - Coordination-temp-ac1fea3af7e1f2c29ce1c62442a06272/2012-2013 2016-12-06T12:09:36.284224+01:00 server02 dovecot: dsync-local(user01): Error: Mailbox EXPUNGED/Trash/C2i - Coordination/2008-2009/Formation FC sync: mailbox_rename failed: Can't rename mailboxes to/from expunge namespace. 2016-12-06T12:09:45.469706+01:00 server02 dovecot: dsync-server(user01): Error: Duplicate mailbox GUID eab35824e49b4658840001001776124f for mailboxes EXPUNGED/Trash/C2i - Coordination-temp-ac1fea3af7e1f2c29ce1c62442a06272/Demissions C2i and EXPUNGED/Trash/C2i - Coordination/Demissions C2i - giving a new GUID 671ae30f799c465882b00000a26dfa6e to EXPUNGED/Trash/C2i - Coordination-temp-ac1fea3af7e1f2c29ce1c62442a06272/Demissions C2i 2016-12-06T12:09:45.470389+01:00 server02 dovecot: dsync-server(user01): Error: Duplicate mailbox GUID 2ab10a2bee9b4658840001001776124f for mailboxes EXPUNGED/Trash/C2i - Coordination-temp-ac1fea3af7e1f2c29ce1c62442a06272/2014-2015 and EXPUNGED/Trash/C2i - Coordination/2014-2015 - giving a new GUID 691ae30f799c465882b00000a26dfa6e to EXPUNGED/Trash/C2i - Coordination-temp-ac1fea3af7e1f2c29ce1c62442a06272/2014-2015 2016-12-06T12:09:45.471214+01:00 server02 dovecot: dsync-server(user01): Error: Duplicate mailbox GUID eeb35824e49b4658840001001776124f for mailboxes EXPUNGED/Trash/C2i - Coordination-temp-ac1fea3af7e1f2c29ce1c62442a06272/2012-2013 and EXPUNGED/Trash/C2i - Coordination/2012-2013 - giving a new GUID 6b1ae30f799c465882b00000a26dfa6e to EXPUNGED/Trash/C2i - Coordination-temp-ac1fea3af7e1f2c29ce1c62442a06272/2012-2013 2016-12-06T12:09:45.485663+01:00 server02 dovecot: dsync-server(user01): Error: Mailbox EXPUNGED/Trash/C2i - Coordination/2008-2009/Formation FC sync: mailbox_rename failed: Can't rename mailboxes to/from expunge namespace. [... Previous line repeated numerous times ...] 2016-12-06T12:14:32.583382+01:00 server02 dovecot: dsync-server(user01): Error: Mailbox EXPUNGED/Trash/C2i - Coordination/2008-2009/Formation FC sync: mailbox_rename failed: Can't rename mailboxes to/from expunge namespace. 2016-12-06T12:14:36.778278+01:00 server02 dovecot: dsync-local(user01): Error: Mailbox EXPUNGED/Trash/C2i - Coordination/2008-2009/Formation FC sync: mailbox_rename failed: Can't rename mailboxes to/from expunge namespace. [... / ...] 2016-12-06T12:19:30.677497+01:00 sauto dovecot: dsync-server(user01): Panic: file dsync-mailbox-tree-sync.c: line 576 (node_mailbox_trees_cmp): assertion failed: (ret != 0) 2016-12-06T12:19:30.678991+01:00 sauto dovecot: dsync-server(user01): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x92d70) [0x7f40f3cd1d70] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x92e4e) [0x7f40f3cd1e4e] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f40f3c6d4e0] -> dovecot/doveadm-server [10.0.0.1 user01 recv_mailbox_tree_deletes]() [0x447584] -> dovecot/doveadm-server [10.0.0.1 user01 recv_mailbox_tree_deletes]() [0x447a57] -> dovecot/doveadm-server [10.0.0.1 user01 recv_mailbox_tree_deletes]() [0x447849] -> dovecot/doveadm-server [10.0.0.1 user01 recv_mailbox_tree_deletes]() [0x447849] -> dovecot/doveadm-server [10.0.0.1 user01 recv_mailbox_tree_deletes](dsync_mailbox_trees_sync_init+0x1f9) [0x447cc9] -> dovecot/doveadm-server [10.0.0.1 user01 recv_mailbox_tree_deletes](dsync_brain_recv_mailbox_tree_deletes+0x207) [0x43a477] -> dovecot/doveadm-server [10.0.0.1 user01 recv_mailbox_tree_deletes](dsync_brain_run+0x5fe) [0x436e4e] -> dovecot/doveadm-server [10.0.0.1 user01 recv_mailbox_tree_deletes]() [0x437280] -> dovecot/doveadm-server [10.0.0.1 user01 recv_mailbox_tree_deletes]() [0x44c5af] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x52) [0x7f40f3ce54d2] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xe7) [0x7f40f3ce6a07] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x3c) [0x7f40f3ce556c] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f40f3ce5728] -> dovecot/doveadm-server [10.0.0.1 user01 recv_mailbox_tree_deletes]() [0x41faae] -> dovecot/doveadm-server [10.0.0.1 user01 recv_mailbox_tree_deletes]() [0x421316] -> dovecot/doveadm-server [10.0.0.1 user01 recv_mailbox_tree_deletes]() [0x433b64] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x52) [0x7f40f3ce54d2] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xe7) [0x7f40f3ce6a07] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x3c) [0x7f40f3ce556c] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f40f3ce5728] -> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f40f3c73513] -> dovecot/doveadm-server [10.0.0.1 user01 recv_mailbox_tree_deletes](main+0x197) [0x413b57] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f40f389eb15] -> dovecot/doveadm-server [10.0.0.1 user01 recv_mailbox_tree_deletes]() [0x413bf5] 2016-12-06T12:19:30.689038+01:00 sauto dovecot: dsync-server(user01): Fatal: master: service(doveadm): child 96318 killed with signal 6 (core dumps disabled) 2016-12-06T12:19:30.749074+01:00 sauto dovecot: dsync-local(user01): Error: read(server01.localdomain) failed: EOF (last sent=mailbox_delete, last recv=mailbox_tree_node) Thanks, Kind regards, Gilles.

1 0

LMTP delivery honours .forward by default?
by MRob 06 Dec '16

06 Dec '16

I was under the impression that Dovecot does not look at .forward by default, since of course, Sieve is where that kind of logic should go. However, I'm seeing a .forward file being honoured by the LMTP delivery agent. Is this to be expected?

2 5