June 2015 - dovecot - dovecot.org

Virtual Mailboxes
by Daniel Miller 21 Jun '15

21 Jun '15

Having some trouble with my first attempt at virtual mailboxes. I've enabled the plugin in 10-mail.conf, added a virtual namespace, and created the dovecot-virtual files in the folders. I see the folder names in the list when I refresh the folders available for subscription - but the folders cannot be subscribed to and I can't view their contents. What obvious mistake did I make? doveconf -n: auth_cache_size = 4 k auth_mechanisms = plain login default_login_user = nobody default_vsz_limit = 1 G dict { acl = mysql:/usr/local/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no listen = * mail_attachment_dir = /var/mail/attachments mail_plugins = fts fts_solr acl zlib virtual mail_shared_explicit_inbox = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mdbox_rotate_interval = 1 days mdbox_rotate_size = 32 M namespace { hidden = no inbox = yes list = yes location = prefix = separator = / subscriptions = yes type = private } namespace { list = children location = mdbox:/var/mail/%%d/%%n/mdbox prefix = shared/%%n/ separator = / subscriptions = no type = shared } namespace { location = virtual:/var/mail/%%d/%%n/mdbox/virtual prefix = virtual/ separator = / } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { acl = vfile:/usr/local/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = proxy::acl fts = solr fts_autoindex = yes fts_solr = break-imap-search url=http://solr.amfes.lan:8983/solr/ sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = mail mode = 0600 user = vmail } } service dict { unix_listener dict { group = mail mode = 0660 user = vmail } } service imap-login { process_limit = 200 process_min_avail = 10 service_count = 1 } service lmtp { process_min_avail = 5 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = mail mode = 0666 user = vmail } } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } process_min_avail = 0 service_count = 1 } ssl_cert =

1 0

Dovecot 2.2.16: disappearing messages, mismatched summaries, duplicated messages, excessive full re-downloads
by David Gessel 20 Jun '15

20 Jun '15

My Dovecot 2.2.16 server is interacting poorly with a variety of mail clients. The symptoms include disappearing messages, mismatched summaries, duplicated messages, and excessive full re-downloads. Dovecot 2.2.16 on FreeBSD 10.1-release r280247M ZFS is interacting suboptimally with various email clients. Output of "dovecot -n" further down. *Client Artifacts* Thunderbird 31.6 Win 7 Ultimate SP1: - messages disappear and reappear with checks somewhat randomly. - Sometimes full re-download of folders is triggered, also seems random - Copies of messages appear, sometimes multiplying with each mail check up to about 5-8 or so of each message. - The summary information in the list pane doesn't match the message headers in the preview pane - quit and restart always cleans it up. - sometimes changing folders cleans it up, but not always - sometimes instigating a manual mail check cleans it up but not always Mulberry 4.08 Win 7 Ultimate SP1: - messages disappear and reappear - full re-scans of entire folders triggered occasionally when changing folders, but not always - message list filled with semi-blank entries (message number, status flags, and dates are listed, but no sender, subject, or body is available). The time stamps do not appear to be correct "date received". - message list fills with duplicates - the last of the set of duplicates, which are not necessarily sequentially numbered, has load-able body text and headers, but the others do not. There may be 3-5 copies, only one of which is the real message. The inbox is sequentially numbered by "message number" (the "natural order of the mailbox") and duplicates have unique numbers. - If there's a deleted message left, the "expunge" button is enabled and this also restores normal mailbox view. - disconnecting and reconnecting usually fixes the issues - changing mailboxes while connected sometimes fixes the issues. Claws 3.10.1 Win 7 Ultimate SP1 - initial load is correct, but once claws tries to sync mail, it locks up and never releases (even after days). Claws is weird like that. Sylpheed 3.4.2 Win 7 Ultimate SP1 - mail list always looks fine, but sometimes clicking a message shows "can't display" in the preview pane - "get messages" restores access to messages. Thunderbird 31.5 Linux Mint 17 Qaina 3.13.0-24-generic - messages disappear and reappear with checks somewhat randomly. - Sometimes full redownload of folders is triggered, also seems random - Copies of messages appear, sometimes multiplying with each mail check - The summary information in the list pane doesn't match the message headers in the preview pane - wrong message total (every other client agrees on 2576 but TB-linux says 3242, perhaps counting dups) Claws 3.9.3 Linux Mint 17 Qaina 3.13.0-24-generic - Occasionally shows "this message can't be displayed" - executing a "get mail" fixes the error K-9 5.004 Android 4.3 - messages appear and then disappear on the next check - message body is occasionally blank (only subject and arrival time appear, no body preview line, clicking on the message shows "no text" *Server artifacts:* dovecot.log shows runs of this error: Apr 15 09:02:37 imap(user(a)domain.com): Warning: Maildir /mail/domain.com/user//.Sent: Expunged message reappeared, giving a new UID (old uid=306286, file=1429102024.M826331P1156.host,S=1678,W=1724:2,S) or this error: Apr 15 10:04:00 indexer-worker(user(a)domain.com): Warning: Maildir /mail/domain.com/user//.Sent: Expunged message reappeared, giving a new UID (old uid=306803, file=1428785463.M645546P27215.host,S=5389,W=5442:2,S) # grep -c "Expunged message reappeared" dovecot.log.0 8981 The first entry in dovecot.log.0 is at Apr 15 10:00:12, and the last entry at Apr 15 12:00:13; Dovecot is logging 4,490 expunged message reappeared messages per hour. Note the server has 4 accounts set up and very limited traffic, so 4,490 per hour would seem like a lot of anything to be logged. # doveadm dump -t mailboxlog /mail/domain.com/user #0: rename 000000803de108b60f00000000000000 (1969-12-31 16:00:00) #24: 00000000000000000000000000000000 (1969-12-31 16:00:00) #48: 000000000000000000000000c8272100 (1969-12-31 16:01:04) #72: 00002e61726368697665732e32303031 (2031-02-20 10:50:40) Fatal: rec read() 10 != 24 "Fatal" seems like an error. I do not see much argument with the assertion that 10 != 24. # doveadm dump -t fts-expunge-log /mail/domain.com/user Fatal: Invalid record_size=2147483648 at offset 0 dovecot-debug.log examples follow after # dovecot -n output, which follows "fixes tried:" *Fixes Tried* # doveadm purge -A # doveadm fts rescan -u user(a)domain.com # doveadm -Dv index -u user INBOX # doveadm force-resync -u user INBOX I've set mbox_dirty_syncs = no mbox_lazy_writes = no ...in an attempt to fix the problem, but none did. I tried removing most of my customizations one by one and testing, but the problem has not been improved. I strongly suspect that the index files are getting thrashed or otherwise abused. I have no idea what's triggering whatever event is resulting in the thousands of expunged messages reappearing, but the client artifacts suggest a mismatch between cache or index files and files on disk: UID, keywords, and set date appear as if from cache but the body of the message does not. It seems as if the UID doesn't match a message on disk any more, which soon gets automatically fixed and everything syncs again until something re-triggers - a false expunge perhaps? Or perhaps the transaction log is suffering some sort of ongoing abuse? *dovecot -n* # dovecot -n # 2.2.16: /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.7 # OS: FreeBSD 10.1-RELEASE amd64 auth_default_realm = blackrosetech.com auth_mechanisms = plain login auth_realms = blackrosetech.com eroticanarchy.org debug_log_path = /var/log/dovecot-debug.log deliver_log_format = msgid=%m: %f $p %$ disable_plaintext_auth = no first_valid_gid = 5000 first_valid_uid = 5000 hostname = blackrosetech.com info_log_path = /var/log/dovecot-info.log last_valid_gid = 5000 last_valid_uid = 5000 lda_mailbox_autocreate = yes listen = 10.3.69.135 log_path = /var/log/dovecot.log mail_debug = yes mail_gid = 5000 mail_location = maildir:~:CONTROL=/var/no-quota/%u mail_plugins = quota mail_log notify fts fts_lucene mail_temp_dir = /var/tmp mail_uid = 5000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate spamtest spamtestplus editheader mbox_dirty_syncs = no mbox_lazy_writes = no namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { expire = Trash fts = lucene fts_autoindex = yes fts_lucene = whitespace_chars=@. mime_parts mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir:User quota quota_exceeded_message = Storage quota for this account has been exceeded, please try again later. quota_rule = *:storage=50G quota_rule2 = Trash:storage=+30%% quota_rule3 = Sent:storage=+30%% quota_warning = storage=90%% quota-warning 90 %u quota_warning2 = storage=75%% quota-warning 75 %u sieve = ~/.dovecot.sieve sieve_before = /usr/local/etc/dovecot/sieve/ sieve_dir = ~/sieve sieve_extensions = +spamtest +spamtestplus +relational +comparator-i;ascii-numeric +editheader sieve_global_dir = /usr/local/etc/dovecot/sieve } postmaster_address = gessel(a)blackrosetech.com protocols = imap lmtp sieve sieve sendmail_path = /usr/local/sbin/sendmail service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0666 user = vmail } } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 1 vsz_limit = 128 M } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl = required ssl_cert = </usr/local/etc/ca/shiofuki-cert.pem ssl_key = </usr/local/etc/ca/shiofuki-unencrypted-key.pem ssl_prefer_server_ciphers = yes userdb { driver = prefetch } userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lda { mail_plugins = quota mail_log notify fts fts_lucene sieve } protocol imap { mail_max_userip_connections = 20 mail_plugins = quota mail_log notify fts fts_lucene imap_quota } protocol lmtp { mail_plugins = quota mail_log notify fts fts_lucene sieve } I appreciate any assistance or insight, -David *dovecot-debug.log excerpts* Apr 15 13:17:26 indexer-worker(user(a)domain.com): user-lookup(user(a)domain.com)Debug: auth input: user(a)domain.com home=/mail/domain.com/user/ mail=maildir:~/ uid=5000 gid=5000 quota_rule=*:bytes=0 Apr 15 13:17:26 indexer-worker(user(a)domain.com): user-lookup(user(a)domain.com)Debug: Added userdb setting: mail=maildir:~/ Apr 15 13:17:26 indexer-worker(user(a)domain.com): user-lookup(user(a)domain.com)Debug: Added userdb setting: plugin/quota_rule=*:bytes=0 Apr 15 13:17:26 indexer-worker(user(a)domain.com): Debug: Effective uid=5000, gid=5000, home=/mail/domain.com/user/ Apr 15 13:17:26 indexer-worker(user(a)domain.com): Debug: Quota root: name=User quota backend=maildir args= Apr 15 13:17:26 indexer-worker(user(a)domain.com): Debug: Quota rule: root=User quota mailbox=* bytes=0 messages=0 Apr 15 13:17:26 indexer-worker(user(a)domain.com): Debug: Quota rule: root=User quota mailbox=Trash bytes=0 (30%) messages=0 Apr 15 13:17:26 indexer-worker(user(a)domain.com): Debug: Quota rule: root=User quota mailbox=Sent bytes=0 (30%) messages=0 Apr 15 13:17:26 indexer-worker(user(a)domain.com): Debug: Quota warning: bytes=0 (90%) messages=0 reverse=no command=quota-warning 90 user(a)domain.com Apr 15 13:17:26 indexer-worker(user(a)domain.com): Debug: Quota warning: bytes=0 (75%) messages=0 reverse=no command=quota-warning 75 user(a)domain.com Apr 15 13:17:26 indexer-worker(user(a)domain.com): Debug: Quota grace: root=User quota bytes=0 (10%) Apr 15 13:17:26 indexer-worker(user(a)domain.com): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/ Apr 15 13:17:26 indexer-worker(user(a)domain.com): Debug: maildir++: root=/mail/domain.com/user/, index=, indexpvt=, control=, inbox=/mail/domain.com/user/, alt= Apr 15 13:17:27 indexer-worker(user(a)domain.com): Debug: Ignoring unknown cache field: pop3.order Apr 15 13:17:27 indexer-worker(user(a)domain.com): Debug: Ignoring unknown cache field: binary.parts Apr 15 13:17:27 indexer-worker(user(a)domain.com): Debug: Ignoring unknown cache field: body.snippet Apr 15 13:44:25 lda(user(a)domain.com): Debug: Quota root: name=User quota backend=maildir args= Apr 15 13:44:25 lda(user(a)domain.com): Debug: Quota rule: root=User quota mailbox=* bytes=53687091200 messages=0 Apr 15 13:44:25 lda(user(a)domain.com): Debug: Quota rule: root=User quota mailbox=Trash bytes=+16106127360 (30%) messages=0 Apr 15 13:44:25 lda(user(a)domain.com): Debug: Quota rule: root=User quota mailbox=Sent bytes=+16106127360 (30%) messages=0 Apr 15 13:44:25 lda(user(a)domain.com): Debug: Quota warning: bytes=48318382080 (90%) messages=0 reverse=no command=quota-warning 90 raw mail user Apr 15 13:44:25 lda(user(a)domain.com): Debug: Quota warning: bytes=40265318400 (75%) messages=0 reverse=no command=quota-warning 75 raw mail user Apr 15 13:44:25 lda(user(a)domain.com): Debug: Quota grace: root=User quota bytes=5368709120 (10%) Apr 15 13:44:25 lda(user(a)domain.com): Debug: none: root=, index=, indexpvt=, control=, inbox=, alt= Apr 15 13:49:32 lda(user(a)domain.com): Debug: Quota root: name=User quota backend=maildir args= Apr 15 13:49:32 lda(user(a)domain.com): Debug: Quota rule: root=User quota mailbox=* bytes=53687091200 messages=0 Apr 15 13:49:32 lda(user(a)domain.com): Debug: Quota rule: root=User quota mailbox=Trash bytes=+16106127360 (30%) messages=0 Apr 15 13:49:32 lda(user(a)domain.com): Debug: Quota rule: root=User quota mailbox=Sent bytes=+16106127360 (30%) messages=0 Apr 15 13:49:32 lda(user(a)domain.com): Debug: Quota warning: bytes=48318382080 (90%) messages=0 reverse=no command=quota-warning 90 raw mail user Apr 15 13:49:32 lda(user(a)domain.com): Debug: Quota warning: bytes=40265318400 (75%) messages=0 reverse=no command=quota-warning 75 raw mail user Apr 15 13:49:32 lda(user(a)domain.com): Debug: Quota grace: root=User quota bytes=5368709120 (10%) Apr 15 13:49:32 lda(user(a)domain.com): Debug: none: root=, index=, indexpvt=, control=, inbox=, alt= Apr 15 13:49:41 lda(user(a)domain.com): Debug: sieve: file storage: Using script storage path: /mail/domain.com/user//sieve Apr 15 13:49:41 lda(user(a)domain.com): Debug: sieve: file storage: Relative path to sieve storage in active link: sieve/ Apr 15 13:49:41 lda(user(a)domain.com): Debug: sieve: file script: Opened script `Forwards' from `/mail/domain.com/user/sieve/Forwards.sieve' Apr 15 13:49:41 lda(user(a)domain.com): Debug: sieve: file script: Opened script `File_Server_Msg' from `/mail/domain.com/user/sieve/File_Server_Msg.sieve' Apr 15 13:49:41 lda(user(a)domain.com): Debug: sieve: file script: Opened script `File_SRL' from `/mail/domain.com/user/sieve/File_SRL.sieve' Apr 15 13:49:41 lda(user(a)domain.com): Debug: sieve: file script: Opened script `File_BULK' from `/mail/domain.com/user/sieve/File_BULK.sieve' Apr 15 13:49:41 lda(user(a)domain.com): Debug: sieve: file script: Opened script `File_Commerce' from `/mail/domain.com/user/sieve/File_Commerce.sieve' Apr 15 13:49:41 lda(user(a)domain.com): Debug: sieve: file script: Opened script `File_Impersonal' from `/mail/domain.com/user/sieve/File_Impersonal.sieve' Apr 15 13:49:41 lda(user(a)domain.com): Debug: sieve: file script: Opened script `File_Travel' from `/mail/domain.com/user/sieve/File_Travel.sieve' Apr 15 13:49:41 lda(user(a)domain.com): Debug: sieve: file script: Opened script `File_Tribe' from `/mail/domain.com/user/sieve/File_Tribe.sieve' Apr 15 13:49:41 lda(user(a)domain.com): Debug: sieve: file script: Opened script `File_Info' from `/mail/domain.com/user/sieve/File_Info.sieve' Apr 15 13:49:41 lda(user(a)domain.com): Debug: sieve: file script: Opened script `File_Lists' from `/mail/domain.com/user/sieve/File_Lists.sieve' Apr 15 13:49:41 lda(user(a)domain.com): Debug: sieve: file script: Opened script `File_Trash' from `/mail/domain.com/user/sieve/File_Trash.sieve' Apr 15 13:49:41 lda(user(a)domain.com): Debug: sieve: file script: Opened script `File_Security' from `/mail/domain.com/user/sieve/File_Security.sieve' Apr 15 13:49:41 lda(user(a)domain.com): Debug: sieve: file script: Opened script `File_Notifications' from `/mail/domain.com/user/sieve/File_Notifications.sieve' Apr 15 13:49:41 lda(user(a)domain.com): Debug: sieve: file script: Opened script `File_Spam' from `/mail/domain.com/user/sieve/File_Spam.sieve' Apr 15 13:49:41 lda(user(a)domain.com): Debug: sieve: Script binary /mail/domain.com/user/.dovecot.svbin successfully loaded Apr 15 13:49:41 lda(user(a)domain.com): Debug: sieve: binary save: not saving binary /mail/domain.com/user/.dovecot.svbin, because it is already stored Apr 15 13:49:41 lda(user(a)domain.com): Debug: sieve: Executing script from `/mail/domain.com/user/.dovecot.svbin' Apr 15 13:49:41 lda(user(a)domain.com): Debug: Quota root: name=User quota backend=maildir args= Apr 15 13:49:41 lda(user(a)domain.com): Debug: Quota rule: root=User quota mailbox=* bytes=53687091200 messages=0 Apr 15 13:30:31 imap(user(a)domain.com): Debug: Loading modules from directory: /usr/local/lib/dovecot Apr 15 13:30:31 imap(user(a)domain.com): Debug: Module loaded: /usr/local/lib/dovecot/lib10_quota_plugin.so Apr 15 13:30:31 imap(user(a)domain.com): Debug: Module loaded: /usr/local/lib/dovecot/lib11_imap_quota_plugin.so Apr 15 13:30:31 imap(user(a)domain.com): Debug: Module loaded: /usr/local/lib/dovecot/lib15_notify_plugin.so Apr 15 13:30:31 imap(user(a)domain.com): Debug: Module loaded: /usr/local/lib/dovecot/lib20_fts_plugin.so Apr 15 13:30:31 imap(user(a)domain.com): Debug: Module loaded: /usr/local/lib/dovecot/lib20_mail_log_plugin.so Apr 15 13:30:31 imap(user(a)domain.com): Debug: Module loaded: /usr/local/lib/dovecot/lib21_fts_lucene_plugin.so Apr 15 13:30:31 imap(user(a)domain.com): Debug: Added userdb setting: plugin/quota_rule=*:bytes=0 Apr 15 13:30:31 imap(user(a)domain.com): Debug: Effective uid=5000, gid=5000, home=/mail/domain.com/user/ Apr 15 13:30:31 imap(user(a)domain.com): Debug: Quota root: name=User quota backend=maildir args= Apr 15 13:30:31 imap(user(a)domain.com): Debug: Quota rule: root=User quota mailbox=* bytes=0 messages=0 Apr 15 13:30:31 imap(user(a)domain.com): Debug: Quota rule: root=User quota mailbox=Trash bytes=0 (30%) messages=0 Apr 15 13:30:31 imap(user(a)domain.com): Debug: Quota rule: root=User quota mailbox=Sent bytes=0 (30%) messages=0 Apr 15 13:30:31 imap(user(a)domain.com): Debug: Quota warning: bytes=0 (90%) messages=0 reverse=no command=quota-warning 90 user(a)domain.com Apr 15 13:30:31 imap(user(a)domain.com): Debug: Quota warning: bytes=0 (75%) messages=0 reverse=no command=quota-warning 75 user(a)domain.com Apr 15 13:30:31 imap(user(a)domain.com): Debug: Quota grace: root=User quota bytes=0 (10%) Apr 15 13:30:31 imap(user(a)domain.com): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~:CONTROL=/var/no-quota/user@domain.com Apr 15 13:30:31 imap(user(a)domain.com): Debug: maildir++: root=/mail/domain.com/user/, index=, indexpvt=, control=/var/no-quota/user(a)domain.com, inbox=/mail/domain.com/user/, alt=

7 29

crash dovecot 2.2.18
by Hajo Locke 19 Jun '15

19 Jun '15

Hello, after upgrading from 2.1.17 to 2.2.18 i have a specific Inbox which causes a crash on dovecot: Jun 19 09:28:03 hostname dovecot: imap(username): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x7075f) [0x7f9298c4375f] -> /usr/lib/dovecot/libdovecot.so.0(+0x707be) [0x7f9298c437be] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f9298bf11aa] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x2bd9a) [0x7f9298ee2d9a] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x98f8b) [0x7f9298f4ff8b] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_get_first_header+0xb9) [0x7f9298f502c9] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_get_first_header+0x38) [0x7f9298ee3d18] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xabfa2) [0x7f9298f62fa2] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xac1e2) [0x7f9298f631e2] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x68) [0x7f9298c4d118] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) [0x7f9298c4da9d] -> /usr/lib/dovecot/libdovecot.so.0(message_parse_header_next+0x65) [0x7f9298c33065] -> /usr/lib/dovecot/libdovecot.so.0(message_parse_header+0x38) [0x7f9298c338d8] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_parse_headers+0x80) [0x7f9298f4f940] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x98cca) [0x7f9298f4fcca] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_get_first_header+0xb9) [0x7f9298f502c9] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_get_first_header+0x38) [0x7f9298ee3d18] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9a85a) [0x7f9298f5185a] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_get_date+0x92) [0x7f9298f51972] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_get_date+0x36) [0x7f9298ee3b76] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa3113) [0x7f9298f5a113] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_sort_list_add+0x31) [0x7f9298f5a471] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_search_next_nonblock+0x3d) [0x7f9298f5801d] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonblock+0x27) [0x7f9298eecd67] -> dovecot/imap [username ip.ip.ip.ip UID SORT](+0x1e7f7) [0x7f92994117f7] -> dovecot/imap [username ip.ip.ip.ip UID SORT](imap_search_start+0xf1) [0x7f9299411c41] -> dovecot/imap [username ip.ip.ip.ip UID SORT](cmd_sort+0x25b) [0x7f9299407c4b] -> dovecot/imap [username ip.ip.ip.ip UID SORT](command_exec+0x4d) [0x7f929940c40d] Mailboxformat is mbox. I think reason was a damaged Index in ~username/mail/.imap/INBOX/ I removed this folder and immediately errors are gone and Mailbox was readable again. Never noticed this kind of errors for years on 2.1.17, maybe this helps to find a Bug in Code. Thanks, Hajo

1 0

dsync selectively
by lejeczek 19 Jun '15

19 Jun '15

hi everybody I've barely started reading on dsync and I wonder.. would you know if it is possible to sync/replicate only specific domain(users)? or it's always the whole lot? many thanks P.

4 8

Dovecot 2.1.9 shared folder deletion causes fault
by JT 19 Jun '15

19 Jun '15

Hey all, I'm getting a fault in dovecot (see below) using shared folders. I have a mailbox that receives new mail (Account X/INBOX). Another account (Account A) has access to the other account (Account X/INBOX) with ACL below. The ACL directory (.DEFAULT) contains: user=AccountA lrwst I removed a test email from Account A/INBOX, which is supposed to remove the email from Account X/INBOX. A fault occurs whenever my client (Thunderbird) attempts to connect and complete the deletion. Other actions complete successfully. It only appears to happen when using Thunderbird. Mac Mail does not cause the fault. I'm not getting a core dump so all I have is what the log outputs. I can reproduce this regularly so let me know if you need any additional information. Thanks, JT === Version: 2.1.9 (Ubuntu Trustydeb package) OS: Ubuntu trusty 14.04 2015-06-18T19:17:50.518505+00:00 mail-103 dovecot: imap(XXX): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x5e271) [0x7eff3f6d1271] -> /usr/lib/dovecot/libdovecot.so.0(+0x5e34e) [0x7eff3f6d134e] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7eff3f68ca9e] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa7cbc) [0x7eff3f9e2cbc] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_copy+0x6c) [0x7eff3f9ac8ec] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_move+0x15) [0x7eff3f9ac985] -> dovecot/imap(+0xded5) [0x7eff3fe78ed5] -> dovecot/imap(command_exec+0x3c) [0x7eff3fe836dc] -> dovecot/imap(+0x176cf) [0x7eff3fe826cf] -> dovecot/imap(+0x17785) [0x7eff3fe82785] -> dovecot/imap(client_handle_input+0x125) [0x7eff3fe82aa5] -> dovecot/imap(client_input+0x75) [0x7eff3fe82e35] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x27) [0x7eff3f6e1247] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xd7) [0x7eff3f6e1fd7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7eff3f6e0de8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7eff3f691c93] -> dovecot/imap(main+0x2c4) [0x7eff3fe772f4] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7eff3f2cfec5] -> dovecot/imap(+0xc460) [0x7eff3fe77460] 2015-06-18T19:17:50.736901+00:00 mail-103 dovecot: imap(XXX): Fatal: master: service(imap): child 4461 killed with signal 6 (core dumped) 2015-06-18T19:17:51.662157+00:00 mail-103 dovecot: imap(XXX): Panic: file mail-index.c: line 380 (mail_index_keywords_unref): assertion failed: (keywords->refcount > 0) 2015-06-18T19:17:51.662969+00:00 mail-103 dovecot: imap(XXX): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x5e271) [0x7f3fa05bb271] -> /usr/lib/dovecot/libdovecot.so.0(+0x5e34e) [0x7f3fa05bb34e] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f3fa0576a9e] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa7cbc) [0x7f3fa08cccbc] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_copy+0x6c) [0x7f3fa08968ec] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_move+0x15) [0x7f3fa0896985] -> dovecot/imap(+0xded5) [0x7f3fa0d62ed5] -> dovecot/imap(command_exec+0x3c) [0x7f3fa0d6d6dc] -> dovecot/imap(+0x176cf) [0x7f3fa0d6c6cf] -> dovecot/imap(+0x17785) [0x7f3fa0d6c785] -> dovecot/imap(client_handle_input+0x125) [0x7f3fa0d6caa5] -> dovecot/imap(client_input+0x75) [0x7f3fa0d6ce35] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x27) [0x7f3fa05cb247] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xd7) [0x7f3fa05cbfd7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f3fa05cade8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f3fa057bc93] -> dovecot/imap(main+0x2c4) [0x7f3fa0d612f4] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f3fa01b9ec5] -> dovecot/imap(+0xc460) [0x7f3fa0d61460] 2015-06-18T19:17:51.857451+00:00 mail-103 dovecot: imap(XXX): Fatal: master: service(imap): child 4464 killed with signal 6 (core dumped) protocol imap { imap_client_workarounds = tb-extra-mailbox-sep imap_idle_notify_interval = 1 mins mail_max_userip_connections = 10 mail_plugins = acl quota acl imap_acl quota imap_quota } plugin { acl = vfile:/etc/dovecot/acl acl_defaults_from_inbox = yes acl_shared_dict = proxy::acl ... } namespace { hidden = no inbox = yes list = yes location = prefix = separator = / subscriptions = yes type = private } namespace { hidden = no list = children location = maildir:/var/mail/public/Maildir:INDEX=/var/mail/%d/%n/Maildir/public prefix = public/ separator = / subscriptions = yes type = public } namespace { list = yes location = maildir:%%h/Maildir:CONTROL=~/Maildir/control/%%u:INDEX=~/Maildir/shared/%%u.indx:INDEXPVT=~/Maildir/shared/%%u.pvt prefix = shared/%%u/ separator = / subscriptions = no type = shared }

2 1

centos 7 dovecot 2.2.10 segmentation fault on devcot/auth using ldap driver in userdb
by Andrea Lisci 19 Jun '15

19 Jun '15

dovecot 2.2.10 packaged by centos 7 configuring ldap driver on userdb cause imap crash during login userdb { driver = ldap args = /etc/dovecot/dovecot-ldap-userdb.conf3.ext # Default fields can be used to specify defaults that LDAP may override #default_fields = home=/home/virtual/%u } #ldap conf --------------- hosts = myDChost base = dc=company,dc=testdomain,dc=dom ldap_version = 3 auth_bind = yes auth_bind_userdn = windowsdomain\%u user_filter = (&(objectclass=person)(|(mail=%u)(sAMAccountName=%n))) user_attrs = =uid=vmail,=gid=vmail,=home=/users/vmail/maildomain.com/%n,=mail_location=maildir:/users/vmail/maildomain.com/%n/Maildir tls = yes tls_require_cert = never dovecot -n --------------- # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-229.4.2.el7.x86_64 x86_64 CentOS Linux release 7.1.1503 (Core) xfs auth_debug = yes auth_mechanisms = plain login listen = * mail_debug = yes mail_location = maildir:/users/vmail/maildomain.com/%n/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-ldap.conf2.ext driver = ldap } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl = required ssl_cert = </etc/pki/dovecot/certs/mail.maildomain.com.crt.pem ssl_key = </etc/pki/dovecot/private/mail.maildomain.com.key.pem userdb { args = /etc/dovecot/dovecot-ldap-userdb.conf3.ext driver = ldap } protocol lmtp { mail_plugins = " quota sieve" passdb { args = password=foo driver = static } } protocol lda { mail_plugins = " sieve" } core dump ------------- gdb /usr/libexec/dovecot/auth /var/core/25122 GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-64.el7 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/libexec/dovecot/auth...Reading symbols from /usr/lib/debug/usr/libexec/dovecot/auth.debug...done. done. [New LWP 25122] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Missing separate debuginfo for /usr/lib64/libnssckbi.so Try: yum --enablerepo='*debug*' install /usr/lib/debug/.build-id/1e/a2171aee7143d971607597b24f2a97e9276cbb.debug Core was generated by `dovecot/auth'. Program terminated with signal 11, Segmentation fault. #0 db_ldap_handle_request_result (res=0x0, idx=<optimized out>, request=<optimized out>, conn=<optimized out>) at db-ldap.c:819 819 request->callback(conn, request, res->msg); (gdb) bt full #0 db_ldap_handle_request_result (res=0x0, idx=<optimized out>, request=<optimized out>, conn=<optimized out>) at db-ldap.c:819 _data_stack_cur_id = 3 srequest = <optimized out> ret = <optimized out> final_result = <optimized out> named_res = <optimized out> #1 db_ldap_handle_result (res=<optimized out>, conn=<optimized out>) at db-ldap.c:886 auth_request = 0x7f3704fa2620 idx = <optimized out> msgid = <optimized out> #2 ldap_input (conn=<optimized out>) at db-ldap.c:918 timeout = {tv_sec = 0, tv_usec = 0} res = 0x7f3704b1b530 msg = 0x7f370489fcb0 prev_reply_diff = <optimized out> ret = <optimized out> #3 0x00007f3702a19a47 in io_loop_call_io (io=0x7f37046dbad0) at ioloop.c:388 ioloop = 0x7f37046bd7b0 t_id = 2 #4 0x00007f3702a1a8cf in io_loop_handler_run (ioloop=ioloop@entry=0x7f37046bd7b0) at ioloop-epoll.c:220 ctx = 0x7f37046c5450 events = <optimized out> list = 0x7f3704998170 io = <optimized out> tv = {tv_sec = 0, tv_usec = 999118} events_count = <optimized out> msecs = <optimized out> ret = 1 i = 0 call = <optimized out> __FUNCTION__ = "io_loop_handler_run" #5 0x00007f3702a19598 in io_loop_run (ioloop=0x7f37046bd7b0) at ioloop.c:412 __FUNCTION__ = "io_loop_run" #6 0x00007f37029c69e3 in master_service_run (service=0x7f37046bd640, callback=<optimized out>) at master-service.c:571 No locals. #7 0x00007f3702ea77f1 in main (argc=1, argv=0x7f37046bd390) at main.c:393 c = <optimized out>

3 2

maildir, how to always add size tags to filename (,S=).
by Sergei Franco 19 Jun '15

19 Jun '15

Hi There, I have thoroughly read through documentation and wiki and I simply cannot find how to force dovecot to always add the size tag to the filename when using maildir format. We are using dovecot 2.2.9 (whatever comes from ubuntu 14.04 LTS). We are using dovecot in conjunction with exim with maidirsize file enabled and maildir quotas. For example exim has following configuration: maildir_tag = ,S=$message_size Which adds the size to the filename of the message. This has enormous (orders of magnitude) performance benefit as to calculate quota one does not need to stat every single file (only getents for each directory). I have wrote a script that adds the size tag retrospectively to files it didn't had it, and rebuild the dovecot index/cache if changes where made. This worked fine. Behaviour of dovecot is very peculiar: it generally adds the ,S= tag (along with ,W= tag) but occasionally it does not (I wander if it depends on the IMAP client). Is there anyway to force dovecot to add size tag to the files (like with exim example above)? Thank you very much. Sergei.

3 2

Best practices for mailbox network file storage?
by Andy Csillag 18 Jun '15

18 Jun '15

Is there a current best practice for a shared filesystem backend for mailboxes (in my case, all virtual accounts)? NFS 3/4, GFS2, GlusterFS, other? In the case of the latter ones, is the use of Director necessary? Any specific anti-recommendations? Also, if it matters, I *think* I want to use BackBlaze racks for storage. Also, I've seen recommendations for the filesystem underneath like XFS or ext4 or whatever. Is there a best practice? Thanks, Drew

3 2

Help with Namespaces
by Steve Matzura 18 Jun '15

18 Jun '15

Most of my users are virtual. Do I need defined namespaces or is the default undefined one OK?

2 2

LMTP with Postfix Problem
by Steve Matzura 18 Jun '15

18 Jun '15

Here's a maillog entry that's one of many I receive when I try sending a test message to my system. The recipient is a virtual user in /etc/virtual. The way I'm expecting it to work is that my message from me to this Postfix virtual user will be sent back to me. Every time the message delivery is attempted, Dovecot says the LMTP directory doesn't exist. I thought it was created automatically as messages arrive. Jun 18 01:48:27 <machine-name> postfix/lmtp[30644]: 4CA5B8CB6: to=<postmaster@<my-domain>>, orig_to=<postmaster>, relay=none, delay=0.02, delays=0.02/0/0/0, dsn=4.4.1, status=deferred (connect to <my-FQDN>[private/dovecot-lmtp]: No such file or directory)

2 2