- dovecot - dovecot.org

Index corruption in 2.2.18
by Anton Chevychalov 21 Mar '16

21 Mar '16

We are using dbox with lz4 compression and dovecot-lda + sieve for delivery. Mail recieved: Mar 16 06:27:55 mail2 dovecot: lda(svs(a)example.com): sieve: msgid=<20160316032754.4C89D2C4(a)rdbms.oldstable.office.example.com>: stored mail into mailbox 'INBOX' Around 08:14:00 user start his thunderbird with local filters. He found one mail without headers. Around that time user do repair folder in thunderbird: Mar 16 10:14:44 mail2 dovecot: imap(svs(a)example.com): Error: read(/mail/boxes/example.com/svs/storage/m.139) failed: Cached message size larger than expected (1572 > 980, box=Logs/Backup, UID=9763, cached Message-Id=<20160316032754.4C89D2C4(a)rdbms.oldstable.office.example.com>) Mar 16 10:14:44 mail2 dovecot: imap(svs(a)example.com): Error: Corrupted index cache file /mail/boxes/example.com/svs/mailboxes/Logs/Backup/dbox-Mails/dovecot.index.cache: Broken physical size for mail UID 9763 Mar 16 10:14:44 mail2 dovecot: imap(svs(a)example.com): Error: read(/mail/boxes/example.com/svs/storage/m.139) failed: Cached message size larger than expected (1572 > 980, box=Logs/Backup, UID=9763, cached Message-Id=<20160316032754.4C89D2C4(a)rdbms.oldstable.office.example.com>) (FETCH BODY[] for mailbox Logs/Backup UID 9763) Mar 16 10:14:44 mail2 dovecot: imap(svs(a)example.com): Disconnected: FETCH read() failed in=14812 out=1077049 Mar 16 10:14:45 mail2 dovecot: imap(svs(a)example.com): Error: read(/mail/boxes/example.com/svs/storage/m.139): FETCH BODY[] for mailbox Logs/Backup UID 9763 got too little data: 1008 vs 1613 Mar 16 10:14:45 mail2 dovecot: imap(svs(a)example.com): Error: Corrupted index cache file /mail/boxes/example.com/svs/mailboxes/Logs/Backup/dbox-Mails/dovecot.index.cache: Broken virtual size for mail UID 9763 Mar 16 10:14:45 mail2 dovecot: imap(svs(a)example.com): Disconnected: FETCH failed in=837 out=641651 And still in a same state: Mar 16 11:42:51 mail2 dovecot: imap(svs(a)example.com): Error: read(/mail/boxes/example.com/svs/storage/m.139): FETCH BODY[] for mailbox Logs/Backup UID 9763 got too little data: 1008 vs 1613 Mar 16 11:42:51 mail2 dovecot: imap(svs(a)example.com): Error: Corrupted index cache file /mail/boxes/example.com/svs/mailboxes/Logs/Backup/dbox-Mails/dovecot.index.cache: Broken virtual size for mail UID 9763 Mar 16 11:42:51 mail2 dovecot: imap(svs(a)example.com): Disconnected: FETCH failed in=12462 out=984043 In thunderbird we see raw lz4 data. I have today backup (duplicate mail on exim to per day mbox). So I can restore that message, but I found older problems in other mailboxes and I can't restore it due to tapes rotation I see a lot of fixes in 2.2.19? Was it fixed after 2.2.18? -- Anton Chevychalov

3 2

doveadm sync/backup doesn't sync the metadata?
by Heiko Schlittermann 21 Mar '16

21 Mar '16

Hello, in a current migration project we want to migrate from Cyrus to Dovecot. The Cyrus server stores expiry information as ANNOTATIONS. On the Dovecot side I've set up the metadata/annotations plugin and it is working. But 'dovecot backup …' doesn't seem to sync these metadata? Did I miss something here? Shouldn't the metadata be part of the backup? (I'm using the latest 2.2.22 for sync purposes, as the stock dovecot just crashed when used as 'doveadm backup …') Best regards from Dresden/Germany Viele Grüße aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ -

2 1

Understand the UID value in mailbox debug
by Tobi 21 Mar '16

21 Mar '16

Hi list we have a client who complains that mails from a certain sender are always put into his spamfolder. Our strong guess is that this is an issue with the webmail (Roundcube), but I need some help in understanding the output from dovecot's mailbox-debug. In order to narrow down the source of the problem we activated the mailbox-debug from Dovecot. Now we can see that the mail is moved into the spamfolder. But the strange thing is that a UID is logged which does not exist on the mailserver The following log is the "move" of the mail into the spamfolder Mar 19 11:53:57 backend3 dovecot: imap(REMOVED_FOR_PRIVACY): OmQLrmQubgDAqDIW copy from INBOX: box=INBOX.Spam, uid=58, msgid=<000601d181cd$71401940$53c04bc0$(a)bluewin.ch>, size=314585 Mar 19 11:53:57 backend3 dovecot: imap(REMOVED_FOR_PRIVACY): OmQLrmQubgDAqDIW delete: box=INBOX, uid=1614, msgid=<000601d181cd$71401940$53c04bc0$(a)bluewin.ch>, size=314585 Mar 19 11:53:57 backend3 dovecot: imapREMOVED_FOR_PRIVACY): OmQLrmQubgDAqDIW expunge: box=INBOX, uid=1614, msgid=<000601d181cd$71401940$53c04bc0$(a)bluewin.ch>, size=314585 I do not understand whose UID this is, as it does not exist on the server. Does dovecot create temporary UID or something like that? Or is it more a process ID? Thanks and regards tobi

4 5

Problem configuring sieve
by Jerry 20 Mar '16

20 Mar '16

This is a fresh installation of dovecot on a FreeBSD 11 system. Trying to run the following command produces this error: sievec ./default.sieve sievec(root): Debug: Loading modules from directory: /usr/local/lib/dovecot sievec(root): Error: Couldn't load required plugin /usr/local/lib/dovecot/lib90_sieve_plugin.so: Can't load plugin sieve_plugin: Plugin is intended to be used only by binaries: lda lmtp (we're sievec) sievec(root): Fatal: Internal error occurred. Refer to server log for more information. *** Error code 89 This is my configuration: # 2.2.22 (fe789d2): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.12 (c1c0a23) # OS: FreeBSD 11.0-CURRENT amd64 zfs auth_mechanisms = plain login lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /var/log/dovecot.log mail_debug = yes mail_location = maildir:/var/mail/vmail/%d/gerard mail_plugins = sieve mail_save_crlf = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = scheme=CRYPT username_format=%u /usr/local/etc/dovecot/users/passwd driver = passwd-file } plugin { sieve_default = /usr/local/etc/dovecot/sieve/default.sieve } postmaster_address = postmaster(a)seibercom.net service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = vmail mode = 0666 user = vmail } } service dict { unix_listener dict { group = vmail mode = 0600 user = vmail } } ssl_cert = </etc/ssl/demoCA/certs/dovecot.pem ssl_key = </etc/ssl/demoCA/private/dovecot.pem ssl_protocols = !SSLv2 !SSLv3 userdb { args = username_format=%u /usr/local/etc/dovecot/users/passwd driver = passwd-file } protocol lda { mail_plugins = sieve } There is nothing in the logs regarding this. Can anyone help me? -- Jerry

2 1

Re: Setting up public mailboxes - user not found
by Marti Markov 20 Mar '16

20 Mar '16

Hey, I actually went with the method suggested by Steffen Kaiser of creating a separate userdb because I use LDAP and I have an OU which contains all the public mailboxes and was just easier. I setup the userdb like so: userdb { args = /etc/dovecot/dovecot-ldap-shared_mailboxes.conf.ext driver = ldap override_fields = mail=maildir:/home/vmail/Public/.%n } Now when I send an email to office3(a)xxx.com I don’t get any errors and it delivers the message to the office3(a)xxx.com: root@mail:~# ls /home/vmail/Public/.office3/new/ 1458427035.M781836P21781.mail.xxx.com,S=2571,W=2630 1458427705.M750112P23338.mail.xxx.com,S=2501,W=2559 But from any mail client I try I don’t seem to be able to list anything in the Public directory. Am I missing something in the configuration? Here is the public namespace configuration: namespace { list = yes location = maildir:/home/vmail/Public:INDEXPVT=~/Maildir/Public prefix = Public/ separator = / subscriptions = yes type = public } > On 17 Mar 2016, at 08:35, Burckhard Schmidt <bschmidt(a)cms.hu-berlin.de> wrote: > > hello Marti, > > Am 17.03.2016 um 06:18 schrieb Marti Markov: >> First of all thanks for the reply Burckhard. >> When you say: >>> You can use a "normal" user who has a sieve script: > > Normal user that's an account with one or more email aliasses. > So this "user" can receive emails like any other user. This I called "normal" user. But this ist not of interest in our case. > So you don't get an "user not found". > > Next you will add an appropriate email alias to the account if necessary (our accounts are cryptic). > > Now you have an account/user and you can write a sieve script for that user to deliver all emails into one public folder or into different folders. Public subfolders are possible too, if you like or are able to distinguish incomming emails. > > In your case it is not necessary to have an user/account "office3". This name you would only use in that sieve script with "fileinto". > > Regards! > > Burckhard > >> what would that user be in my case? office3? Because I don’t really see how it will solve the problem of getting the error 550 user not found when making the LDAP search. >> >>> On 10 Mar 2016, at 08:50, Burckhard Schmidt <bschmidt(a)cms.hu-berlin.de> wrote: >>> >>> Am 09.03.2016 um 20:02 schrieb Marti Markov: >>>> Hi all, >>>> >>>> This is the first time I use the dovecot mail list so I’m sorry if I forget something. >>>> >>>> My problem is that for some reason I can get public mailboxes to work. I have setup the directory Public, the folders and the cur,tmp and new folder in them: >>>> >>>> Public >>>> | >>>> |- .office3 >>>> | >>>> |- cur >>>> |- tmp >>>> |- new >>>> |- dovecot-acl >>>> >>>> >>>> Here is my dovecot conf: >>>> >>> >>>> namespace { >>>> list = yes >>>> location = maildir:/home/vmail/xxx.com/Public:INDEXPVT=~/Maildir/Public >>>> prefix = Public/ >>>> separator = / >>>> subscriptions = yes >>>> type = public >>>> } >>> >>> You can use a "normal" user who has a sieve script: >>> require ... >>> any filter if necessary >>> fileinto "Public"; >>> discard; >>> ... >>> >>> "Public" is your prefix >>> I have several subfolder so I use: fileinto "Public/subfolder1"; >>> >>> You would have >>> /home/vmail/xxx.com/Public/subfolder1 >>> with cur new tmp >>> and a file "dovecot-acl" containing anyone lrs >>> >>> -- >>> Burckhard Schmidt >> > > > -- > Mit freundlichen Grüßen --- Burckhard Schmidt > > Abteilung Systemsoftware und Kommunikation > ZE Computer- und Medienservice der Humboldt-Universität zu Berlin > Postanschrift: Unter den Linden 6, 10099 Berlin > Standort: Rudower Chaussee 26; 12489 Berlin > Tel.: +49-30-2093-70058 Fax: +49-30-2093-70199 > Mail: bschmidt(a)cms.hu-berlin.de <mailto:bschmidt@cms.hu-berlin.de>

1 0

dovecot-sieve in ports?
by unknown＠example.com 20 Mar '16

20 Mar '16

# portmaster mail/dovecot-sieve ===>>> The dependency for mail/dovecot seems to be handled by dovecot2-2.2.22 ===>>> Initial dependency check complete for mail/dovecot-sieve ===>>> Starting build for mail/dovecot-sieve <<<=== ===>>> All dependencies are up to date ===> Cleaning for dovecot-sieve-1.2+0.1.19_2 ===>>> Waiting on fetch & checksum for mail/dovecot-sieve <<<=== ===> License LGPL21 accepted by the user ===> dovecot-sieve-1.2+0.1.19_2 depends on file: /usr/local/sbin/pkg - found ===> Fetching all distfiles required by dovecot-sieve-1.2+0.1.19_2 for building ===> dovecot-1.2.17_6 conflicts with installed package(s): dovecot2-2.2.22 Sooo… Is dovecot-sieve for dovecot 1.x only? Or is the ports tree wonky? -- 2+2=5 for sufficiently large values of 2.

3 2

v2.2.22 released
by Timo Sirainen 19 Mar '16

19 Mar '16

http://dovecot.org/releases/2.2/dovecot-2.2.22.tar.gz http://dovecot.org/releases/2.2/dovecot-2.2.22.tar.gz.sig + Added doveadm HTTP API: See http://wiki2.dovecot.org/Design/DoveadmProtocol/HTTP + virtual plugin: Mailbox filtering can now be done based on the mailbox metadata. See http://wiki2.dovecot.org/Plugins/Virtual + stats: Added doveadm stats reset to reset global stats. + stats: Added authentication statistics if auth_stats=yes. + dsync, imapc, pop3c & pop3-migration: Many optimizations, improvements and error handling fixes. + doveadm: Most commands now stop soon after SIGINT/SIGTERM. - auth: Auth caching was done too aggressively when %variables were used in default_fields, override_fields or LDAP pass/user_attrs. userdb result_* were also ignored when user was found from cache. - imap: Fixed various assert-crashes caused v2.2.20+. Some of them caught actual hangs or otherwise unwanted behavior towards IMAP clients. - Expunges were forgotten in some situations, for example when pipelining multiple IMAP MOVE commands. - quota: Per-namespaces quota were broken for dict and count backends in v2.2.20+ - fts-solr: Search queries were using OR instead of AND as the separator for multi-token search queries in v2.2.20+. - Single instance storage support wasn't really working in v2.2.16+ - dbox: POP3 message ordering wasn't working correctly. - virtual plugin: Fixed crashes related to backend mailbox deletions.

3 2

lmtp timeout, locks and crashes
by Tom Sommer 19 Mar '16

19 Mar '16

I'm seeing some problems on accounts which get a lot of spam (like, a lot). I get these errors: Mar 15 10:43:49 lmtp(12390): Error: Timeout (29s) while waiting for lock for transaction log file /var/spool/mail/dovecot/xxxx/xxxx/dovecot.list.index.log (WRITE lock held by pid 12193) --- Mar 15 10:43:49 lmtp(12034): Error: Transaction log file /var/spool/mail/dovecot/xxxxx/xxxx/dovecot.list.index.log: marked corrupted --- Mar 15 10:43:49 lmtp(12391): Error: Transaction log /var/spool/mail/dovecot/xxxx/xxxx/dovecot.list.index.log: duplicate transaction log sequence (3) ---- Mar 15 10:43:59 lmtp(xxxx(a)xxxx.dk): Error: Timeout (29s) while waiting for lock for transaction log file /var/spool/mail/dovecot/xxxx/xxxx/dovecot.list.index.log ---- Mar 15 10:43:51 lmtp(12294): Panic: file mail-transaction-log-file.c: line 104 (mail_transaction_log_file_free): assertion failed: (!file->locked) Mar 15 10:43:51 lmtp(12294): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0 [0x30d567e07a] -> /usr/lib/dovecot/libdovecot.so.0 [0x30d567e0e6] -> /usr/lib/dovecot/libdovecot.so.0 [0x30d567d4ac] -> /usr/lib/dovecot/libdovecot-storage.so.0 [0x30d5299ca7] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_transaction_logs_clean+0x28) [0x30d529b698] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_transaction_log_view_close+0x49) [0x30d529a7e9] -> /usr/lib/dovecot/libdovecot-storage.so.0 [0x30d5293c03] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_list_index_refresh_later+0x92) [0x30d525af62] -> /usr/lib/dovecot/libdovecot-storage.so.0 [0x30d52589eb] -> /usr/lib/dovecot/libdovecot-storage.so.0 [0x30d5259217] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit_get_changes+0x54) [0x30d52352c4] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit+0x16) [0x30d5235386] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_user_autoexpunge+0x1f8) [0x30d522a9b8] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_user_unref+0x2a) [0x30d523a16a] -> dovecot/lmtp [DATA 172.17.165.14] [0x40696b] -> dovecot/lmtp [DATA 172.17.165.14] [0x406bd6] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x49) [0x30d5690579] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xdc) [0x30d5691c3c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xa9) [0x30d56906b9] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x30d56909e8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x30d562e533] -> dovecot/lmtp [DATA 172.17.165.14](main+0x15c) [0x404c8c] -> /lib64/libc.so.6(__libc_start_main+0xf4) [0x36aea1d9f4] -> dovecot/lmtp [DATA 172.17.165.14] [0x404a79] ---- The data is stored on NFS, with single-node access. mmap_disable is set to 'yes' Anyone know how to solve this? or can share some insights. Running dovecot 2.2.21 (5345f22) -- Tom Sommer

4 9

poppassd
by Adrian Minta 19 Mar '16

19 Mar '16

Hello, does anyone of you knows a poppassd implementation compatible with dovecot mysql or ldap backends ? -- Best regards, Adrian Minta

1 0

TLS handshake issue
by John Oliver 18 Mar '16

18 Mar '16

dovecot-2.0.9 on CentOS 6.7 The system in question is not connected to the Internet, so I can't copy-and-paste. I have to type anything required :-( Brand-new out-of-the-box install with a really minimal dovecot.conf including: service imap-login { inet_listener imaps { address = 192.168.1.10 port = 143 ssl = yes } } ssl_cert=</etc/pki/tls/certs/dovecot.pem ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL ssl_key =</etc/pki/tls/private/dovecot.pem That's very, very similar to an existing dovecot server on an old VM I need to replace. Certs are self-signed, I know that's a horrible thing to do, but right now we don't have any choice. I'm connecting with Apple Mail 8.2 running on OS X 10.10.5, another thing we have no choice about :-/ The Apple Mail just sits there stupidly. It's "Connection Doctor" just helpfully reports that it can't establish a connection. I can use 'openssl s_client -showcerts -connect mail:143' and see what I expect to see. The dovecot log with lots of verbosity enabled tells me: imap-login: Info: Disconnected (no auth attempts): rip=192.168.1.200, lip=192.168.1.10, TLS handshaking: Disconnected auth: Debug: auth client connected (pid=21006) imap-login: Warning: SSL: where=0x10, ret=1: before/accept initialization [192.168.1.200] imap-login: Warning: SSL: where=0x2001, ret=1: before/accept initialization [192.168.1.200] imap-login: Warning: SSL: where=0x2002, ret=1: SSLv2/v3 read client hello A [192.168.1.200] And that's it... those lines get repeated every minute that Mail is running. I'm not seeing anything in any logs that even hints at what it's unhappy about, or any way to increase verbosity any more. Any hints appreciated! -- *********************************************************************** * John Oliver http://www.john-oliver.net/ * * * ***********************************************************************

4 5