- dovecot - dovecot.org

[Dovecot] syncing problems Dovecot 2.2.13
by Götz Reinicke - IT Koordinator 16 May '14

16 May '14

Hi, we still migrate a couple of users from mbox to maildir. Most of them are fine without any problem. Some crash and dont get all mails synced. first I sync backup the mbox to a maildir-NEW, than copy mbox to a new name, mv maildir-NEW to maildir. Now some problematic mailusers miss some mails from the old mbox, so I start a dsync mirror and BAM crash. e.g.: dsync(s108019): Panic: file mbox-lock.c: line 799 (mbox_lock): assertion failed: (lock_type == F_RDLCK || mbox->mbox_lock_type != F_RDLCK) dsync(s108019): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0 [0x333fe67f2a] -> /usr/lib64/dovecot/libdovecot.so.0(default_fatal_handler+0x35) [0x333fe68025] -> /usr/lib64/dovecot/libdovecot.so.0 [0x333fe67073] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mbox_lock+0x11c) [0x334025826c] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mbox_save_begin+0x98d) [0x334025a5ad] -> /usr/lib64/dovecot/lib10_quota_plugin.so [0x2b5b619183c2] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_save_begin+0x51) [0x334027b3f1] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_storage_copy+0x66) [0x3340275716] -> /usr/lib64/dovecot/lib15_notify_plugin.so [0x2b5b61b1f215] -> /usr/lib64/dovecot/lib10_quota_plugin.so [0x2b5b6191852d] -> /usr/lib64/dovecot/lib01_acl_plugin.so [0x2b5b61707b37] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_copy+0x72) [0x334027b282] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_move+0x1d) [0x334027b36d] -> dsyn [0x42f7a5] -> dsyn [0x43351a] -> dsyn(dsync_mailbox_import_changes_finish+0x165) [0x4339b5] -> dsyn(dsync_brain_sync_mails+0x58c) [0x42ed6c] -> dsyn(dsync_brain_run+0x594) [0x42af94] -> dsyn [0x429550] -> dsyn [0x411eef] -> dsyn(doveadm_mail_try_run+0x25c) [0x41231c] -> dsyn(main+0x32e) [0x41a92e] -> /lib64/libc.so.6(__libc_start_main+0xf4) [0x3d3161d9c4] -> dsyn [0x411699] ANY help and suggestion is very welcome on how to procede. Regards . Götz -- Götz Reinicke IT-Koordinator Tel. +49 7141 969 82 420 E-Mail goetz.reinicke(a)filmakademie.de Filmakademie Baden-Württemberg GmbH Akademiehof 10 71638 Ludwigsburg www.filmakademie.de Eintragung Amtsgericht Stuttgart HRB 205016 Vorsitzender des Aufsichtsrats: Jürgen Walter MdL Staatssekretär im Ministerium für Wissenschaft, Forschung und Kunst Baden-Württemberg Geschäftsführer: Prof. Thomas Schadt

1 0

[Dovecot] Disconnected from database, retrying commit?
by Ralf Hildebrandt 15 May '14

15 May '14

In my log I have a lot of log entries like this one (covering one hour of logs here): May 15 09:00:49 dict: Info: maildatabase: Disconnected from database, retrying commit May 15 09:17:10 dict: Info: maildatabase: Disconnected from database, retrying commit May 15 09:22:30 dict: Info: maildatabase: Disconnected from database, retrying commit May 15 09:31:07 dict: Info: maildatabase: Disconnected from database, retrying commit May 15 09:37:15 dict: Info: maildatabase: Disconnected from database, retrying commit May 15 09:39:36 dict: Info: maildatabase: Disconnected from database, retrying commit May 15 09:50:49 dict: Info: maildatabase: Disconnected from database, retrying commit 7 times per hour, that's quite a bit. Dovecot 2.1.17 Related logs look like: ======================= May 15 00:16:44 imap(username): Info: save: box=Sent, uid=1196, msgid=<5373EB4A.90306(a)sys4.de>, size=11152 May 15 00:16:44 dict: Info: maildatabase: Disconnected from database, retrying commit or: May 14 16:10:34 lmtp(7225, username@domain): Info: NTuhEvF1c1M5HAAAcnFQ7Q: msgid=<20090714161800.121200(a)gmx.net>: saved mail to INBOX May 14 16:10:34 dict: Info: maildatabase: Disconnected from database, retrying commit May 14 16:10:34 dict: Info: mysql(sql.sys4.de): Connected to database maildatabase So it's reconnecting, but why is it disconnecting/being disconnected in the first place? Does it matter? should I worry? -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein

1 0

[Dovecot] v2.2.13 released
by Timo Sirainen 15 May '14

15 May '14

http://dovecot.org/releases/2.2/dovecot-2.2.13.tar.gz http://dovecot.org/releases/2.2/dovecot-2.2.13.tar.gz.sig A few minor changes since v2.2.13.rc1, mainly making the Pigeonhole tests pass. * Fixed a DoS attack against imap/pop3-login processes. If SSL/TLS handshake was started but wasn't finished, the login process attempted to eventually forcibly disconnect the client, but failed to do it correctly. This could have left the connections hanging arond for a long time. (Affected Dovecot v1.1+) + mdbox: Added mdbox_purge_preserve_alt setting to keep the file within alt storage during purge. (Should become enforced in v2.3.0?) + fts: Added support for parsing attachments via Apache Tika. Enable with: plugin { fts_tika = http://tikahost:9998/tika/ } + virtual plugin: Delay opening backend mailboxes until it's necessary. This requires mailbox_list_index=yes to work. (Currently IMAP IDLE command still causes all backend mailboxes to be opened.) + mail_never_cache_fields=* means now to disable all caching. This may be a useful optimization as doveadm/dsync parameter for some admin tasks which shouldn't really update the cache file. + IMAP: Return SPECIAL-USE flags always for LSUB command. - pop3 server was still crashing in v2.2.12 with some settings - maildir: Various fixes and improvements to handling compressed mails, especially when they have broken/missing S=sizes in filenames. - fts-lucene, fts-solr: Fixed crash on search when the index contained duplicate entries. - Many fixes and performance improvements to dsync and replication - director was somewhat broken when there were exactly two directors in the ring. It caused errors about "weak users" getting stuck. - mail_attachment_dir: Attachments with the last base64-encoded line longer than the rest wasn't handled correctly. - IMAP: SEARCH/SORT PARTIAL was handled completely wrong in v2.2.11+ - acl: Global ACL file handling was broken when multiple entries matched the mailbox name. (Only the first entry was used.)

8 14

[Dovecot] v2.2.13.rc1 released
by Timo Sirainen 15 May '14

15 May '14

http://dovecot.org/releases/2.2/rc/dovecot-2.2.13.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.13.rc1.tar.gz.sig There have been a ton of smaller and some larger changes since v2.2.12, so I thought I'd first make a RC release and the final v2.2.13 tomorrow. Please try it out! BTW. I've read most of the mails again in Dovecot mailing list, but there are still a few reported bugs I decided to leave until later. Anyway feel free to re-send any bug reports for things that haven't been fixed in v2.2.13 yet. * Fixed a DoS attack against imap/pop3-login processes. If SSL/TLS handshake was started but wasn't finished, the login process attempted to eventually forcibly disconnect the client, but failed to do it correctly. This could have left the connections hanging arond for a long time. (Affected Dovecot v1.1+) + mdbox: Added mdbox_purge_preserve_alt setting to keep the file within alt storage during purge. (Should become enforced in v2.3.0?) + fts: Added support for parsing attachments via Apache Tika. Enable with: plugin { fts_tika = http://tikahost:9998/tika/ } + virtual plugin: Delay opening backend mailboxes until it's necessary. This requires mailbox_list_index=yes to work. (Currently IMAP IDLE command still causes all backend mailboxes to be opened.) + mail_never_cache_fields=* means now to disable all caching. This may be a useful optimization as doveadm/dsync parameter for some admin tasks which shouldn't really update the cache file. + IMAP: Return SPECIAL-USE flags always for LSUB command. - pop3 server was still crashing in v2.2.12 with some settings - maildir: Various fixes and improvements to handling compressed mails, especially when they have broken/missing S=sizes in filenames. - fts-lucene, fts-solr: Fixed crash on search when the index contained duplicate entries. - Many fixes and performance improvements to dsync and replication - director was somewhat broken when there were exactly two directors in the ring. It caused errors about "weak users" getting stuck. - mail_attachment_dir: Attachments with the last base64-encoded line longer than the rest wasn't handled correctly. - IMAP: SEARCH/SORT PARTIAL was handled completely wrong in v2.2.11+ - acl: Global ACL file handling was broken when multiple entries matched the mailbox name. (Only the first entry was used.)

4 8

[Dovecot] 2.2.12 RPM Needed
by Asai 15 May '14

15 May '14

Greetings, I'm not very experienced in building Source RPMs and we're in need of a Dovecot RPM for version 2.2.12 for CentOS 5. It seems that the ATRPMs repo maintained by Axel Thimm hasn't seen much activity lately and the task of updating a source RPM is more than I have time for as a busy developer who does systems admin on the side. Could anybody recommend another repo where there would be the latest Dovecot binaries? Or perhaps tell me of any gotchas and catches which might arise if I go ahead and build 2.2.12 from source over the RPM I currently have installed (2.2.10)? Thanks. -- --Asai

4 6

[Dovecot] LMTP SSL?
by Ralf Hildebrandt 15 May '14

15 May '14

Can I enable STARTTLS/SSL on the LMTP listener in Dovecot 2.1.x? service lmtp { unix_listener lmtp { } inet_listener lmtp { address = * port = 24 --> ssl=yes } } -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | http://www.charite.de

4 4

[Dovecot] Enterprise repository for free now? ;-) (Bug?)
by Daniel Thielemann 14 May '14

14 May '14

Hi, I bought repo access for 0,00$ one minute ago. If its free now then I wanna say thank you very much, otherwise somebody should fix it :) Take a look here: http://shop.dovecot.fi/home/8-dovecot-ee-repository-access.html Screen: http://www.directupload.net/file/d/3622/oxyo89zm_png.htm Regards

3 3

[Dovecot] Pigeonhole Sieve: Vacation Extension answer to every E-Mail
by Götz Reinicke - IT Koordinator 14 May '14

14 May '14

Hi, dont ask :) but I have two accounts for which I have to configure an auto reply for any e-mail send to it. As far as I understand Sieve yet, I do have to set sieve_vacation_min_period = 0 in the dovecot config and set the days value in the sieve rule. But as there is a limit of 1 day, I think this is not working. Right? Is there any other way to accomplish that? regards . Götz -- Götz Reinicke IT-Koordinator Tel. +49 7141 969 82 420 E-Mail goetz.reinicke(a)filmakademie.de Filmakademie Baden-Württemberg GmbH Akademiehof 10 71638 Ludwigsburg www.filmakademie.de Eintragung Amtsgericht Stuttgart HRB 205016 Vorsitzender des Aufsichtsrats: Jürgen Walter MdL Staatssekretär im Ministerium für Wissenschaft, Forschung und Kunst Baden-Württemberg Geschäftsführer: Prof. Thomas Schadt

3 4

[Dovecot] Broken IMAPS Connects Create Lingering imap-login Processes
by Jochen Bern 14 May '14

14 May '14

Hello everyone, we are running a central server (CentOS 6.5, dovecot-2.0.9-7.el6 with a small patch to disable the IMAP CREATE command, and openssl-1.0.1e-16.el6_5.7) and distribute standard client software to customer( site)s. The clients do IMAPS connects in regular intervals (no IDLE, no lingering logins) and authenticate with certs issued by a dedicated PKI ("auth_ssl_username_from_cert = yes" and a static global password). One of the customers has a major networking problem that hasn't been fully analyzed yet. Sniffing his IMAPS connects on the server side, I see no (necessarily fragmented) TLSv1 Client Cert + Key Exchange happen; instead, after ~60s, we receive a single packet with "TLSv1 Certificate Verify, Change Cipher Spec, Encrypted Handshake Message" *and* the TCP FIN+PSH+ACK flags set. The problem I'ld like to ask for help with here is that dovecot's imap-login process doesn't terminate when the FIN is received, or when the IMAP protocol's inactivity timeout is reached, it takes *more than two hours* for it to go away. Because of that, this single client racks up 1100+ processes (counting against dovecot's configured limits), TCP connections, and the associated RAM usage. (Since the client cert is obviously never received, the default mail_max_userip_connections of 10 doesn't come into play, either.) Is there any way - short of hexing a negative feedback loop straight into the iptables - to prevent this kind of buildup? Kind regards, J. Bern > [root ~]# date ; ps auwwwx --forest | grep -A 12 '/dove[c]ot' > Mo 5. Mai 21:45:39 CEST 2014 > root 25297 0.8 0.0 19568 824 ? Ss Apr30 64:44 /usr/sbin/dovecot > dovecot 25299 0.1 0.1 17996 5828 ? S Apr30 11:52 \_ dovecot/anvil [1147 connections] > root 25300 0.1 0.0 13388 1220 ? S Apr30 8:07 \_ dovecot/log > root 25301 0.0 0.0 39596 1564 ? S Apr30 2:21 \_ dovecot/ssl-params > dovecot 25304 0.3 0.0 78384 3552 ? S Apr30 22:13 \_ dovecot/auth [0 wait, 0 passdb, 0 userdb] > root 13161 0.3 0.3 25236 13352 ? S May04 7:11 \_ dovecot/config > root 18384 0.2 0.2 20080 8200 ? S 08:20 1:37 \_ dovecot/config [... long-running IMAP login by the operators ...] > dovenull 12064 0.0 0.0 42440 3656 ? S 19:32 0:00 \_ dovecot/imap-login [1 connections (1 TLS)] > dovenull 12441 0.0 0.0 42440 3656 ? S 19:32 0:00 \_ dovecot/imap-login [1 connections (1 TLS)] > dovenull 12495 0.0 0.0 42440 3656 ? S 19:32 0:00 \_ dovecot/imap-login [1 connections (1 TLS)] > dovenull 12496 0.0 0.0 42440 3652 ? S 19:32 0:00 \_ dovecot/imap-login [1 connections (1 TLS)] > [root ~]# doveconf -n > # 2.0.9: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-431.3.1.el6.x86_64 x86_64 CentOS release 6.5 (Final) > auth_ssl_require_client_cert = yes > auth_ssl_username_from_cert = yes > listen = [...] > login_greeting = [...] > mail_location = maildir:~ > mail_log_prefix = "%s(%u)[%p]: " > mbox_write_locks = fcntl > passdb { > args = password=[...] > driver = static > } > plugin { > mail_log_events = delete undelete expunge > mail_log_fields = uid msgid size vsize flags > } > protocols = imap > service anvil { > client_limit = 3605 > } > service auth { > client_limit = 7000 > } > service imap-login { > process_limit = 3500 > } > service imap { > process_limit = 3500 > } > ssl = required > ssl_ca = </etc/pki/dovecot/certs/[...].pem > ssl_cert = </etc/pki/dovecot/certs/[...].pem > ssl_key = </etc/pki/dovecot/private/[...].pem > ssl_verify_client_cert = yes > userdb { > args = uid=mandanten gid=mandanten home=/[...]/%Ld_[...]/%Ln > driver = static > } > verbose_proctitle = yes > protocol imap { > mail_plugins = " mail_log notify" > } -- *NEU* - NEC IT-Infrastruktur-Produkte im <http://www.linworks-shop.de/>: Server--Storage--Virtualisierung--Management SW--Passion for Performance Jochen Bern, Systemingenieur --- LINworks GmbH <http://www.LINworks.de/> Postfach 100121, 64201 Darmstadt | Robert-Koch-Str. 9, 64331 Weiterstadt PGP (1024D/4096g) FP = D18B 41B1 16C0 11BA 7F8C DCF7 E1D5 FAF4 444E 1C27 Tel. +49 6151 9067-231, Zentr. -0, Fax -299 - Amtsg. Darmstadt HRB 85202 Unternehmenssitz Weiterstadt, Geschäftsführer Metin Dogan, Oliver Michel

3 5

[Dovecot] imapc problem
by FUSTE Emmanuel 14 May '14

14 May '14

hello, I have some trouble when trying to migrate from another imap server to dovecot with imapc/dsync/doveadm. when running "doveadm -o imapc_user=foo -o imapc_password=bar backup -R -u localuser imapc:" all is ok. But when running "doveadm -o imapc_user=foo -o imapc_password=bar sync -R -u localuser imapc:", all local mods are overwritten. For example deleting a message or moving it to trash and it reappear in INBOX (and disappear from trash if moved there). If I try without the reverse flag "-R", local delete reappear two times after two or more sync. If I try without the reverse flag "-R", a message moved to trash is propagated to trash too, but reappear as many times as you "sync" on the two sides in the INBOX. With other users, first "backup -R" is ok but on next sync or backup try, I get : dsync(other_user): Error: Mailbox INBOX sync: mailbox_delete failed: INBOX can't be deleted. Other IMAP server is CriticalPath 8.5.137. Any clue ?? Emmanuel. Dovecot conf : 2.2.12.12 (03196f188677+): /etc/dovecot/dovecot.conf # OS: Linux 3.11.0-19-generic x86_64 Ubuntu 12.04.4 LTS auth_master_user_separator = * auth_username_format = %u disable_plaintext_auth = no doveadm_password = **************** imapc_features = rfc822.size fetch-headers imapc_host = my_old_imap_server imapc_user = %u lda_mailbox_autocreate = yes listen = * mail_gid = vmail mail_location = maildir:~/Maildir mail_plugins = quota notify replication mail_prefetch_count = 20 mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { hidden = no inbox = yes list = yes location = prefix = separator = / subscriptions = yes type = private } namespace { hidden = no inbox = no list = children location = maildir:%%h/Maildir:INDEXPVT=~/Maildir/shared/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } passdb { args = /etc/dovecot/dovecot-master-ldap.conf.ext driver = ldap master = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile acl_anyone = allow acl_shared_dict =file:/appli/vmail/shared-mailboxes quota = dict:userquota::file:/appli/vmail/local_userquota/%u/dovecot-quota quota_rule = *:storage=100M quota_rule2 = INBOX:storage=+20%% quota_rule3 = Trash:storage=+10%% sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap sieve service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service auth { unix_listener auth-userdb { group = vmail mode = 0660 } } service doveadm { inet_listener { port = 12345 } user = vmail } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0666 } } ssl = no userdb { args = /etc/dovecot/dovecot-users-ldap.conf.ext driver = ldap } protocol lda { mail_plugins = quota sieve } protocol imap { mail_plugins = quota imap_quota }

1 1