- dovecot - dovecot.org

doveadm-server hangs with 100% cpu usage
by Dennis Kuhn 02 Sep '15

02 Sep '15

we have some replication issues. From time to time a doveadm-server process takes 100% cpu in the state recv_mailbox_tree_deletes on the replica. The process runs forever until it is manually killed. Strace on this process doesn't show anything. Sometimes we have several doveadm-server processes in this state, all for the same account, all with 100% CPU Load. The logfile on the master says: Error: Timeout during state=recv_mailbox_tree Error: dsync(dobby5.heinlein-support.de): I/O has stalled, no activity for 600 seconds My workaround is to delete the user directory on the replica so that the whole account is replicated again. This solves the problem for this specific account. The dovecot version is 2.2.15 on the master and 2.2.16 on the replica. Dennis Kuhn -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-57 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin

3 4

Moving mails from shared folder to shared folder doesn't work
by Peer Heinlein 02 Sep '15

02 Sep '15

Hi, having the attached configuration with Dovecot 2.2.9, Dovecot's crashing when moving a mail from a shared folder into another shared folder. Moving that mail into a "real" folder of the user or moving that mail from a real folder into the shared folder is always working. But moving from a shared folder into a shared folder doesn't work. Sep 2 11:47:42 mail02 dovecot: imap(peer.heinlein(a)example.com): Panic: file mail-index.c: line 380 (mail_index_keywords_unref): assertion failed: (keywords->refcount > 0) Sep 2 11:47:42 mail02 dovecot: imap(peer.heinlein(a)example.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x62792) [0x7f87e455f792] -> /usr/lib/dovecot/libdovecot.so.0(+0x627f6) [0x7f87e455f7f6] -> /usr/lib/dovecot/libdovecot.so.0(+0x1c5ca) [0x7f87e45195ca] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb032f) [0x7f87e487232f] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_copy+0x75) [0x7f87e4838315] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_move+0x1d) [0x7f87e48383dd] -> dovecot/imap [peer.heinlein(a)example.com 172.31.xxx.xxx UID move](+0xec4a) [0x7f87e4cfbc4a] -> dovecot/imap [peer.heinlein(a)example.com 172.31.xxx.xxx UID move](command_exec+0x3d) [0x7f87e4d06acd] -> dovecot/imap [peer.heinlein(a)example.com 172.31.xxx.xxx UID move](+0x18b60) [0x7f87e4d05b60] -> dovecot/imap [peer.heinlein(a)example.com 172.31.xxx.xxx UID move](+0x18c4d) [0x7f87e4d05c4d] -> dovecot/imap [peer.heinlein(a)example.com 172.31.xxx.xxx UID move](client_handle_input+0x11d) [0x7f87e4d05edd] -> dovecot/imap [peer.heinlein(a)example.com 172.31.xxx.xxx UID move](client_input+0x72) [0x7f87e4d06292] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f87e456f5f6] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xaf) [0x7f87e45706cf] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f87e456f568] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f87e451e0c3] -> dovecot/imap [peer.heinlein(a)example.com 172.31.xxx.xxx UID move](main+0x303) [0x7f87e4d10043] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7f87e4192c8d] -> dovecot/imap [peer.heinlein(a)example.com 172.31.xxx.xxx UID move](+0xd0e9) [0x7f87e4cfa0e9] Sep 2 11:47:42 mail02 dovecot: imap(peer.heinlein(a)example.com): Fatal: master: service(imap): child 18707 killed with signal 6 (core dumps disabled) Peer -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin

1 0

Single Instance Storage after the fact
by Joe Beaubien 02 Sep '15

02 Sep '15

Hi everyone, I tried to find info about this online, but I couldn't find anything talking specifically about what I am trying to accomplish. I am sending and receiving the same attachments pretty often and as a result my email accounts are growing much more rapidly than I would like. *My goal* is to set up a second instance of Dovecot with SIS enabled and transfer everything over to this new Dovecot setup. I was thinking of simply running a dsync backup from the old instance and then a dsync backup -R on the new instance with SIS. Would this effectively create a new set of mailboxes without attachments (and a new folder full of attachments)? If not, can someone point me to information on how to accomplish this? Best regards, Joe B.

1 0

Problem with the Path argument in pigeonhole 0.4.3 at ubuntu 14.04
by Jakob-Matthias Böttger 31 Aug '15

31 Aug '15

Hello list members I want dovecot managesieve to store its config unter ~/.dovecot.sieve additionally dovecot should read as sieve_after the file ~/.sieve It's working fine BUT dovecot managesieve is always symlinking the sieve file at activation to ~/.sieve and not to ~/.dovecot.sieve as i configured it. Any idears about this strange behaviour? dovecot is configured as follows # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 3.13.0-53-generic x86_64 Ubuntu 14.04.3 LTS auth_mechanisms = plain login dict { acl = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext expire = mysql:/etc/dovecot/dovecot-dict-expire.conf.ext } disable_plaintext_auth = no lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = *,[::] log_timestamp = "%Y-%m-%d %H:%M:%S " mail_plugins = " expire quota acl" mail_privileged_group = vmail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = prefix = separator = / type = private } namespace { list = children location = maildir:%%h/Maildir:INDEXPVT=~/Maildir/shared/%%u prefix = shared/%%u/ separator = / subscriptions = yes type = shared } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { acl = vfile acl_shared_dict = proxy::acl expire = Trash expire2 = Trash/* expire3 = Spam expire4 = Junk expire_dict = proxy::expire quota = dict:user::file:/var/vmail/%d/%n/.quotausage sieve = file:~/sieve;active=~/.dovecot.sieve sieve_after = ~/.sieve } postmaster_address = admin(a)domain.tld protocols = imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = root } service dict { unix_listener dict { group = vmail mode = 0660 user = vmail } } service imap-login { client_limit = 1000 process_limit = 500 } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } } service managesieve-login { inet_listener sieve { port = 4190 } } ssl = required ssl_cert = </etc/ssl/own_certs/certs/stern.domain.tld.chain.crt ssl_key = </etc/ssl/own_certs/private/stern.domain.tld.key ssl_protocols = !SSLv2 !SSLv3 userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol imap { mail_max_userip_connections = 100 mail_plugins = quota imap_quota expire acl imap_acl } protocol pop3 { mail_plugins = quota pop3_uidl_format = %08Xu%08Xv } protocol lda { mail_plugins = sieve quota } protocol lmtp { mail_plugins = sieve quota }

2 1

HG port=0 changes still problematic
by Thomas Leuxner 31 Aug '15

31 Aug '15

Aug 30 12:35:04 nihlus.leuxner.net systemd[1]: Started Dovecot IMAP/POP3 email server. Aug 30 12:35:04 nihlus.leuxner.net dovecot[60404]: doveconf: Fatal: Error in configuration file /etc/dovecot/conf.d/10-master.conf line 24: Invalid port number 0 Aug 30 12:35:04 nihlus.leuxner.net systemd[1]: dovecot.service: main process exited, code=exit System with HG e1c7fcd62813 fails on inet_listener imaps { port = 0 #port = 993 #ssl = yes } Regards Thomas

3 3

Post-migration questions
by Grant 29 Aug '15

29 Aug '15

I just migrated from courier-imap to dovecot and dovecot is a lot faster. My only client is webmail on the same server as dovecot. Max ~10 users. Should I still run imapproxy like I did with courier? Does dovecot periodically empty the trash folder like courier does? I reduced the number of processes courier was running to save RAM and improve performance. Are there any performance tweaks I should consider for dovecot? - Grant

2 2

IMAP hibernate feature committed
by Timo Sirainen 29 Aug '15

29 Aug '15

http://hg.dovecot.org/dovecot-2.2/rev/64c73e6bd397 Today I finally committed the "imap-hibernate" feature that I first started developing about a year ago (and had been thinking about for several years before that). The main purpose here is to reduce the number of imap processes and the amount of memory they use by moving IDLEing connections into imap-hibernate processes where they are waiting for something to happen (changes to the mailbox or IMAP client to do something). Mailbox changes are noticed only if inotify/kqueue is enabled, so without them the hibernation feature won't work well. For now only the IMAP connections that are in IDLE can be hibernated. In theory non-IDLEing connections could also be hibernated, but the main problem here is that if EXPUNGEs can't be sent at the time the hibernation is over, the current code can't handle it. They would require recreating a view that contains such expunged messages, which could be possible with some extra work. Some IMAP extensions aren't supported yet: - NOTIFY: This would just need to import/export the NOTIFY state. Also hibernation could be done at any time if VANISHED replies could be sent to the selected mailbox. - SEARCH=CONTEXT: A bit more troublesome extension to support. I'm not sure how big the trouble would be. It's rarely used though. - COMPRESS: The [de]compression code would need to be moved to login process instead of being done in imap-compress plugin. This would be a larger change, although it's also useful for various other purposes. The hibernation can be enabled for now simply by setting imap_hibernate_timeout setting to non-zero, which controls how long to wait after client has issued IDLE (and nothing happens) until the process hibernates. Some future work might include: - Moving the hibernation all the way to the first Dovecot proxy layer to avoid the need for imap-hibernate process at all. (Also moving the COMPRESS handling all the way there.) - Dynamically adjusting when the client should be hibernated instead of just a fixed imap_hibernate_timeout setting. Especially try to avoid such a situation where the hibernation is almost always terminated soon after it started. Although that probably is unlikely, so maybe not worth the trouble to prevent.. - Hibernation could be used to control the load of the server. If it's running too slow or if there are already too many imap processes, some of the connections could be moved to hibernation stage. Or just kept in hibernation for a while longer even after the imap process would normally be recreated (e.g. wait for several minutes before reporting new emails, or several seconds before replying to DONE). - The EXPUNGE/VANISHED and FETCH FLAGS replies aren't sent asynchronously, but instead fully buffered to the output buffer. This works as long as the list of changes after hibernation isn't huge, but otherwise it's wasting a lot of memory. - Maybe support non-inotify/kqueue change notifications from lib-storage. This could be complicated though to do efficiently. Or a simple way would be to just send a UDP or UNIX datagram packet on all changes to all the imap-hibernate processes and see if anybody cares about those. I'm not sure how efficient that would be.

4 13

virtual users with postfix
by Andreas Meyer 29 Aug '15

29 Aug '15

Hello! My passwd file looks like this: info@nimmini.de:{CRAM-MD5}e33cbd144b372a9c919228923464c806b57e35df2d6d1484ceaabe7e3530cd58:5000:5000::/var/spool/vhosts/nimmini.de/info::userdb_mail=maildir:~/ When I login for the first time, I see this in the log: Aug 29 09:52:57 imap(info(a)nimmini.de): Error: mkdir_parents(/var/lib/dovecot/db) failed: Permission denied Aug 29 09:52:57 imap(info(a)nimmini.de): Error: acl: dict commit failed Although the mailbox "info" is created with this content: drwx------ 2 5000 5000 4096 29. Aug 09:52 cur -rw------- 1 5000 5000 0 29. Aug 09:52 dovecot-acl-list -rw------- 1 5000 5000 272 29. Aug 09:52 dovecot.index.log -rw------- 1 5000 5000 72 29. Aug 09:52 dovecot.mailbox.log -rw------- 1 5000 5000 51 29. Aug 09:52 dovecot-uidlist -rw------- 1 5000 5000 8 29. Aug 09:52 dovecot-uidvalidity -r--r--r-- 1 5000 5000 0 29. Aug 09:52 dovecot-uidvalidity.55e164dc drwx------ 5 5000 5000 4096 29. Aug 09:52 .Drafts drwx------ 2 5000 5000 4096 29. Aug 09:52 new drwx------ 5 5000 5000 4096 29. Aug 09:52 .Sent -rw------- 1 5000 5000 18 29. Aug 09:52 subscriptions drwx------ 2 5000 5000 4096 29. Aug 09:52 tmp drwx------ 5 5000 5000 4096 29. Aug 09:52 .Trash /var/lib looks like this: ... drwxr-xr-x 2 root root 4096 29. Aug 00:28 dovecot ... Should the folder dovecot be owned by the user dovecot? What causes the error above? Greetings Andreas

4 9

Dovecot proxy and password scheme updating
by efs efefsfse 28 Aug '15

28 Aug '15

Hello, I run a dovecot proxy (which is doing authentification itself) and a dovecot backend on separate boxes. I want to change the actual password scheme. Since postlogin scripting is not supported by proxies and %w variable is not forwaded to dovecot backend is there any way that I can update the password scheme with my current setup ? Thanks in advance.

1 0

Allow delivery to existing accounts only with LDAP and static
by Miloslav Hůla 28 Aug '15

28 Aug '15

Hi, we are using LDAP binding as a passdb, and static with allow_all_users=yes as an userdb. Works fine, but problem is, Maildirs are created for non-existent accounts too. We would like to prevent it. The LDAP binding does not supporta user lookups. Is the correct way to use checkpassword as a passdb before LDAP, check for account existency here and: result_success=continue result_failure=return-fail ? Thank you, regards, Miloslav

2 4