- dovecot - dovecot.org

IMAP hibernate feature committed
by Timo Sirainen 29 Aug '15

29 Aug '15

http://hg.dovecot.org/dovecot-2.2/rev/64c73e6bd397 Today I finally committed the "imap-hibernate" feature that I first started developing about a year ago (and had been thinking about for several years before that). The main purpose here is to reduce the number of imap processes and the amount of memory they use by moving IDLEing connections into imap-hibernate processes where they are waiting for something to happen (changes to the mailbox or IMAP client to do something). Mailbox changes are noticed only if inotify/kqueue is enabled, so without them the hibernation feature won't work well. For now only the IMAP connections that are in IDLE can be hibernated. In theory non-IDLEing connections could also be hibernated, but the main problem here is that if EXPUNGEs can't be sent at the time the hibernation is over, the current code can't handle it. They would require recreating a view that contains such expunged messages, which could be possible with some extra work. Some IMAP extensions aren't supported yet: - NOTIFY: This would just need to import/export the NOTIFY state. Also hibernation could be done at any time if VANISHED replies could be sent to the selected mailbox. - SEARCH=CONTEXT: A bit more troublesome extension to support. I'm not sure how big the trouble would be. It's rarely used though. - COMPRESS: The [de]compression code would need to be moved to login process instead of being done in imap-compress plugin. This would be a larger change, although it's also useful for various other purposes. The hibernation can be enabled for now simply by setting imap_hibernate_timeout setting to non-zero, which controls how long to wait after client has issued IDLE (and nothing happens) until the process hibernates. Some future work might include: - Moving the hibernation all the way to the first Dovecot proxy layer to avoid the need for imap-hibernate process at all. (Also moving the COMPRESS handling all the way there.) - Dynamically adjusting when the client should be hibernated instead of just a fixed imap_hibernate_timeout setting. Especially try to avoid such a situation where the hibernation is almost always terminated soon after it started. Although that probably is unlikely, so maybe not worth the trouble to prevent.. - Hibernation could be used to control the load of the server. If it's running too slow or if there are already too many imap processes, some of the connections could be moved to hibernation stage. Or just kept in hibernation for a while longer even after the imap process would normally be recreated (e.g. wait for several minutes before reporting new emails, or several seconds before replying to DONE). - The EXPUNGE/VANISHED and FETCH FLAGS replies aren't sent asynchronously, but instead fully buffered to the output buffer. This works as long as the list of changes after hibernation isn't huge, but otherwise it's wasting a lot of memory. - Maybe support non-inotify/kqueue change notifications from lib-storage. This could be complicated though to do efficiently. Or a simple way would be to just send a UDP or UNIX datagram packet on all changes to all the imap-hibernate processes and see if anybody cares about those. I'm not sure how efficient that would be.

4 13

virtual users with postfix
by Andreas Meyer 29 Aug '15

29 Aug '15

Hello! My passwd file looks like this: info@nimmini.de:{CRAM-MD5}e33cbd144b372a9c919228923464c806b57e35df2d6d1484ceaabe7e3530cd58:5000:5000::/var/spool/vhosts/nimmini.de/info::userdb_mail=maildir:~/ When I login for the first time, I see this in the log: Aug 29 09:52:57 imap(info(a)nimmini.de): Error: mkdir_parents(/var/lib/dovecot/db) failed: Permission denied Aug 29 09:52:57 imap(info(a)nimmini.de): Error: acl: dict commit failed Although the mailbox "info" is created with this content: drwx------ 2 5000 5000 4096 29. Aug 09:52 cur -rw------- 1 5000 5000 0 29. Aug 09:52 dovecot-acl-list -rw------- 1 5000 5000 272 29. Aug 09:52 dovecot.index.log -rw------- 1 5000 5000 72 29. Aug 09:52 dovecot.mailbox.log -rw------- 1 5000 5000 51 29. Aug 09:52 dovecot-uidlist -rw------- 1 5000 5000 8 29. Aug 09:52 dovecot-uidvalidity -r--r--r-- 1 5000 5000 0 29. Aug 09:52 dovecot-uidvalidity.55e164dc drwx------ 5 5000 5000 4096 29. Aug 09:52 .Drafts drwx------ 2 5000 5000 4096 29. Aug 09:52 new drwx------ 5 5000 5000 4096 29. Aug 09:52 .Sent -rw------- 1 5000 5000 18 29. Aug 09:52 subscriptions drwx------ 2 5000 5000 4096 29. Aug 09:52 tmp drwx------ 5 5000 5000 4096 29. Aug 09:52 .Trash /var/lib looks like this: ... drwxr-xr-x 2 root root 4096 29. Aug 00:28 dovecot ... Should the folder dovecot be owned by the user dovecot? What causes the error above? Greetings Andreas

4 9

Dovecot proxy and password scheme updating
by efs efefsfse 28 Aug '15

28 Aug '15

Hello, I run a dovecot proxy (which is doing authentification itself) and a dovecot backend on separate boxes. I want to change the actual password scheme. Since postlogin scripting is not supported by proxies and %w variable is not forwaded to dovecot backend is there any way that I can update the password scheme with my current setup ? Thanks in advance.

1 0

Allow delivery to existing accounts only with LDAP and static
by Miloslav Hůla 28 Aug '15

28 Aug '15

Hi, we are using LDAP binding as a passdb, and static with allow_all_users=yes as an userdb. Works fine, but problem is, Maildirs are created for non-existent accounts too. We would like to prevent it. The LDAP binding does not supporta user lookups. Is the correct way to use checkpassword as a passdb before LDAP, check for account existency here and: result_success=continue result_failure=return-fail ? Thank you, regards, Miloslav

2 4

Merging maildirs
by Petter Gunnerud 28 Aug '15

28 Aug '15

Hi List How do I merge files from one maildir into another maildir? Long version: I've been using dovecot/postfix for a 12 user environment for several years, and today was the first time I ran into an issue I could not solve from reading the documentation. The server is gentoo running on esxi. A few times a year I've made a snapshot of the server and copied the files to my home esxi server as a full system backup. 5 days ago the internet connection at the office broke, and I decided to power on the last copy of the server at my home and change public dns so that users could access their new mails via webmail from this spare server. (I didn't allow imap through firewall.) Now the network at the office is back. DNS is updated so that mails are delivered to the office server again. But all the mails sent / received during those 5 days is only at my house. How can I merge those into the office server?I'd like to do this as a server administrator. I do not want to have each user forward the mails to them self from the temporarily webmail. I tried to just copy the mail files on my own account - like I did once when I used courier-imap, but that did not work with dovecot. I hope someone has an idea of how to solve this.

3 2

Panic in service(log)
by Eugene Paskevich 27 Aug '15

27 Aug '15

Hello, I'm using the Debian builds of dovecot-2.2 by S[r]us. Today I started getting the following panic: log: Panic: file priorityq.c: line 122 (priorityq_remove_idx): assertion failed: (idx < count) log: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x7f1ae) [0x7fd368dc31ae] -> /usr/lib/dovecot/libdovecot.so.0(+0x7f269) [0x7fd368dc3269] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fd368d6c81e] -> /usr/lib/dovecot/libdovecot.so.0(+0xa0de8) [0x7fd368de4de8] -> /usr/lib/dovecot/libdovecot.so.0(priorityq_remove+0xb) [0x7fd368de4eeb] -> /usr/lib/dovecot/libdovecot.so.0(+0x9179c) [0x7fd368dd579c] -> /usr/lib/dovecot/libdovecot.so.0(master_status_update+0x93) [0x7fd368d71263] -> /usr/lib/dovecot/libdovecot.so.0(+0x2ecf3) [0x7fd368d72cf3] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7fd368dd6b2c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xc5) [0x7fd368dd7e35] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7fd368dd6bb5] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fd368dd6d58] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fd368d72213] -> dovecot/log(main+0x11c) [0x555e9f81771c] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7fd3689bcb45] -> dovecot/log(+0x277d) [0x555e9f81777d] master: Error: service(log): child 28845 killed with signal 6 (core dumps disabled) Could anyone please advise? -- Eugene Paskevich | *==)----------- | Plug me into eugene(a)raptor.kiev.ua | -----------(==* | The Matrix

2 2

mailbox deletion is not replicated correctly
by Dennis Kuhn 27 Aug '15

27 Aug '15

i can't delete several folders at once. I have a folder structure like folder1/folder2/folder3 (we use slash as hierarchy separator). When i delete folder1 server1 executes the command correctly Apr 16 13:44:28 server1 dovecot: imap(d.kuhn(a)heinlein-support.de): Mailbox deleted: INBOX/Trash/folder1/folder2/folder3 Apr 16 13:44:28 server1 dovecot: imap(d.kuhn(a)heinlein-support.de): Mailbox deleted: INBOX/Trash/folder1/folder2 Apr 16 13:44:28 server1 dovecot: imap(d.kuhn(a)heinlein-support.de): Mailbox deleted: INBOX/Trash/folder1 but the replica server2 reverses the order: Apr 16 13:44:29 server2 dovecot: dsync-server(d.kuhn(a)heinlein-support.de): Mailbox deleted: INBOX/Trash/folder1 Apr 16 13:44:29 server2 dovecot: dsync-server(d.kuhn(a)heinlein-support.de): Mailbox deleted: INBOX/Trash/folder1/folder2 Apr 16 13:44:29 server2 dovecot: dsync-server(d.kuhn(a)heinlein-support.de): Mailbox deleted: INBOX/Trash/folder1/folder2/folder3 This creates the folder INBOX/Trash/folder1/folder2 on server2 again which replicates back to server1. Regards, Dennis Kuhn -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-57 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin

3 4

mail_log proposal: recipient
by Peter Chiochetti 27 Aug '15

27 Aug '15

As Timo seems to be on a committing spree; I'd like to propose that in the mail_log_fields the recipient would be available. Like probably many dovecot users, my MUAs save to Sent; so when grepping the log to learn etc. that would just be useful Kind regards -- peter

2 2

Duplicate folder names
by Paulo Matos 26 Aug '15

26 Aug '15

Hi, I have setup message filtering and for example, I might move a message which a given sender to folder special/. This folder is created and shows up twice in INBOX/special/ and special/. They are however exactly the same folder. If I subscribe to one I am automatically subscribed to the other and viceversa. Not too happy with this configuration. I have a feeling this is related to namespaces but I am not sure in which way to configure things in order to have only INBOX/special but not special/. How can I set this up. My current configuration is: $ dovecot -n # 2.2.18: /etc/dovecot/dovecot.conf # OS: Linux 4.1.0-x86_64-linode59 x86_64 Fedora release 22 (Twenty Two) ext4 auth_debug = yes auth_mechanisms = plain login digest-md5 cram-md5 auth_verbose = yes auth_verbose_passwords = yes default_internal_user = root imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_debug = yes mail_location = maildir:/home/vmail/%d/%n/Maildir maildir_very_dirty_syncs = yes mbox_write_locks = fcntl namespace { inbox = yes location = prefix = INBOX. separator = . type = private } namespace inbox { location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } postmaster_address = pmatos protocols = imap quota_full_tempfail = yes service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-master { mode = 0600 user = vmail } user = $default_internal_user } ssl = required ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { args = uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes driver = static } protocol lda { auth_socket_path = /var/run/dovecot/auth-master deliver_log_format = msgid=%m: %$ log_path = /home/vmail/dovecot-deliver.log } protocol imap { mail_max_userip_connections = 100 } Regards, Paulo Matos

2 2

PROXY protocol
by Hoggins! 26 Aug '15

26 Aug '15

Hello folks, Any plans on implementing the PROXY protocol to allow Dovecot being behind a TCP proxy, and still logging the real IP address of the users ? See : http://blog.haproxy.com/haproxy/proxy-protocol/ Thanks !

6 15