February 2015 - dovecot

dsync and smooth conversion of Maildir to mdbox
by Sven Strickroth 06 Feb '15

06 Feb '15

Hi, I was experimenting with dsync for Maildir to mdbox conversion. Two isses arised there: * "subscriptions" only contained subscriptions of that namespace, all others were lost - Is there any configuration missing on my side? * "dovecot-acl-list" and "dovecot-acl" shares/permissions were not synced; UPDATE: if I also add "acl" to "mail_plugins" when running dsync (using -o mail_plugins="acl zlib") it works, however, all documentation, e.g. the wiki2, only mention to add acl to lmtp/lda and imap services. - Is there any problem adding acl to "global" mail_plugins? Regarding shared namespace: I cannot convert all mailboxes at once and have to do it mailbox after mailbox. The setting "location = maildir:%%h/Maildir" won't work as some mailboxes are already converted and others aren't. Is there a way to cope with that? "doveconf -n" outout: # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.16-0.bpo.3-amd64 x86_64 Debian 7.8 dict { acldict = mysql:/etc/dovecot/dovecot-dict-user_shares.conf quotadict = mysql:/etc/dovecot/dovecot-dict-quota.conf } mail_gid = 8 mail_location = maildir:%h/Maildir mail_plugins = quota zlib mail_privileged_group = mail mail_shared_explicit_inbox = yes mail_uid = 999 namespace { list = children location = maildir:%%h/Maildir prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes list = yes location = prefix = separator = / subscriptions = yes type = private } passdb { args = /etc/dovecot/79p-sql.conf driver = sql } plugin { acl = vfile acl_anyone = allow acl_shared_dict = proxy::acldict quota = dict:user::proxy::quotadict quota_grace = 0 quota_rule2 = Trash:bytes=+100M } protocols = " imap lmtp" service auth { unix_listener auth-userdb { group = mail mode = 0660 user = vmail } } service dict { unix_listener dict { group = mail mode = 0660 user = vmail } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { mode = 0600 user = postfix } user = vmail } userdb { driver = prefetch } userdb { args = /etc/dovecot/79p-sql.conf driver = sql } -- Best regards, Sven Strickroth PGP key id F5A9D4C4 @ any key-server

1 0

Interface for managing users?
by Thomas Moulia 06 Feb '15

06 Feb '15

Hey all, I'd like to use dovecot as an testing/benchmarking IMAP server for switchboard (https://github.com/thusfresh/switchboard) As part of the tests, I need to add and destroy dovecot users. Is there an interface or tool for this sort of user management at a higher-level than editing the user store, e.g. passwd-file, sql, ...? Also, more generally, are there any tools for using dovecot to test against? Thanks! -Thomas

2 1

dovecot.index.log in Maildir/cur
by George Sexton 06 Feb '15

06 Feb '15

I'm seeing two bogus messages appearing my Maildir/cur directory. They're dovecot.index.log and dovecot-uidlist. -rw------- 1 gsexton users 51 Feb 4 09:04 Maildir/cur/dovecot-uidlist:2,S -rw------- 1 gsexton users 244 Feb 4 09:04 Maildir/cur/dovecot.index.log These files are only appearing in the Maildir/cur directory, and not in any other directory that mail is delivered to (e.g. mail list directories). I'm using the current version of Mozilla Thunderbird as my mail client. Mail is delivered to Maildir/ by procmail. I was originally using Courier, but changed to dovecot. I did run the conversion program on all the mail boxes. Other than these If anyone could help me understand what I'm doing wrong, I'd appreciate it. I've researched the mail list and read the wiki articles, but I'm just not seeing anything that looks helpful. Here's my config info: # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.16.7-7-default x86_64 openSUSE 13.2 (x86_64) -> File System: btrfs mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX. separator = . } passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap service imap-login { inet_listener imaps { port = 993 ssl = yes } } ssl = required ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem ssl_key_password = ************ ssl_protocols = !SSLv2 !SSLv3 userdb { driver = passwd } protocol imap { mail_max_userip_connections = 20 } -- George Sexton *MH Software, Inc.* Voice: 303 438 9585 http://www.mhsoftware.com

2 4

Re: Re: auth: Warning: DNS lookup took 1.550 s
by Joseph Tam 06 Feb '15

06 Feb '15

ML mail writes: > My fault here, I should have precised that the DNS query it tries to > resolve is simply the DNS name of my mailbox server > (mailboxserver.domain.com) So domain.com is hosted locally on DNS > servers on that very same network as the dovecot servers are located. > Furthermore all the resolvers I use in /etc/resolv.conf are also local > DNS resolvers on that same LAN. All the DNS servers (authoritative and > recursive) are using PowerDNS. That's why I was really puzzled to see > that it can take up to 1.5 seconds to query my own local DNS servers > which are in no way overloaded. That really doesn't change my answer, but since you control the entire DNS chain of query and response, you can further debug it at the server end via logs or network tracing. And you should still run a comprehesive DNS trace (e.g. http://dns.squish.net/) since a misconfiguration could still cause your DNS resolver to go on wild goose chases. In either case, it's probably not Dovecot related. If you want to triage it, just enter static entries into your proxy's /etc/hosts file. Or use bare IPs in your configurations. Joseph Tam <jtam.home(a)gmail.com>

1 0

Block folder names in IMAP
by André Peters 05 Feb '15

05 Feb '15

Hi, I just merged to Maildir "/" as separator with a fs layout. So I am afraid of running into problems when someone creates a folder named "new", "cur", etc. ... I know I could work this around, but isn't there a way to block folders by their name from being created? Thank you! -- André Peters

1 0

Bug when using METADATA and ACLs
by Sergio Talens-Oliag 05 Feb '15

05 Feb '15

Hi, I'm trying to use the current version of dovecot as the IMAP server for Kolab and things seem to be working more or less OK, but while working on it I've found a dovecot bug that can probably be fixed quickly by someone familiar with the code. I've found the problem using shared folders, but it is related to ACLs and METADATA. The problem appears when a user only has the 'l' (lookup) permission on a folder and later tries to get METADATA from all his or her folders; a sample session to ilustrate the problem: A1 GETACL "foo" * ACL foo admin akxeilprwtscd -admin "" sto(a)iti.es lrwstipekxacd A1 OK Getacl completed. A2 SETACL "foo" sto(a)iti.es l A2 OK Setacl complete. A3 GETACL "foo" * ACL foo admin akxeilprwtscd -admin "" sto(a)iti.es al A3 OK Getacl completed. A4 GETMETADATA "foo" (/private/vendor/kolab/folder-type /shared/vendor/kolab/folder-type) A4 NO [NOPERM] Permission denied A5 GETMETADATA "*" (/private/vendor/kolab/folder-type /shared/vendor/kolab/folder-type) <CORE DUMPED> The dovecot.log messages are: Feb 05 13:59:12 imap(sto(a)iti.es): Panic: file imap-client.c: line 636 (client_command_free): assertion failed: (client->output_cmd_lock == NULL) Feb 05 13:59:12 imap(sto(a)iti.es): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x756cf) [0x7f55ab3da6cf] -> /usr/lib/dovecot/libdovecot.so.0(+0x7572e) [0x7f55ab3da72e] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f55ab38c575] -> dovecot/imap [sto(a)iti.es 127.0.0.1 GETMETADATA](+0x18c9e) [0x7f55abba5c9e] -> dovecot/imap [sto(a)iti.es 127.0.0.1 GETMETADATA](+0x18e9a) [0x7f55abba5e9a] -> dovecot/imap [sto(a)iti.es 127.0.0.1 GETMETADATA](+0x18ef4) [0x7f55abba5ef4] -> dovecot/imap [sto(a)iti.es 127.0.0.1 GETMETADATA](client_handle_input+0x115) [0x7f55abba61e5] -> dovecot/imap [sto(a)iti.es 127.0.0.1 GETMETADATA](client_input+0x75) [0x7f55abba65b5] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7f55ab3ec51b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbb) [0x7f55ab3ed59b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7f55ab3ec5a9] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f55ab3ec628] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f55ab391a43] -> dovecot/imap [sto(a)iti.es 127.0.0.1 GETMETADATA](main+0x2b7) [0x7f55abb9a717] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f55aaff7ead] -> dovecot/imap [sto(a)iti.es 127.0.0.1 GETMETADATA](+0xd889) [0x7f55abb9a889] Feb 05 13:59:12 imap(sto(a)iti.es): Fatal: master: service(imap): child 32404 killed with signal 6 (core dumped) And the gdb output is (my binary has no debug symbols, so I guess it is not really useful): (gdb) bt full #0 0x00007f55ab00b165 in raise () from /lib/x86_64-linux-gnu/libc.so.6 No symbol table info available. #1 0x00007f55ab00e3e0 in abort () from /lib/x86_64-linux-gnu/libc.so.6 No symbol table info available. #2 0x00007f55ab3da6c5 in ?? () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #3 0x00007f55ab3da72e in ?? () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #4 0x00007f55ab38c575 in i_panic () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #5 0x00007f55abba5c9e in client_command_free () No symbol table info available. #6 0x00007f55abba5e9a in ?? () No symbol table info available. #7 0x00007f55abba5ef4 in ?? () No symbol table info available. #8 0x00007f55abba61e5 in client_handle_input () No symbol table info available. #9 0x00007f55abba65b5 in client_input () No symbol table info available. #10 0x00007f55ab3ec51b in io_loop_call_io () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #11 0x00007f55ab3ed59b in io_loop_handler_run_internal () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #12 0x00007f55ab3ec5a9 in io_loop_handler_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #13 0x00007f55ab3ec628 in io_loop_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #14 0x00007f55ab391a43 in master_service_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #15 0x00007f55abb9a717 in main () No symbol table info available. If someone can look into it I will be grateful; of course if more info is needed I'm available to help as much as I can. Thanks in advance, Sergio. -- Sergio Talens-Oliag <sto(a)iti.es> <http://www.iti.es/> Key fingerprint = FF77 A16B 9D09 FC7B 6656 CFAD 261D E19A 578A 36F2

1 0

auth: Warning: DNS lookup took 1.550 s
by ML mail 05 Feb '15

05 Feb '15

Hello, I am running a dovecot and proxy server on two different virtual machines and on the dovecot proxy server I see around 5-6 times per day the following warning: Feb 03 16:15:12 auth: Warning: proxy(email(a)domain.com,xxx.xxx.xxx.xxx,<ABC123456789>): DNS lookup for mailboxserver.domain.com took 1.550 s I do not really understand how from time to time DNS queries are slow, I tried replicate this issue using dig to resolve the same DNS entry and it was always very fast. Is there any way I can debug better this issue? or is this nothing to worry about really? Regards ML

4 5

lazy_expunge + mdbox
by fernando.figaro＠uol.com.br 05 Feb '15

05 Feb '15

Hi, I'm studying an issue related to mdbox and lazy_expunge. If it is active in the imap configuration, when you delete a message (copy to new folder + expunge old folder), a copy is also sent to the expunge namespace. I realize that is something related to mdbox and lazy_expunge_only_last_instance and found almost no references except this one (http://hg.dovecot.org/dovecot-2.2/file/24d6708cea76/TODO) lazy_expunge_only_last_instance=yes + mdbox doesn't work, because refcounts don't update immediately One other suggestion I found was to disable 'move' capability. Does anyone has other news regarding this issue ? Thanks, Fernando

1 0

Re: auth: Warning: DNS lookup took 1.550 s
by Joseph Tam 05 Feb '15

05 Feb '15

ML mail writes: > I am not running a local DNS cache on the server. As suggested using a > local DNS cache would simply fix this issue but I am more interested to > know what is generating these slow DNS queries... As another poster has commented, a slow DNS response may not be anything related to your setup, but caused by upstream DNS service. Slow DNS response or timeouts is not uncommon, and if you're only getting 6 per day, you should count yourself lucky. If it's the same FDQN that's being delayed, you can maybe run an online DNS diagnostic -- they're may be some bum delegation or unsync'd data you can report, but it will probably not be worth your time to chase this to ground unless it's causing your client to get hung. Joseph Tam <jtam.home(a)gmail.com>

1 0

Hitting wall at 2048 IMAP connections
by Ron Cleven 04 Feb '15

04 Feb '15

We are gradually rolling out Dovecot (IMAP only, no POP3) to our customer base. We are replicating between a pair of CentOS 7 boxes. All has been working wonderfully. However, to be sure our rollout continues to go smoothly, we put together a simple benchmark client program to fire up X persistent IMAP connections (using hundreds of mailboxes) that login, list the folders, select the INBOX, periodically FETCH an email from the INBOX, and otherwise sit in an IDLE loop. As long as we keep the number of concurrent IMAP connections under 2k, everything works fine. 2,000 processes generate a minimal load on the server. However, 2048 is a "hard limit" for number of IMAP processes on a box the way things are configured right now. The number (2048) is a bit too magical to be anything but a misconfiguration of either some kernel limit or a dovecot configuration limit. I have been unable to figure out where else to look. I have pasted in my dovecot config below along with the kernel setting for max_user_instances. Hoping someone can tell me what stupid mistake I have made or what else to check. Just an aside, we front-end this with a separate set of proxy servers that provide an SSL front-end, hence, no SSL in the dovecot config. /proc/sys/fs/inotify/max_user_instances = 4096 # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-123.13.1.el7.x86_64 x86_64 CentOS Linux release 7.0.1406 (Cor e) auth_master_user_separator = * auth_mechanisms = plain login default_internal_user = sysadm default_login_user = sysadm default_vsz_limit = 512 M disable_plaintext_auth = no doveadm_password = ************** doveadm_port = 12345 imap_max_line_length = 128 k listen = * mail_gid = 2001 mail_location = maildir:~/Maildir mail_plugins = notify replication mail_uid = 2001 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=PLAIN /z/imap/usr/master-users driver = passwd-file master = yes pass = yes } passdb { args = scheme=PLAIN username_format=%u /z/imap/usr/users driver = passwd-file } plugin { mail_replica = tcp:imap0s sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_max_actions = 32 sieve_max_redirects = 4 sieve_max_script_size = 1M } protocols = imap pop3 sieve service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = sysadm } unix_listener replication-notify { mode = 0666 user = sysadm } } service auth { unix_listener auth-userdb { mode = 0777 } } service doveadm { inet_listener { port = 12345 } user = sysadm } service imap { process_limit = 4096 } service managesieve-login { inet_listener sieve { port = 4190 } } service managesieve { process_limit = 4096 } service replicator { process_min_avail = 1 } ssl = no userdb { args = username_format=%u /z/imap/usr/users driver = passwd-file } protocol lda { mail_plugins = sieve notify replication } protocol imap { mail_max_userip_connections = 40 } protocol sieve { managesieve_max_line_length = 65536 }

5 4