April 2015 - dovecot - dovecot.org

concurrent IMAP connections (plus 2 easy questions)
by Ron Cleven 15 Apr '15

15 Apr '15

I posted a few days back asking about configuration issues with a modestly large number of IMAP connections. Several people were kind enough to respond with various ideas. Armed with those ideas and Google, I was able to determine the underlying configuration issues with CentOS 7 and Dovecot 2.2.10. I did some further benchmarking to ensure that we could properly plan our server configuration requirements as we continue to roll this out, so I thought I would share this, partly to see if anyone has experiences that conflict with my findings. I found that each incremental dovecot IMAP process required about 750k of RAM. Simulations confirmed that 8 gig of RAM was sufficient to support approximately 10,000 concurrent IMAP connections (no POP3). CentOS settings: Added: fs/inotify/max_user_instances = 28000 to: /etc/sysctl.d/99-sysctl.conf Added: * hard nproc 28000 * soft nproc 28000 * hard nofile 28000 * soft nofile 28000 to: /etc/security/limits.d/20-nproc.conf To verify the settings after reboot: cat /proc/sys/fs/inotify/max_user_instances ulimit -Hn ulimit -Sn Dovecot: Added: process_limit = 28000 to service imap{} /etc/dovecot/conf.d/10-master.conf Added (using proxies, would like to just turn off this check): mail_max_userip_connections = 40 to protocol imap {}: /etc/dovecot/conf.d/20-imap.conf Added: process_limit = 28000 to service managesieve {}: /etc/dovecot/conf.d/20-managesieve.conf Hope this helps someone. Feedback would be welcome. It would be great if there was a blessed cookbook for Dovecot scaling. At this point it seems a bit like an art form. Now to my two "easy" questions. 1) TCP replication between our two CentOS boxes has been working great, no complaints. However, I realized I did not know how to check the status of replications, as one might want to do if one of the two boxes was taken offline for maintenance or repair. On the surface, it would appear that the command: doveadm replicator status '*' would likely show me what I want to know. However, when I tried it, I got: doveadm(): Fatal: net_connect_unix(/var/run/dovecot/replicator-doveadm) failed: No such file or directory The documentation says that doveadm assumes the socket /var/run/dovecot/replicator-doveadm and the command format provides a "-a" override. In that directory, there is no "replicator-doveadm" socket, but I do see a "replicator" socket. So, should I be using the command: doveadm replicator status -a /var/run/dovecot/replicator '*' or is the non-existence of the replicator-doveadm socket indicative of something I might have done wrong with the config? I hate experimenting more than I have to with a production box. The 2.2.10 Dovecot I am running was installed via yum, so there are no potential compilation issues. 2) It seems logical to me that: dovecot stop would first lock out any new user connections, then do the equivalent of a: doveadm kick "*" before actually cycling everything else down to ensure the shutdown is as graceful as possible. I suppose I could experiment and find out for certain, but I have the sense that the "stop" command is not quite that elegant, so we have implemented procedures to work around it. Just a point of curiosity. An alternative would be if there was a doveadm command to lock out any new user connections, which could then be followed by kick and stop. I have found that many clients are VERY quick to reconnect after a kick.

2 1

Authcache and user changing
by Lazy 13 Apr '15

13 Apr '15

Hi, I have noticed that during auth cache hits usernames are not updated. (We use ldap backend and change username with user_attrs = uid=user, mailMessageStore=home, mailQuotaSize=quota_rule=*:bytes=%$ cold cache lmtp(14414): Debug: auth input: testmon_testmon home=/vmail/te/testmon_testmon quota_rule=*:bytes=104857600 lmtp(14414): Debug: changed username to testmon_testmon lmtp(14414): Debug: Added userdb setting: plugin/quota_rule=*:bytes=104857600 hot cache lmtp(14715): Debug: auth input: iqmon(a)mon.test.pl home=/vmail/iq/testmon_testmon quota_rule=*:bytes=104857600 lmtp(14715): Debug: Added userdb setting: plugin/quota_rule=*:bytes=104857600 this kills our dictionary based quota (users have multiple quota instances depending on timing and alias the email was delivered to. Is there a way to force addition of user to the auth cache ? -- Michal Grzedzicki

3 5

special_use folders
by Roman Gelfand 13 Apr '15

13 Apr '15

My setup is postfix mail gateway/spamassassin on one machine and dovecot mail box server on another machine. Will the following configuration work? 1. On the mail gateway, I install dovecot-lmtp and use it to deliver locally spam. 2. On the mailbox server, I configure special use junk folder to point mail gateway's spam folder via nfs. If this is correct, how can I configure special use folder to point to a path (variable based on email) which is different from default path? If there is a different way to handle this, I would interested. Thanks in advance

3 2

dovecot debug message help required
by Rajesh M 13 Apr '15

13 Apr '15

hi i was carrying out some testing using qmailtoaster with dovecot on a centos 64 server i am connecting to the mailbox using a windows machine with outlook across the lan. the mailbox is around 7gb in size with around 12000 emails spread across multiple folders and i am trying to sync the mailbox folders to by creating an account imap account in outlook. the mails are getting downloaded in the imap folder but the sync does not complete and outlook throws an error. Task 'Synchronizing subscribed folders for user(a)mydomain.com.' reported error (0x800CCC0F) : 'Outlook cannot synchronize subscribed folders for user(a)mydomain.com.. Error: The TCP/IP connection was unexpectedly terminated by the server. If you continue to receive this message, contact your server administrator or Internet service provider (ISP).' in the dovecot log i see the following. Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: Effective uid=89, gid=89, home=/home/vpopmail/domains/mydomain.com/user Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: Quota root: name=ignore=Trash backend=maildir args= Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: Quota rule: root=ignore=Trash mailbox=? bytes=0 messages=0 Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: Quota grace: root=ignore=Trash bytes=0 (10%) Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: Namespace : type=private, prefix=, sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location= Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: mdbox: access(/home/vpopmail/domains/mydomain.com/user/mdbox, rwx): failed: No such file or directory Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: mdbox: couldn't find root dir Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: sdbox: access(/home/vpopmail/domains/mydomain.com/user/sdbox, rwx): failed: No such file or directory Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: sdbox: couldn't find root dir Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: maildir: root exists (/home/vpopmail/domains/mydomain.com/user/Maildir) Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: maildir++: root=/home/vpopmail/domains/mydomain.com/user/Maildir, index=, indexpvt=, control=, inbox=/home/vpopmail/domains/mydomain.com/user/Maildir, alt= Apr 10 17:12:40 imap(user(a)mydomain.com): Info: Disconnected: Logged out in=28 out=460 i would like to know what the following means -- specifically mdbox and sdbox #################### Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: mdbox: access(/home/vpopmail/domains/mydomain.com/user/mdbox, rwx): failed: No such file or directory Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: mdbox: couldn't find root dir Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: sdbox: access(/home/vpopmail/domains/mydomain.com/user/sdbox, rwx): failed: No such file or directory Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: sdbox: couldn't find root dir #################### my dovecot config file # dovecot -n # 2.2.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-431.el6.x86_64 x86_64 CentOS release 6.5 (Final) auth_cache_size = 32 M auth_mechanisms = plain login digest-md5 cram-md5 default_login_user = vpopmail disable_plaintext_auth = no first_valid_gid = 89 first_valid_uid = 89 log_path = /var/log/dovecot.log login_greeting = Dovecot toaster ready. mail_debug = yes mail_max_userip_connections = 50 mail_plugins = " quota" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = prefix = separator = . type = private } passdb { args = cache_key=%u webmail=127.0.0.1 driver = vpopmail } plugin { quota = maildir:ignore=Trash quota_rule = ?:storage=0 } protocols = imap pop3 service imap-login { client_limit = 1000 process_limit = 400 process_min_avail = 4 service_count = 0 vsz_limit = 512 M } service pop3-login { client_limit = 1000 process_limit = 400 process_min_avail = 12 service_count = 0 vsz_limit = 512 M } ssl_cert = </var/qmail/control/servercert.pem ssl_dh_parameters_length = 2048 ssl_key = </var/qmail/control/servercert.pem userdb { args = cache_key=%u quota_template=quota_rule=*:backend=%q driver = vpopmail } protocol imap { imap_client_workarounds = delay-newmail mail_plugins = " quota imap_quota" } protocol pop3 { pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_fast_size_lookups = yes pop3_lock_session = no pop3_no_flag_updates = yes } thank you, rajesh

2 1

Disabling of userdb/passdb modules using config statements
by Jeroen Massar 13 Apr '15

13 Apr '15

Hola, Debian (and possibly other distros) use the /etc/dovecot/conf.d/* setup where default config files are stuffed and then one can just add a 99-myconfig.conf et voila, variables are overruled. This allows the distro to supply updates to the files at package upgrade time without any/much user intervention. The problem (for me ;) is that the system comes provided with: auth-system.conf.ext containing: passdb { driver = pam } userdb { driver = passwd } Hence pam & /etc/passwd based are always enabled. This while I don't have any local users. Replication seems to then always pick up the local users, which are vmail + nobody (65536). doveadm user '*' thus reports vmail, nobody + virtual users Setting: first_valid_uid = 5000 last_valid_uid = 5000 only keeps vmail in there, but apparently some module (guess replication) is still able to figure out that 'nobody' exists: Apr 10 09:48:25 mail dovecot: doveadm(IPADDR,nobody): Error: Mail access for users with UID 65534 not permitted (see first_valid_uid in config file, uid from userdb lookup). Apr 10 09:48:25 mail dovecot: doveadm(IPADDR,nobody): Error: dsync-server: User init failed Apr 10 09:49:38 mail dovecot: doveadm(nobody): Error: sync: Failed to start remote dsync-server command: Remote exit_code=75 and on the other side: Apr 10 09:54:38 mail dovecot: doveadm(nobody): Error: sync: Unknown user in remote This can be resolved by commenting out the entries in auth-system.conf.ext but then I'll have to do that again at package upgrade time. Hence, would it be a cool option to be able (in the 99-myconfig.conf) file to put: passdb { driver = pam enabled = false } userdb { driver = passwd enabled = false } And thereby disabling those modules completely? Thus avoiding upgrade conflicts etc. Greets, Jeroen

4 7

doveadm save ...
by Hardy Flor 13 Apr '15

13 Apr '15

Hello, when I have with "doveadm save ..." store a message, how do I get the guid or uid of this message for the setthe flags? Hardy

4 8

dovecot-2.2.16 compilation problems under Solaris Studio 11
by Jürgen Obermann 13 Apr '15

13 Apr '15

Hello here are two problems I get when I compile dovecot 2.2.16 under Solaris 10 with Studio 11 Compiler: gmake[4]: Entering directory `/net/fileserv/export/sunsrc/src/dovecot-2.2.16/src/lib' source='test-array.c' object='test_lib-test-array.o' libtool=no \ DEPDIR=.deps depmode=none /bin/bash ../../depcomp \ cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib-test -I/opt/local/include -I/usr/local/include -fast -xarch=v8plusa -I/opt/local/include -c -o test_lib-test-array.o `test -f 'test-array.c' || echo './'`test-array.c "test-array.c", line 129: warning: initializer does not fit or is out of range: -32768 "test-array.c", line 129: warning: initializer does not fit or is out of range: -16384 "test-array.c", line 129: warning: initializer does not fit or is out of range: -512 "test-array.c", line 129: warning: initializer does not fit or is out of range: -256 "test-array.c", line 129: warning: initializer does not fit or is out of range: -128 "test-array.c", line 129: warning: initializer does not fit or is out of range: -64 "test-array.c", line 129: warning: initializer does not fit or is out of range: -2 "test-array.c", line 129: warning: initializer does not fit or is out of range: -1 "test-array.c", line 154: operand cannot have void type: op "=" "test-array.c", line 154: assignment type mismatch: unsigned short "=" const void "test-array.c", line 206: operand cannot have void type: op "=" "test-array.c", line 206: assignment type mismatch: pointer to const char "=" const void cc: acomp failed for test-array.c gmake[4]: [test_lib-test-array.o] Fehler 2 (ignoriert) gmake[3]: Entering directory `/net/fileserv/export/sunsrc/src/dovecot-2.2.16/src/lib-imap' source='imap-match.c' object='imap-match.lo' libtool=yes \ DEPDIR=.deps depmode=none /bin/bash ../../depcomp \ /bin/bash ../../libtool --tag=CC --mode=compile cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-test -I../../src/lib-charset -I../../src/lib-mail -I/opt/local/include -I/usr/local/include -fast -xarch=v8plusa -I/opt/local/include -c -o imap-match.lo imap-match.c libtool: compile: cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-test -I../../src/lib-charset -I../../src/lib-mail -I/opt/local/include -I/usr/local/include -fast -xarch=v8plusa -I/opt/local/include -c imap-match.c -KPIC -DPIC -o .libs/imap-match.o "imap-match.c", line 199: identifier redeclared: imap_match_dup current : function(pointer to struct pool {pointer to const struct pool_vfuncs {..} v, unsigned int alloconly_pool :1, unsigned int datastack_pool :1}, pointer to const struct imap_match_glob {pointer to struct pool {..} pool, pointer to struct imap_match_pattern {..} patterns, char sep, array[-1] of char patterns_data}) returning pointer to struct imap_match_glob {pointer to struct pool {..} pool, pointer to struct imap_match_pattern {..} patterns, char sep, array[-1] of char patterns_data} previous: function(pointer to struct pool {pointer to const struct pool_vfuncs {..} v, unsigned int alloconly_pool :1, unsigned int datastack_pool :1}, pointer to const struct imap_match_glob {pointer to struct pool {..} pool, pointer to struct imap_match_pattern {..} patterns, char sep, array[-1] of char patterns_data}) returning pointer to struct imap_match_glob {pointer to struct pool {..} pool, pointer to struct imap_match_pattern {..} patterns, char sep, array[-1] of char patterns_data} : "imap-match.h", line 33 "imap-match.c", line 214: identifier redeclared: imap_match_globs_equal current : function(pointer to const struct imap_match_glob {pointer to struct pool {..} pool, pointer to struct imap_match_pattern {..} patterns, char sep, array[-1] of char patterns_data}, pointer to const struct imap_match_glob {pointer to struct pool {..} pool, pointer to struct imap_match_pattern {..} patterns, char sep, array[-1] of char patterns_data}) returning _Bool previous: function(pointer to const struct imap_match_glob {pointer to struct pool {..} pool, pointer to struct imap_match_pattern {..} patterns, char sep, array[-1] of char patterns_data}, pointer to const struct imap_match_glob {pointer to struct pool {..} pool, pointer to struct imap_match_pattern {..} patterns, char sep, array[-1] of char patterns_data}) returning _Bool : "imap-match.h", line 36 cc: acomp failed for imap-match.c gmake[3]: [imap-match.lo] Fehler 1 (ignoriert) Greetings Jürgen -- Hochschulrechenzentrum der | Mail: Juergen.Obermann(a)hrz.uni-giessen.de Justus-Liebig-Universitaet | WWW: http://www.uni-giessen.de/obermann/ Heinrich-Buff-Ring 44 | Tel: 0641-99-13054 (0641-99-13001) D-35392 Giessen, Germany | Fax: 0641-99-13009

4 4

sieve plugin editheader not found
by Patrick Lion 11 Apr '15

11 Apr '15

Hey folks, ive already made a bug report in the gentoo bugtracker last year. In the meantime after some releases the error persists with the current dovecot and pigeonhole versions. Please see my bug report at gentoo mailinglist: https://bugs.gentoo.org/show_bug.cgi?id=522148 Okay, here are some informations: Steps to Reproduce: add "editheader" to sieve options or "sievec" a file that need the extension if header :matches "Something" "*" { addheader "Anything" "${1}"; deleteheader "Something"; } sievec mod_header.sieve sievec(root): Fatal: Plugin 'editheader' not found from directory /usr/lib64/dovecot/sieve dovecot --version 2.2.16 pigeonhole-0.4.7 sources got downloaded and used in the build process dovecot -n # 2.2.16: /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.7 # OS: Linux 2.6.32-openvz-042stab102.9-amd64 x86_64 Gentoo Base System release 2.2 auth_mechanisms = plain login auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no imap_capability = CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=ORDEREDSUBJECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED CONTEXT=SEARCH LIST-STATUS SPECIAL-USE XLIST QUOTA info_log_path = syslog login_log_format_elements = user=<%u> method=%m ip=%r mpid=%e mail_location = maildir:%h/.maildir mail_plugins = quota managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include body enotify environment mailbox date index ihave duplicate imapflags notify namespace compat { alias_for = hidden = yes inbox = no list = no location = prefix = INBOX. separator = . } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { args = /etc/dovecot/dovecot-sql.conf.ext quota = maildir quota_rule2 = trash:storage=+150M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +notify +imapflags sieve_global_extensions = variables } postmaster_address = mailer-daemon(a)dev.someserver service auth { unix_listener /var/spool/postfix/private/auth { group = mail mode = 0660 user = postfix } unix_listener auth-userdb { group = mail mode = 0660 user = postfix } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service quota-warning { executable = script /etc/postfix/quota-warning.sh unix_listener quota-warning { group = hosting mode = 0660 user = postfix } user = postfix } ssl_cert = </etc/ssl/dovecot/server.pem ssl_key = </etc/ssl/dovecot/server.key userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lda { mail_plugins = quota sieve postmaster_address = mailer-daemon(a)dev.someserver } protocol imap { mail_plugins = quota imap_quota } protocol pop3 { mail_plugins = quota } protocol lmtp { hostname = dev.someserver mail_plugins = quota sieve postmaster_address = postmaster(a)dev.someserver } Please let me know if you need any additional informations, build logs ( log from 2.213 attached at bugs.gentoo.org ) or whatever may be helpful to track this down.

3 4

Calendar and address book with Dovecot
by mimicafe＠gmail.com 10 Apr '15

10 Apr '15

Hi all I recently install Postfix and Dovecot, and so far so good. My assumption was that calendar and address book (similar to Gmail calendar or address book) are part of Dovecot. However I do not see anything concerning them in any of the docs I have read so far. What other software (ideally open source, free) do I need in other to have Calendar and address book so my users can manage their contact on their phones or computer? I have looked at Roundcube already, but my requirement is not necessarily to provide web mail. Thanks Mimi

15 16

pop3 retr responses too large lines - Fails with current python's poplib
by Guillermo M. Narvaja 10 Apr '15

10 Apr '15

Since December 2014, clients using the Python poplib library (getmail in my case) started to limit the line length of RETR and other command responses sent from the server to 2048 bytes: See https://bugs.python.org/issue16041 (Included at least in Python-2.7.9) Dovecot doesn't have line split in the POP3 module, so Python fails to retrieve messages with long lines. RFC 1939 says: " Responses in the POP3 consist of a status indicator and a keyword possibly followed by additional information. All responses are terminated by a CRLF pair. Responses may be up to 512 characters long, including the terminating CRLF. " I think dovecot-pop3 should split it's output in lines of at least 512 characters. Or at least it should have a "pop3_max_line_length" setting (like "imap_max_line_length"). Server side: # dovecot --version 2.2.13 Client side: # python --version Python 2.7.9 # getmail --version getmail 4.47.0 Temporary workaround (on client-side): --- poplib.py.orig 2015-04-08 18:46:48.280879211 -0300 +++ poplib.py 2015-04-08 18:26:01.288556028 -0300 @@ -36,7 +36,7 @@ # reading arbitrary length lines. RFC 1939 limits POP3 line length to # 512 characters, including CRLF. We have selected 2048 just to be on # the safe side. -_MAXLINE = 2048 +_MAXLINE = 2048*100 class POP3: Thanks in advance. -- Guillermo M. Narvaja Lambda Sistemas S.R.L. www.fierro-soft.com.ar - radiocut.fm Tel: (5411) 3220-1520 (rotativas) y 4857-6662 Cel: (5411) 15-6783-4435 Email: guillermo.narvaja(a)fierro-soft.com.ar MSN: guillermo_narvaja(a)hotmail.com Skype: guillermonarvaja Lavalleja 519 1er Piso - Ciudad de Buenos Aires - Argentina

2 4