April 2015 - dovecot - dovecot.org

Disk space usage with mdbox
by Alexandros Soumplis 15 Apr '15

15 Apr '15

Hello, I am using dovecot with mdbox+sis and I notice an ever increasing disk space usage since I converted mailboxes from Maildir to mdboxes. I have checked with a user and while it actually uses only 65K, his mdbox files on disk are more than 6G. The backup of his mailbox is just 64K. Any suggestions ? Below are some relevant commands: [root@mail ~]# doveadm quota get -u test Quota name Type Value Limit % User quota STORAGE 10135 31457280 0 User quota MESSAGE 186 - 0 [root@mail ~]# du -k --max-depth=1 /mdboxes/test/ 220 /mdboxes/test/mailboxes 6029348 /mdboxes/test/storage 6029592 /mdboxes/test/ [root@mail ~]# doveadm purge -u test [root@mail ~]# du -k --max-depth=1 /mdboxes/test/ 220 /mdboxes/test/mailboxes 6029348 /mdboxes/test/storage 6029592 /mdboxes/test/ [root@mail ~]# doveadm backup -u test mdbox:/tmp/MDBOX_TEMP/ [root@mail ~]# du -k --max-depth=1 /tmp/MDBOX_TEMP/ 16 /tmp/MDBOX_TEMP/mailboxes 65540 /tmp/MDBOX_TEMP/storage 65568 /tmp/MDBOX_TEMP/

4 9

setting service_count results in Permission denied
by Micah Anderson 15 Apr '15

15 Apr '15

The load on our machines has gotten too high, so I started to see log messages that directed me to look at this page for config changes: https://pad.riseup.net/redirect#http%3A//wiki2.dovecot.org/SocketUnavailable One of the things I wanted to do on that page was to set the: service imap { service_count = 100 } but when I added that to the service imap block, and restarted dovecot, I see this in the logs: Error: net_connect_unix(/run/dovecot/imap-postlogin) failed: Permission denied I dont understand how this impacts permissions. Its clearly this setting, because when I remove it, things work again. I found nothing else in the logs indicating a configuration problem. This is what the config block looked like: service imap { # Most of the memory goes to mmap()ing files. You may need to increase this # limit if you have huge mailboxes. #vsz_limit = 256M # Max. number of IMAP processes (connections) process_limit = 4096 service_count = 100 executable = imap imap-postlogin } service imap-postlogin { # the script process runs as the user specified here (v2.0.14+): #user = $default_internal_user executable = script-login /usr/local/sbin/postlogin_imap # this UNIX socket listener must use the same name as given to imap executable unix_listener imap-postlogin { } } This is with dovecot 2.2.13, thanks for any help you can provide! micah

4 3

concurrent IMAP connections (plus 2 easy questions)
by Ron Cleven 15 Apr '15

15 Apr '15

I posted a few days back asking about configuration issues with a modestly large number of IMAP connections. Several people were kind enough to respond with various ideas. Armed with those ideas and Google, I was able to determine the underlying configuration issues with CentOS 7 and Dovecot 2.2.10. I did some further benchmarking to ensure that we could properly plan our server configuration requirements as we continue to roll this out, so I thought I would share this, partly to see if anyone has experiences that conflict with my findings. I found that each incremental dovecot IMAP process required about 750k of RAM. Simulations confirmed that 8 gig of RAM was sufficient to support approximately 10,000 concurrent IMAP connections (no POP3). CentOS settings: Added: fs/inotify/max_user_instances = 28000 to: /etc/sysctl.d/99-sysctl.conf Added: * hard nproc 28000 * soft nproc 28000 * hard nofile 28000 * soft nofile 28000 to: /etc/security/limits.d/20-nproc.conf To verify the settings after reboot: cat /proc/sys/fs/inotify/max_user_instances ulimit -Hn ulimit -Sn Dovecot: Added: process_limit = 28000 to service imap{} /etc/dovecot/conf.d/10-master.conf Added (using proxies, would like to just turn off this check): mail_max_userip_connections = 40 to protocol imap {}: /etc/dovecot/conf.d/20-imap.conf Added: process_limit = 28000 to service managesieve {}: /etc/dovecot/conf.d/20-managesieve.conf Hope this helps someone. Feedback would be welcome. It would be great if there was a blessed cookbook for Dovecot scaling. At this point it seems a bit like an art form. Now to my two "easy" questions. 1) TCP replication between our two CentOS boxes has been working great, no complaints. However, I realized I did not know how to check the status of replications, as one might want to do if one of the two boxes was taken offline for maintenance or repair. On the surface, it would appear that the command: doveadm replicator status '*' would likely show me what I want to know. However, when I tried it, I got: doveadm(): Fatal: net_connect_unix(/var/run/dovecot/replicator-doveadm) failed: No such file or directory The documentation says that doveadm assumes the socket /var/run/dovecot/replicator-doveadm and the command format provides a "-a" override. In that directory, there is no "replicator-doveadm" socket, but I do see a "replicator" socket. So, should I be using the command: doveadm replicator status -a /var/run/dovecot/replicator '*' or is the non-existence of the replicator-doveadm socket indicative of something I might have done wrong with the config? I hate experimenting more than I have to with a production box. The 2.2.10 Dovecot I am running was installed via yum, so there are no potential compilation issues. 2) It seems logical to me that: dovecot stop would first lock out any new user connections, then do the equivalent of a: doveadm kick "*" before actually cycling everything else down to ensure the shutdown is as graceful as possible. I suppose I could experiment and find out for certain, but I have the sense that the "stop" command is not quite that elegant, so we have implemented procedures to work around it. Just a point of curiosity. An alternative would be if there was a doveadm command to lock out any new user connections, which could then be followed by kick and stop. I have found that many clients are VERY quick to reconnect after a kick.

2 1

Authcache and user changing
by Lazy 13 Apr '15

13 Apr '15

Hi, I have noticed that during auth cache hits usernames are not updated. (We use ldap backend and change username with user_attrs = uid=user, mailMessageStore=home, mailQuotaSize=quota_rule=*:bytes=%$ cold cache lmtp(14414): Debug: auth input: testmon_testmon home=/vmail/te/testmon_testmon quota_rule=*:bytes=104857600 lmtp(14414): Debug: changed username to testmon_testmon lmtp(14414): Debug: Added userdb setting: plugin/quota_rule=*:bytes=104857600 hot cache lmtp(14715): Debug: auth input: iqmon(a)mon.test.pl home=/vmail/iq/testmon_testmon quota_rule=*:bytes=104857600 lmtp(14715): Debug: Added userdb setting: plugin/quota_rule=*:bytes=104857600 this kills our dictionary based quota (users have multiple quota instances depending on timing and alias the email was delivered to. Is there a way to force addition of user to the auth cache ? -- Michal Grzedzicki

3 5

special_use folders
by Roman Gelfand 13 Apr '15

13 Apr '15

My setup is postfix mail gateway/spamassassin on one machine and dovecot mail box server on another machine. Will the following configuration work? 1. On the mail gateway, I install dovecot-lmtp and use it to deliver locally spam. 2. On the mailbox server, I configure special use junk folder to point mail gateway's spam folder via nfs. If this is correct, how can I configure special use folder to point to a path (variable based on email) which is different from default path? If there is a different way to handle this, I would interested. Thanks in advance

3 2

dovecot debug message help required
by Rajesh M 13 Apr '15

13 Apr '15

hi i was carrying out some testing using qmailtoaster with dovecot on a centos 64 server i am connecting to the mailbox using a windows machine with outlook across the lan. the mailbox is around 7gb in size with around 12000 emails spread across multiple folders and i am trying to sync the mailbox folders to by creating an account imap account in outlook. the mails are getting downloaded in the imap folder but the sync does not complete and outlook throws an error. Task 'Synchronizing subscribed folders for user(a)mydomain.com.' reported error (0x800CCC0F) : 'Outlook cannot synchronize subscribed folders for user(a)mydomain.com.. Error: The TCP/IP connection was unexpectedly terminated by the server. If you continue to receive this message, contact your server administrator or Internet service provider (ISP).' in the dovecot log i see the following. Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: Effective uid=89, gid=89, home=/home/vpopmail/domains/mydomain.com/user Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: Quota root: name=ignore=Trash backend=maildir args= Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: Quota rule: root=ignore=Trash mailbox=? bytes=0 messages=0 Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: Quota grace: root=ignore=Trash bytes=0 (10%) Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: Namespace : type=private, prefix=, sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location= Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: mdbox: access(/home/vpopmail/domains/mydomain.com/user/mdbox, rwx): failed: No such file or directory Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: mdbox: couldn't find root dir Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: sdbox: access(/home/vpopmail/domains/mydomain.com/user/sdbox, rwx): failed: No such file or directory Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: sdbox: couldn't find root dir Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: maildir: root exists (/home/vpopmail/domains/mydomain.com/user/Maildir) Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: maildir++: root=/home/vpopmail/domains/mydomain.com/user/Maildir, index=, indexpvt=, control=, inbox=/home/vpopmail/domains/mydomain.com/user/Maildir, alt= Apr 10 17:12:40 imap(user(a)mydomain.com): Info: Disconnected: Logged out in=28 out=460 i would like to know what the following means -- specifically mdbox and sdbox #################### Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: mdbox: access(/home/vpopmail/domains/mydomain.com/user/mdbox, rwx): failed: No such file or directory Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: mdbox: couldn't find root dir Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: sdbox: access(/home/vpopmail/domains/mydomain.com/user/sdbox, rwx): failed: No such file or directory Apr 10 17:12:10 imap(user(a)mydomain.com): Debug: sdbox: couldn't find root dir #################### my dovecot config file # dovecot -n # 2.2.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-431.el6.x86_64 x86_64 CentOS release 6.5 (Final) auth_cache_size = 32 M auth_mechanisms = plain login digest-md5 cram-md5 default_login_user = vpopmail disable_plaintext_auth = no first_valid_gid = 89 first_valid_uid = 89 log_path = /var/log/dovecot.log login_greeting = Dovecot toaster ready. mail_debug = yes mail_max_userip_connections = 50 mail_plugins = " quota" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = prefix = separator = . type = private } passdb { args = cache_key=%u webmail=127.0.0.1 driver = vpopmail } plugin { quota = maildir:ignore=Trash quota_rule = ?:storage=0 } protocols = imap pop3 service imap-login { client_limit = 1000 process_limit = 400 process_min_avail = 4 service_count = 0 vsz_limit = 512 M } service pop3-login { client_limit = 1000 process_limit = 400 process_min_avail = 12 service_count = 0 vsz_limit = 512 M } ssl_cert = </var/qmail/control/servercert.pem ssl_dh_parameters_length = 2048 ssl_key = </var/qmail/control/servercert.pem userdb { args = cache_key=%u quota_template=quota_rule=*:backend=%q driver = vpopmail } protocol imap { imap_client_workarounds = delay-newmail mail_plugins = " quota imap_quota" } protocol pop3 { pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_fast_size_lookups = yes pop3_lock_session = no pop3_no_flag_updates = yes } thank you, rajesh

2 1

Disabling of userdb/passdb modules using config statements
by Jeroen Massar 13 Apr '15

13 Apr '15

Hola, Debian (and possibly other distros) use the /etc/dovecot/conf.d/* setup where default config files are stuffed and then one can just add a 99-myconfig.conf et voila, variables are overruled. This allows the distro to supply updates to the files at package upgrade time without any/much user intervention. The problem (for me ;) is that the system comes provided with: auth-system.conf.ext containing: passdb { driver = pam } userdb { driver = passwd } Hence pam & /etc/passwd based are always enabled. This while I don't have any local users. Replication seems to then always pick up the local users, which are vmail + nobody (65536). doveadm user '*' thus reports vmail, nobody + virtual users Setting: first_valid_uid = 5000 last_valid_uid = 5000 only keeps vmail in there, but apparently some module (guess replication) is still able to figure out that 'nobody' exists: Apr 10 09:48:25 mail dovecot: doveadm(IPADDR,nobody): Error: Mail access for users with UID 65534 not permitted (see first_valid_uid in config file, uid from userdb lookup). Apr 10 09:48:25 mail dovecot: doveadm(IPADDR,nobody): Error: dsync-server: User init failed Apr 10 09:49:38 mail dovecot: doveadm(nobody): Error: sync: Failed to start remote dsync-server command: Remote exit_code=75 and on the other side: Apr 10 09:54:38 mail dovecot: doveadm(nobody): Error: sync: Unknown user in remote This can be resolved by commenting out the entries in auth-system.conf.ext but then I'll have to do that again at package upgrade time. Hence, would it be a cool option to be able (in the 99-myconfig.conf) file to put: passdb { driver = pam enabled = false } userdb { driver = passwd enabled = false } And thereby disabling those modules completely? Thus avoiding upgrade conflicts etc. Greets, Jeroen

4 7

doveadm save ...
by Hardy Flor 13 Apr '15

13 Apr '15

Hello, when I have with "doveadm save ..." store a message, how do I get the guid or uid of this message for the setthe flags? Hardy

4 8

dovecot-2.2.16 compilation problems under Solaris Studio 11
by Jürgen Obermann 13 Apr '15

13 Apr '15

Hello here are two problems I get when I compile dovecot 2.2.16 under Solaris 10 with Studio 11 Compiler: gmake[4]: Entering directory `/net/fileserv/export/sunsrc/src/dovecot-2.2.16/src/lib' source='test-array.c' object='test_lib-test-array.o' libtool=no \ DEPDIR=.deps depmode=none /bin/bash ../../depcomp \ cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib-test -I/opt/local/include -I/usr/local/include -fast -xarch=v8plusa -I/opt/local/include -c -o test_lib-test-array.o `test -f 'test-array.c' || echo './'`test-array.c "test-array.c", line 129: warning: initializer does not fit or is out of range: -32768 "test-array.c", line 129: warning: initializer does not fit or is out of range: -16384 "test-array.c", line 129: warning: initializer does not fit or is out of range: -512 "test-array.c", line 129: warning: initializer does not fit or is out of range: -256 "test-array.c", line 129: warning: initializer does not fit or is out of range: -128 "test-array.c", line 129: warning: initializer does not fit or is out of range: -64 "test-array.c", line 129: warning: initializer does not fit or is out of range: -2 "test-array.c", line 129: warning: initializer does not fit or is out of range: -1 "test-array.c", line 154: operand cannot have void type: op "=" "test-array.c", line 154: assignment type mismatch: unsigned short "=" const void "test-array.c", line 206: operand cannot have void type: op "=" "test-array.c", line 206: assignment type mismatch: pointer to const char "=" const void cc: acomp failed for test-array.c gmake[4]: [test_lib-test-array.o] Fehler 2 (ignoriert) gmake[3]: Entering directory `/net/fileserv/export/sunsrc/src/dovecot-2.2.16/src/lib-imap' source='imap-match.c' object='imap-match.lo' libtool=yes \ DEPDIR=.deps depmode=none /bin/bash ../../depcomp \ /bin/bash ../../libtool --tag=CC --mode=compile cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-test -I../../src/lib-charset -I../../src/lib-mail -I/opt/local/include -I/usr/local/include -fast -xarch=v8plusa -I/opt/local/include -c -o imap-match.lo imap-match.c libtool: compile: cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-test -I../../src/lib-charset -I../../src/lib-mail -I/opt/local/include -I/usr/local/include -fast -xarch=v8plusa -I/opt/local/include -c imap-match.c -KPIC -DPIC -o .libs/imap-match.o "imap-match.c", line 199: identifier redeclared: imap_match_dup current : function(pointer to struct pool {pointer to const struct pool_vfuncs {..} v, unsigned int alloconly_pool :1, unsigned int datastack_pool :1}, pointer to const struct imap_match_glob {pointer to struct pool {..} pool, pointer to struct imap_match_pattern {..} patterns, char sep, array[-1] of char patterns_data}) returning pointer to struct imap_match_glob {pointer to struct pool {..} pool, pointer to struct imap_match_pattern {..} patterns, char sep, array[-1] of char patterns_data} previous: function(pointer to struct pool {pointer to const struct pool_vfuncs {..} v, unsigned int alloconly_pool :1, unsigned int datastack_pool :1}, pointer to const struct imap_match_glob {pointer to struct pool {..} pool, pointer to struct imap_match_pattern {..} patterns, char sep, array[-1] of char patterns_data}) returning pointer to struct imap_match_glob {pointer to struct pool {..} pool, pointer to struct imap_match_pattern {..} patterns, char sep, array[-1] of char patterns_data} : "imap-match.h", line 33 "imap-match.c", line 214: identifier redeclared: imap_match_globs_equal current : function(pointer to const struct imap_match_glob {pointer to struct pool {..} pool, pointer to struct imap_match_pattern {..} patterns, char sep, array[-1] of char patterns_data}, pointer to const struct imap_match_glob {pointer to struct pool {..} pool, pointer to struct imap_match_pattern {..} patterns, char sep, array[-1] of char patterns_data}) returning _Bool previous: function(pointer to const struct imap_match_glob {pointer to struct pool {..} pool, pointer to struct imap_match_pattern {..} patterns, char sep, array[-1] of char patterns_data}, pointer to const struct imap_match_glob {pointer to struct pool {..} pool, pointer to struct imap_match_pattern {..} patterns, char sep, array[-1] of char patterns_data}) returning _Bool : "imap-match.h", line 36 cc: acomp failed for imap-match.c gmake[3]: [imap-match.lo] Fehler 1 (ignoriert) Greetings Jürgen -- Hochschulrechenzentrum der | Mail: Juergen.Obermann(a)hrz.uni-giessen.de Justus-Liebig-Universitaet | WWW: http://www.uni-giessen.de/obermann/ Heinrich-Buff-Ring 44 | Tel: 0641-99-13054 (0641-99-13001) D-35392 Giessen, Germany | Fax: 0641-99-13009

4 4

sieve plugin editheader not found
by Patrick Lion 11 Apr '15

11 Apr '15

Hey folks, ive already made a bug report in the gentoo bugtracker last year. In the meantime after some releases the error persists with the current dovecot and pigeonhole versions. Please see my bug report at gentoo mailinglist: https://bugs.gentoo.org/show_bug.cgi?id=522148 Okay, here are some informations: Steps to Reproduce: add "editheader" to sieve options or "sievec" a file that need the extension if header :matches "Something" "*" { addheader "Anything" "${1}"; deleteheader "Something"; } sievec mod_header.sieve sievec(root): Fatal: Plugin 'editheader' not found from directory /usr/lib64/dovecot/sieve dovecot --version 2.2.16 pigeonhole-0.4.7 sources got downloaded and used in the build process dovecot -n # 2.2.16: /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.7 # OS: Linux 2.6.32-openvz-042stab102.9-amd64 x86_64 Gentoo Base System release 2.2 auth_mechanisms = plain login auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no imap_capability = CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=ORDEREDSUBJECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED CONTEXT=SEARCH LIST-STATUS SPECIAL-USE XLIST QUOTA info_log_path = syslog login_log_format_elements = user=<%u> method=%m ip=%r mpid=%e mail_location = maildir:%h/.maildir mail_plugins = quota managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include body enotify environment mailbox date index ihave duplicate imapflags notify namespace compat { alias_for = hidden = yes inbox = no list = no location = prefix = INBOX. separator = . } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { args = /etc/dovecot/dovecot-sql.conf.ext quota = maildir quota_rule2 = trash:storage=+150M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +notify +imapflags sieve_global_extensions = variables } postmaster_address = mailer-daemon(a)dev.someserver service auth { unix_listener /var/spool/postfix/private/auth { group = mail mode = 0660 user = postfix } unix_listener auth-userdb { group = mail mode = 0660 user = postfix } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service quota-warning { executable = script /etc/postfix/quota-warning.sh unix_listener quota-warning { group = hosting mode = 0660 user = postfix } user = postfix } ssl_cert = </etc/ssl/dovecot/server.pem ssl_key = </etc/ssl/dovecot/server.key userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lda { mail_plugins = quota sieve postmaster_address = mailer-daemon(a)dev.someserver } protocol imap { mail_plugins = quota imap_quota } protocol pop3 { mail_plugins = quota } protocol lmtp { hostname = dev.someserver mail_plugins = quota sieve postmaster_address = postmaster(a)dev.someserver } Please let me know if you need any additional informations, build logs ( log from 2.213 attached at bugs.gentoo.org ) or whatever may be helpful to track this down.

3 4