March 2012 - dovecot - dovecot.org

[Dovecot] Zero-sized Maildir files and index corruption on over-quota
by Edgar Fuß 28 Jun '13

28 Jun '13

We are using Postfix as an MTA delivering via Dovecot's LDA (with sieve). We also use Dovecot as a POP/IMAP server. Mail storage is Maildir on NFS, indexes are stored locally. Quotas are FS quotas enforced by the NFS server. The Dovecot version is 1.2.11. Recently, for one user being over quota, some attempts to deliver mail bounced as expected while some others resulted in zero-sized Maildir files. I can verify zero-sized files in cur, while I cannot tell whether they were already zero-sized in new. However, since nobody except Dovecot is accessing the mail storage, it's either Dovecot's LDA or Dovecot's IMAP producing the zero-sized files. Logs show successful delivery (deliver: stored mail into mailbox 'INBOX') but ,,Corrupted index cache file /var/db/dovecot/indexes/XXX/.INBOX/dovecot.index.cache: Corrupted virtual size for uid=YYY: 0 != NNNN'' alerts on IMAP access. It looks like all mails which correctly got bounced are >64KB while those resulting in zero-sized files are <64KB (there is one >32KB). My first guess would be that maybe for large files, write() fails, while for small ones, close() fails or there is some buffering involved for large files.

3 10

[Dovecot] traffic statistics
by Martin Waschbüsch 28 Jun '13

28 Jun '13

Hi all, Does anyone know a good tool to gather traffic statistics for imap? And what level of logging would I need to set for dovecot to make it happen. Ideally this would be able to filter per domain, too. ;-) Thanks a lot, Martin -- Martin Waschbüsch IT-Dienstleistungen Lautensackstr. 16 80687 München Telefon: +49 89 57005708 Fax: +49 89 57868023 Mobil: +49 170 2189794 martin(a)waschbuesch.de http://martin.waschbuesch.de

4 5

[Dovecot] Released Pigeonhole v0.2.4 for Dovecot v2.0.14
by Stephan Bosch 26 Mar '13

26 Mar '13

Hello Dovecot users, I release a new version of Pigeonhole tonight. This release extends the configuration capabilities of the include and vacation extensions, adds the ihave extension (RFC 5463), and fixes a few small other things. Judging by Timo's plans, this will be one of the last feature releases for Dovecot v2.0, since I'll be switching active development to Dovecot v2.1 once that matures a little more. Changelog v0.2.4: + Vacation extension: finally added support for using the original recipient in vacation address check. It is also possible to disable the recipient address check entirely. Check doc/vacation.txt for configuration information. + Include extension: made limits on the include depth and the total number of included scripts configurable. Check doc/include.txt for configuration information. + Implemented ihave extension. This allows checking for the availability of Sieve language extensions at 'runtime'. Actually, this is checked at compile time. At runtime the interpreter checks whether extensions that were not previously available are still unavailable. If the situation changed, the script is re-compiled and the ihave tests are evaluated again. + Sieve: optimized compilation of tests that yield constant results (i.e. known at compile tme), such as 'true' and 'false'. No code is produced anymore for script sections that are never executed. Also, semantics are not verified anymore in uncompiled script sections. + Made vnd.dovecot.debug extension available to the LDA plugin instead of only the command line tools. + Sieve: redirect action now adds X-Sieve-Redirected-From header (mainly for people using SPF/SRS). - Sieve: fixed bug in handling flags and keywords; in case of error an assertion was triggered. - Script storage: improved handling of unconfigured user home directory. Originally this would produce an unhelpful error message. - Imap4flags extension: prevent forcibly enabling imap4flags when imapflags is enabled. - Fixed various -Wunused-but-set-variable compiler warnings. - Include extension: forgot to check variable identifier syntax for 'global' command. - Sieve: fixed debug mode; no messages were logged in some situations. - sievec tool: forgot to enable -D (debug) parameter. The release is available as follows: http://www.rename-it.nl/dovecot/2.0/dovecot-2.0-pigeonhole-0.2.4.tar.gz http://www.rename-it.nl/dovecot/2.0/dovecot-2.0-pigeonhole-0.2.4.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.0 wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan(a)rename-it.nl

2 1

[Dovecot] Auth worker max line size
by Emmanuel Dreyfus 09 Sep '12

09 Sep '12

Hello I have been playing with some exotic authentication scheme with Dovecot and PAM. That involves sending really large base64 encoded data as the IMAP password, and I have hit a line limit in Dovecot, with AUTH_WORKER_MAX_LINE_LENGTH set to 1024. This limit is especially frustrating since other parts of the software use much larger limits: MAX_INBUF_SIZE 4096 MAX_IMAP_LINE 8192 AUTH_CLIENT_MAX_LINE_LENGTH 8192 I had to make the patch attached below to get my authentication working. I can live with this local patch, but given the much more liberal limits of MAX_INBUF_SIZE at 4096 makes we wonder if this 1024 limit on AUTH_WORKER_MAX_LINE_LENGTH could not be a bug. Or is there a security concern at using more than 1kB? Opinions? (please Cc: me, I'm not subscribed ot the list) --- src/auth/auth-worker-client.h.orig 2009-06-23 18:32:15.000000000 +0200 +++ src/auth/auth-worker-client.h 2009-06-23 18:32:33.000000000 +0200 @@ -1,8 +1,8 @@ #ifndef AUTH_WORKER_CLIENT_H #define AUTH_WORKER_CLIENT_H -#define AUTH_WORKER_MAX_LINE_LENGTH 1024 +#define AUTH_WORKER_MAX_LINE_LENGTH 4096 struct auth_worker_client *auth_worker_client_create(struct auth *auth, int fd); void auth_worker_client_destroy(struct auth_worker_client **client); void auth_worker_client_unref(struct auth_worker_client **client); -- Emmanuel Dreyfus manu(a)netbsd.org

3 5

[Dovecot] dsync replication available for testing
by Timo Sirainen 25 Aug '12

25 Aug '12

In dovecot-2.1 hg you can now test dsync-based replication. Everything isn't finished yet, but it appears to work and I've enabled it for my @dovecot.fi mails. Some issues: - public namespace isn't replicated at all - shared namespace is replicated, but not private mail flags - I've only tested SSH replication setup now, not director replication setup (and director setup is still missing many things) - SSH replication setup uses aggregator process, which isn't really necessary and can probably be avoided in future Below is a configuration for virtual user setup. System user configuration works pretty much the same, except doveadm/ssh is run as root. Try first that dsync works successfully with ssh in host1: doveadm sync -u user@domain remote:vmail@host2.example.com and also in host2: doveadm sync -u user@domain remote:vmail@host1.example.com ------ mail_plugins = $mail_plugins notify replication service aggregator { # give enough permissions for mail processes fifo_listener replication-notify-fifo { user = vmail mode = 0600 } unix_listener replication-notify { user = vmail mode = 0600 } } service replicator { # start replication at startup process_min_avail = 1 } plugin { # host1 replicates to host2 mail_replica = remote:vmail@host2.example.com # host2 replicates to host1 #mail_replica = remote:vmail@host1.example.com } #dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u -l%{lock_timeout} -n%{namespace} service doveadm { # if you're using a single virtual user, set this to # start ssh as vmail (not root) user = vmail }

4 20

Re: [Dovecot] problem with virtual plugin/index files?
by Lutz Preßler 24 Aug '12

24 Aug '12

Hello, Timo: on Oct 29th I wrote ---- with latest 2.0.6 (601065674f74), the following virtual mailbox is not behaving correctly. I'm quite sure that the problem has not existed in some earlier 2.0.x (maybe only INBOX as mailbox then though): ~user/Maildir/virtual/.week/dovecot-virtual contains INBOX INBOX.* -INBOX.Trash inthread refs younger 604800 This works as expected for the first time. But later on, also older messages/ threads are included until I delete the virtual/.week/dovecot.index*. ---- Just want to state that this problem still persists with latest version. Any idea? Thanks and greetings, Lutz -- Lutz Preßler <Lutz.Pressler(a)SerNet.DE> http://www.SerNet.DE/ SerNet Service Network GmbH, Bahnhofsallee 1b, D-37081 Göttingen Tel.: +49-551-370000-2, FAX: +49-551-370000-9 AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen

2 4

[Dovecot] mysql auth failover failing
by Paul B. Henson 29 Jul '12

29 Jul '12

We are running dovecot to provide authentication for postfix, using two mysql servers in a multi-master replication set as the password source: ---------------------------------------- # 2.0.13: /etc/dovecot/dovecot.conf # OS: Linux 2.6.37-gentoo-r4 x86_64 Gentoo Base System release 2.0.2 auth_mechanisms = plain login digest-md5 cram-md5 auth_verbose = yes passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocols = none service auth-worker { unix_listener auth-worker { user = postfix } user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = postfix } ssl = no userdb { driver = passwd } --------------------------------------- With an sql config of: ------------------------- driver = mysql connect = host=mysql-1.unx.csupomona.edu host=mysql-2.unx.csupomona.edu dbname=idmgmt user=postfix password=XXXXXXX default_pass_scheme = PLAIN password_query = XXXXXXXXX ------------------------- According to the sample SQL configuration file "HA / round-robin load-balancing is supported by giving multiple host settings, like: host=sql1.host.org host=sql2.host.org". However, as far as I can tell dovecot only connects to the first listed host, and processes all queries through it, there does not appear to be any load-balancing going on. That's not necessarily a dealbreaker; however, high-availability does not appear to be working either. If I shutdown the first mysql server, dovecot starts to log connection failures: Sep 9 15:47:34 tweak dovecot: auth: Error: mysql(mysql-1.unx.csupomona.edu): Connect failed to database (idmgmt): Can't connect to MySQL server on 'mysql-1.unx.csupomona.edu' (111) - waiting for 1 seconds before retry Sep 9 15:47:39 tweak dovecot: auth: Error: mysql(mysql-1.unx.csupomona.edu): Connect failed to database (idmgmt): Can't connect to MySQL server on 'mysql-1.unx.csupomona.edu' (111) - waiting for 25 seconds before retry And postfix starts to fail authentications: Sep 9 15:47:35 tweak postfix/smtpd[5119]: warning: bender.iitsys.csupomona.edu[134.71.250.134]: SASL DIGEST-MD5 authentication failed: Connection lost to authentication server Now and again the authentication process dies: Sep 9 15:47:39 tweak dovecot: auth: Panic: file auth-request-handler.c: line 697 (auth_request_handler_flush_failures): assertion failed: (auth_request->state == AUTH_REQUEST_STATE_FINISHED) Sep 9 15:47:39 tweak dovecot: auth: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x3f71a) [0x7f25822ca71a] -> /usr/lib64/dovecot/libdovecot.so.0(+0x3f766) [0x7f25822ca766] -> /usr/lib64/dovecot/libdovecot.so.0(+0x198ca) [0x7f25822a48ca] -> dovecot/auth() [0x4137f4] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0xd4) [0x7f25822d5fe4] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x5b) [0x7f25822d6bcb] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f25822d5c48] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f25822c3de3] -> dovecot/auth(main+0x2be) [0x4179de] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f2581898bbd] -> dovecot/auth() [0x40bdc9] Sep 9 15:47:39 tweak dovecot: master: Error: service(auth): child 4154 killed with signal 6 (core dumps disabled) Requests start to pile up: Sep 9 15:51:46 tweak dovecot: auth: Warning: auth workers: Auth request was queued for 25 seconds, 45 left in queue Lookups time out: Sep 9 15:57:22 tweak dovecot: auth: Error: auth worker: Aborted request: Lookup timed out This occasionally pops up: Sep 9 15:58:38 tweak dovecot: auth: Fatal: net_connect_unix(auth-worker) failed: Resource temporarily unavailable And sometimes the auth process gets temporarily disabled: Sep 9 15:58:57 tweak dovecot: master: Error: service(auth): command startup failed, throttling Resulting in more postfix authentication failures: Sep 9 15:58:57 tweak postfix/smtpd[6531]: warning: bender.iitsys.csupomona.edu[134.71.250.134]: SASL DIGEST-MD5 authentication failed: Sep 9 15:59:08 tweak postfix/smtpd[6551]: fatal: no SASL authentication mechanisms To the point where postfix also temporarily throttles smtpd: Sep 9 15:59:21 tweak postfix/master[6526]: warning: /usr/lib64/postfix/smtpd: bad command startup -- throttling Resulting in a complete unavailability of smtp service, not just unavailability of authenticated services. I don't think all authentications fail during the scenario, but I think the majority do. Based on the network traffic, dovecot is almost continuously trying to connect to the first listed server. It sometimes connects to the second listed server, but when it does, the connection does not persist, it goes away almost immediately. Ideally, I would like no authentications to fail if one of the MySQL servers is unavailable. If a few fail just when the server dies, that would be undesirable but acceptable as long as they do not continuously fail while the server is down. Am I doing something wrong? Does the example sql config have incorrect information? We were previously running dovecot 1.2.11, we just recently upgraded to 2. In the previous version, we actually had two different passdb's configured, each one listing only one of the mysql servers. I seem to recall that was the recommendation at the time for high-availability. When that configuration did not seem to work under version 2, I found an updated recommendation to list both servers in the same passdb, which also does not appear to work correctly. I actually went back and tested the older version, and determined it seemed to work okay in the case where the server was up but the service was down, and connections were refused, but also failed a large number of authentication attempts when the server was completely down and connections were timing out. Thanks much... -- Paul B. Henson | (909) 979-6361 | http://www.csupomona.edu/~henson/ Operating Systems and Network Analyst | henson(a)csupomona.edu California State Polytechnic University | Pomona CA 91768

5 19

[Dovecot] [PATCH] GSSAPI authorization and virtual users
by Sam Morris 02 Jul '12

02 Jul '12

The attached patch makes it possible for Kerberos principals to be associated with a password database entry by adding a new "k5principals" passdb setting. A client that successfully authenticates using GSSAPI will be able to log in as any user who has been associated with the client's Kerberos principal. This means that users can now use their Kerberos identities to access virtual mail accounts. The patch definitely needs review by someone familiar with Dovecot. It works for me on a small test installation using the passwd-file backend. Things that should probably be improved: 1. The list of authorized principals is stored in struct auth_request. I would prefer to store it in struct gssapi_auth_request, but auth-request.c does not know about structs that are specific to the different authentication plugins. This could be fixed in a more general way by adding a new function to struct mech_module to allow authplugins to read fields during passdb lookups. 2. The gssapi authplugin now does a credential lookup in order to trigger parsing of the k5principals setting. In order for this to work, auth_request_set_username is now called before mech_gssapi_userok. AFAICT the only impact of this is that messages logged by this function (and the functions it calls) will now use the name of the virtual account. 3. The credentials lookup triggers an info log message saying that credentials for GSSAPI were requested, "but we have only (e.g.) MD5-CRYPT". The authplugin doesn't actually want the credential, but I think that the only way the authplugin can trigger a passdb lookup is by requesting it. 4. The final part of the code in mech_gssapi_unwrap was moved to the callback that's triggered when the credentials lookup is complete. The code still needs access to the GSSAPI data, so the buffer pointer & length are now stored in struct gssapi_auth_request, making the inbuf parameter to the mech_gssapi_{sec_context,wrap,unwrap} functions superfluous. The parameters should be removed. 5. The k5principals list won't be processed on Solaris. The code added to the end of mech_gssapi_krb5_userok would have to be moved to a separate function and then be called from the Solaris code. 6. GCC tells me about assignment to incompatible pointer types in the code that iterates through gssapi_k5principals. I must be missing something. The patch is licensed under the MIT license. Please let me know what you think. -- Sam Morris <sam(a)robots.org.uk>

2 4

[Dovecot] permissions on auth-userdb
by spamvoll＠googlemail.com 28 Jun '12

28 Jun '12

Hi.. im still trying to upgrade to 2.0. Im getting: dovecot: lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: Permission denied (euid=10000(vmail) egid=10000(vmail) missing +r perm: /var/run/dovecot/auth-userdb, euid is not dir owner) the error is correct caus its owned by root. My Questions is who should own it ? Im not sure how that works, what process/user calls the auth-userdb ? The auth-userdb returns the args generated in master.conf, right ? i think comment out the user and group setting in master.conf will fix it but im not sure if that is the securest way. the mails come from postfix via dovecot-lda Hans master.conf service auth { # auth_socket_path points to this userdb socket by default. It's typically # used by dovecot-lda, doveadm, possibly imap process, etc. Its default # permissions make it readable only by root, but you may need to relax these # permissions. Users that have access to this socket are able to get a list # of all usernames and get results of everyone's userdb lookups. unix_listener auth-userdb { mode = 0600 #user = vmail #group = vmail } auth-ldap.conf.ext passdb { driver = ldap args = /etc/dovecot/dovecot-ldap.conf.ext } userdb { driver = static args = uid=vmail gid=vmail home=/home/MAILBOXES/%u/ mail=/home/MAILBOXES/%u/mail }

5 8

[Dovecot] gnutls support
by Daniel L. Miller 12 Jun '12

12 Jun '12

Other than license issues, is there an advantage to using gnutls vs openssl? Or is openssl superior - at least in the current implementations? -- Daniel

2 3