December 2013 - dovecot

[Dovecot] [dovecot-2.2.5] Duplicates in mdbox
by radek 23 Dec '13

23 Dec '13

Hi I have strange situation with one of mailboxes in mdbox format. Some messages are duplicated, not all. I am sure that is not sieve or another mail client issue, because access is via roundcube without sieve plugin. In maillog I have not found any suspicious information about this problem and i can see that from postfix do dovecot-lda was piped only one copy of message. Radek

2 1

[Dovecot] Horde + dovecot metadata support
by Andreas Schulze 23 Dec '13

23 Dec '13

Hello, Horde (http://horde.org) has the ability to store its preferences direct into a users imap mailbox using imap metadata extension. There are two implementations of the imap extension. 1. a dovecot plugin: http://hg.dovecot.org/dovecot-metadata-plugin 2. dovecot itself: http://www.dovecot.org/list/dovecot/2013-November/093243.html I setup Horde and dovecot-2.2.5 to use this extension and it works well using 1.) Now, with dovecot-2.2.9, the plugin fail to compile (unrelated problem on my side). So I tried 2.) But here I get a Horde-Error "Error in IMAP command GETMETADATA: Invalid arguments." I enables mail_debug but saw no helpfull logging. mail_attribute_dict = file:/tmp/%Lu/dovecot-attributes remote horde.testserver.ip/32 { imap_metadata = yes mail_debug = yes } Also I created directory /tmp/$my_login_id (mode 777) but dovecot also created no files. neither it logs anything about it. This is a sample imap session generated by horde: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] Dovecot ready. 1 AUTHENTICATE PLAIN *** 1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE METADATA QUOTA] Logged in 2 NAMESPACE * NAMESPACE (("" ".")) NIL NIL 2 OK Namespace completed. 3 LIST () "" (*) * LIST () "." INBOX 3 OK List completed. 4 GETMETADATA "*" (/shared/vendor/kolab/folder-type) 4 BAD Error in IMAP command GETMETADATA: Invalid arguments. 5 LOGOUT * BYE Logging out 5 OK Logout completed. Are there any hints or is the metadata extension just not fully implemented in dovecot? Thanks, Andreas -- Andreas Schulze Internetdienste | P252 DATEV eG 90329 Nürnberg | Telefon +49 911 319-0 | Telefax +49 911 319-3196 E-Mail info @datev.de | Internet www.datev.de Sitz: 90429 Nürnberg, Paumgartnerstr. 6-14 | Registergericht Nürnberg, GenReg Nr.70 Vorstand Prof. Dieter Kempf (Vorsitzender) Dipl.-Kfm. Wolfgang Stegmann (stellvertretender Vorsitzender) Dipl.-Kfm. Michael Leistenschneider Dipl.-Kfm. Dr. Robert Mayr Jörg Rabe v. Pappenheim Dipl.-Vw. Eckhard Schwarzer Vorsitzender des Aufsichtsrates: Reinhard Verholen

3 3

[Dovecot] auth: Error: net_accept() failed: Too many open files
by boyd yang 23 Dec '13

23 Dec '13

Hi All, I got below errors when I do some 2000 users concurrency test against dovecot 2.1.7. Dec 23 10:18:52 auth: Error: net_accept() failed: Too many open files Dec 23 10:18:52 auth: Error: net_accept() failed: Too many open files Dec 23 10:18:52 auth: Error: net_accept() failed: Too many open files Dec 23 10:18:52 auth: Error: net_accept() failed: Too many open files Dec 23 10:18:52 auth: Error: net_accept() failed: Too many open files Dec 23 10:18:52 auth: Error: net_accept() failed: Too many open files Dec 23 10:18:52 auth: Error: net_accept() failed: Too many open files Dec 23 10:18:52 auth: Error: net_accept() failed: Too many open files Dec 23 10:18:52 auth: Error: net_accept() failed: Too many open files Dec 23 10:18:52 auth: Error: net_accept() failed: Too many open files Dec 23 10:18:52 auth: Error: net_accept() failed: Too many open files I seems the auth process become the bottleneck as there is only one auth process. service auth-worker { process_limit = 1000 } service imap-login { process_min_avail = 48 service_count = 0 } service imap { process_limit = 10240 process_min_avail = 8 } I find that the auth process does not open too many files with lsof, and my limit is very big(1000000). How to fix this? Thanks!

1 0

[Dovecot] move mailbox to LAYOUT=fs
by Andreas Schulze 23 Dec '13

23 Dec '13

Hello, I think about migrating small servers storage to LAYOUT=fs (http://wiki2.dovecot.org/MailLocation/Maildir#Directory_layout) Is there a smooth way to adjust the existing folders? Andreas

1 0

[Dovecot] TLS SNI with Dovecot
by Michael Neurohr 23 Dec '13

23 Dec '13

Hi! I've been using Dovecot for one single Domain with SSL certificates. Now I would like to use Dovecot with several Domains and proper SSL certificates. I tried to setup TLS SNI but it does not work. What I basically did was just adding local_name imap.samsoft.at { ssl_cert = </etc/ssl/private/mailserver/imap.samsoft.at.crt ssl_key = </etc/ssl/private/mailserver/imap.samsoft.at.key } for the additional domain. When trying to login via IMAP, I see the following lines in the log file: =================================================================== Dec 22 21:01:05 mx0 dovecot: imap-login: Warning: SSL alert: where=0x4004, ret=554: fatal bad certificate [151.236.5.22] Dec 22 21:01:05 mx0 dovecot: imap-login: Warning: SSL failed: where=0x2002: SSLv3 read client certificate A [151.236.5.22] Dec 22 21:01:05 mx0 dovecot: imap-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=151.236.5.22, lip=151.38.7.25, TLS handshaking: SSL_accept() failed: error:14094412:SSL routines:SSL3_READ_BYTES:sslv3 alert bad certificate: SSL alert number 42, session=<Rg5EzyXutgCX7AaF> =================================================================== The actual Dovecot config is pasted below. How can I solve that and server different certificates on the same IP Address? Thanks, Michael doveconf -n # 2.2.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-042stab076.8 i686 CentOS release 6.5 (Final) simfs auth_mechanisms = plain login mail_location = maildir:/var/mail/vhosts/%d/%n mail_plugins = notify mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { antispam_backend = dspam antispam_dspam_args = --source=error;--signature=%%s;--user;%u antispam_dspam_binary = /usr/bin/dspam antispam_signature = X-DSPAM-Signature antispam_signature_missing = move antispam_spam = Spam antispam_trash = trash;Trash;Deleted Items;Deleted Messages sieve = /var/mail/vhosts/sieve-scripts/%d/%n/.dovecot.sieve sieve_before = /var/mail/vhosts/sieve-scripts/before sieve_dir = /var/mail/vhosts/sieve-scripts/%d/%n/sieve sieve_extensions = +imapflags } postmaster_address = postmaster(a)changed.at protocols = imap pop3 lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } user = dovecot } service dict { unix_listener dict { user = vmail } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } ssl = required ssl_ca = </etc/ssl/private/mailserver/ca-bundle.crt ssl_cert = </etc/ssl/private/mailserver/mx0.domain1.at.pem ssl_key = </etc/ssl/private/mailserver/mx0.domain1.at.key userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol lmtp { mail_plugins = sieve } protocol lda { mail_plugins = sieve } protocol imap { mail_plugins = antispam } local_name imap.samsoft.at { ssl_cert = </etc/ssl/private/mailserver/imap.samsoft.at.crt ssl_key = </etc/ssl/private/mailserver/imap.samsoft.at.key }

3 3

[Dovecot] Duplicated (but only for unread) messages after second rsync pass...
by Charles Marcus 22 Dec '13

22 Dec '13

Ok, I know I should probably be using dsync for this, but I'm more familiar with rsync, and didn't think this would be a problem. Here is what I am doing, and what is apparently happening (only done this once, but I'm guessing it will happen each time): rsync -avHP /mnt/vmail/example.com/user1/ /var/vmail/example.com/user1/ chown -R vmail:vmail /var/vmail/example.com Ok, so far so good, everything works fine accessing the mails (I'm using Thunderbird). Now I do another rsync, but this time I did the entire mailstore: rsync -avHP /mnt/vmail/example.com/ /var/vmail/example.com/ chown -R vmail:vmail /var/vmail/example.com Right after I started up dovecot again, I hot a bunch of these in the logs: 2013-12-22T09:52:51-05:00 newhost dovecot: imap(user1(a)example.com): Warning: Maildir /var/vmail/example.com/user1/: Expunged message reappeared, giving a new UID (old uid=45516, file=1387468268.Vfe02Ic1343fM845754.oldhost.sub.example.com:2,ST) 2013-12-22T09:52:51-05:00 newhost dovecot: imap(user1(a)example.com): Warning: Maildir /var/vmail/example.com/user1/: Expunged message reappeared, giving a new UID (old uid=45518, file=1387468645.Vfe02Ic73e0eM530539.oldhost.sub.example.com:2,ST) The problem is, it looks like every single message that was flagged as unread is now duplicated on the new system. Also - not that this is a problem, but all of the old messages have the fqdn of the old server in the individual email file names - oldhost.sub.example.com - while the new ones only have the local hostname (newhost) Did I do something wrong? Or is this going to be an unavoidable problem if I want to use rsync to do the final migration that using dsync would avoid? Thanks, -- Best regards, */Charles /*

3 4

[Dovecot] Dovecot proxying with LDAP - an example?
by Graham Leggett 22 Dec '13

22 Dec '13

Hi all, I have been struggling trying to understand the syntax to configure dovecot as a proxy against an LDAP server. As a single server dovecot works great. Now I want to add the "mailHost" attribute which, if present, should indicate the DNS name of the host that contains the person's mailbox. If the mailHost matches the local hostname, the mailbox should be served locally as it already does now. If the mailHost is not the local hostname, dovecot should proxy the request to the given server using SSL on port 993. The docs at http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/Proxy suggest that proxy_always is what I am looking for, but I cannot find any worked examples of how this might work. Has anyone attempted this before? What I have so far is this, but no proxying seems to take place: pass_attrs = mail=user,userPassword=password,\ mailMessageStore=userdb_mail,=proxy_always=y,mailHost=host,=ssl=yes Can anyone confirm? I am using dovecot v2.0.9 as provided out the box by RHEL6. Regards, Graham --

1 0

[Dovecot] 'unknown user' using dovecot LDA
by Charles Marcus 22 Dec '13

22 Dec '13

Ok, one more issue to resolve. The old server was still using the postfix/virtual for delivery, but the new one is using the dovecot LDA. Now, when an email generated locally by a cron job is delivered, this shows in the log: 2013-12-22T10:29:55-05:00 host postfix/pickup[31400]: C67FD90F676B2: uid=0 from=<newsrv+rkhunter(a)example.com> 2013-12-22T10:29:55-05:00 host postfix/cleanup[22349]: C67FD90F676B2: message-id=<20131222152955.C67FD90F676B2(a)smtp2.example.com> 2013-12-22T10:29:55-05:00 host postfix/qmgr[31401]: C67FD90F676B2: from=<newsrv+rkhunter(a)example.com>, size=1555, nrcpt=1 (queue active) 2013-12-22T10:29:55-05:00 host dovecot: auth-worker(22365): passwd(user1(a)example.com): unknown user 2013-12-22T10:29:55-05:00 host dovecot: lda(user1(a)example.com): msgid=<20131222152955.C67FD90F676B2(a)smtp2.example.com>: saved mail to INBOX 2013-12-22T10:29:55-05:00 host postfix/pipe[22361]: C67FD90F676B2: to=<user1+rkhunter-report(a)example.com>, relay=dovecot, delay=0.06, delays=0.02/0/0/0.04, dsn=2.0.0, status=sent (delivered via dovecot service) 2013-12-22T10:29:55-05:00 host postfix/qmgr[31401]: C67FD90F676B2: removed The mail was delivered (as the log also clearly shows), but what is with the 'unknown user'? I'm not sure how I can test my maps with dovecot, but testing with postfix shows: # postmap -q user1(a)example.com mysql:/etc/postfix/maps/mysql/vmm.cf example.com/user1/ Which is correct. And my dovecot-sql-conf.ext contains: driver = mysql connect = "host=localhost dbname=db_name user=dbuser password=password" default_pass_scheme = CRYPT password_query = \ SELECT username AS user, password \ FROM mailbox WHERE username = '%u' user_query = \ SELECT CONCAT('/var/vmail/', maildir) AS home \ FROM mailbox WHERE username = '%u' iterate_query = SELECT username AS user FROM mailbox I confirmed the db info is identical to what postfix is using. Oh - and in order to make sure that all usernames are lowercased, shouldn't I change '%u' above to '%Lu'? Appreciate any suggestions on where to go next... thanks -- Best regards, */Charles/*

3 4

[Dovecot] New server missing auth-worker entries in logs
by Charles Marcus 22 Dec '13

22 Dec '13

Hi all, Ok, I'm making progress on my new server, but there is an entry in the logs on the old/current server that is not showing p on the new one... lines like: dovecot: auth-worker(20284): mysql(localhost): Connected to database my_auth_db I've looked everywhere, compared settings, etc, and can't seem to find why I get this line on the old server and odn't on the new one. Anyone have a clue stick? Thanks

2 2

[Dovecot] File descriptor leak in sieve-extprograms
by Kirill A. Shutemov 21 Dec '13

21 Dec '13

I've tried to use sieve-filter for my inbox (~6000 messages). It failed after some work with this backtrace: sieve-filter(kas): Error: socketpair() failed: Too many open files sieve-filter(kas): Panic: file script-client-local.c: line 155 (script_client_local_disconnect): assertion failed: (pid >= 0) sieve-filter(kas): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x66a71) [0x7f020e717a71] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x2a) [0x7f020e717ada] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f020e6d32ee] -> /usr/lib/dovecot/modules/sieve/lib90_sieve_extprograms_plugin.so(+0x4baf) [0x7f020dae0baf] -> /usr/lib/dovecot/modules/sieve/lib90_sieve_extprograms_plugin.so(+0x58ca) [0x7f020dae18ca] -> /usr/lib/dovecot/modules/sieve/lib90_sieve_extprograms_plugin.so(script_client_fail+0x2f) [0x7f020dae199f] -> /usr/lib/dovecot/modules/sieve/lib90_sieve_extprograms_plugin.so(script_client_run+0xc4) [0x7f020dae1fd4] -> /usr/lib/dovecot/modules/sieve/lib90_sieve_extprograms_plugin.so(+0x6134) [0x7f020dae2134] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_result_execute+0x3ac) [0x7f020ecd1adc] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x4a) [0x7f020ece05aa] -> /usr/bin/sieve-filter(main+0x6a9) [0x403b49] -> /usr/lib/libc.so.6(__libc_start_ma in+0xf5) [0x7f020e327bc5] -> /usr/bin/sieve-filter() [0x404111] The reason is file descriptor leak in "script client local". script_client_local_close_output() shutdowns write side of descriptor and sets sclient->fd_out to -1, but never closes the descriptor. I've tried to fix this by the patch below. Tests works fine, but sieve-filter crashed in other way: sieve-filter(kas): Panic: epoll_ctl(del, 13) failed: Bad file descriptor sieve-filter(kas): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x66a71) [0x7f0ed406fa71] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x2a) [0x7f0ed406fada] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f0ed402b2ee] -> /usr/lib/dovecot/libdovecot.so.0(+0x776f5) [0x7f0ed40806f5] -> /usr/lib/dovecot/libdovecot.so.0(+0x8009a) [0x7f0ed408909a] -> /usr/lib/dovecot/libdovecot.so.0(io_stream_unref+0x25) [0x7f0ed40764c5] -> /usr/lib/dovecot/libdovecot.so.0(o_stream_unref+0x26) [0x7f0ed4088136] -> /usr/lib/dovecot/libdovecot.so.0(+0x80e48) [0x7f0ed4089e48] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x27) [0x7f0ed407fa47] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xd7) [0x7f0ed40807d7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f0ed407f5e8] -> /usr/lib/dovecot/modules/sieve/lib90_sieve_extprograms_plugin.so(script_client_run+0x9c) [0x7f0ed343af8c] -> /usr/lib/dovecot/modules/sieve/lib90_sieve_extprograms_plugin.so(+0x6114) [0x 7f0ed343b114] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_result_execute+0x3ac) [0x7f0ed4629adc] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x4a) [0x7f0ed46385aa] -> /usr/bin/sieve-filter(main+0x6a9) [0x403b49] -> /usr/lib/libc.so.6(__libc_start_main+0xf5) [0x7f0ed3c7fbc5] -> /usr/bin/sieve-filter() [0x404111] I failed to unfold this in a reasonable time. Any ideas? diff --git a/src/plugins/sieve-extprograms/script-client-local.c b/src/plugins/sieve-extprograms/script-client-local.c index 56d32f707c09..f54ef82132cf 100644 --- a/src/plugins/sieve-extprograms/script-client-local.c +++ b/src/plugins/sieve-extprograms/script-client-local.c @@ -139,7 +139,6 @@ static int script_client_local_close_output(struct script_client *sclient) i_error("shutdown(%s, SHUT_WR) failed: %m", sclient->path); return -1; } - sclient->fd_out = -1; return 1; } diff --git a/src/plugins/sieve-extprograms/script-client.c b/src/plugins/sieve-extprograms/script-client.c index 06a5d5d54ba9..6fefca4c9f26 100644 --- a/src/plugins/sieve-extprograms/script-client.c +++ b/src/plugins/sieve-extprograms/script-client.c @@ -68,7 +68,8 @@ static void script_client_disconnect if (sclient->fd_in != -1 && close(sclient->fd_in) < 0) i_error("close(%s) failed: %m", sclient->path); - if (sclient->fd_out != -1 && sclient->fd_out != sclient->fd_out) + if (sclient->fd_out != -1 && sclient->fd_out != sclient->fd_in + && close(sclient->fd_out) < 0) i_error("close(%s/out) failed: %m", sclient->path); sclient->fd_in = sclient->fd_out = -1; -- Kirill A. Shutemov

3 6