July 2012 - dovecot - dovecot.org

[Dovecot] mysql auth failover failing
by Paul B. Henson 29 Jul '12

29 Jul '12

We are running dovecot to provide authentication for postfix, using two mysql servers in a multi-master replication set as the password source: ---------------------------------------- # 2.0.13: /etc/dovecot/dovecot.conf # OS: Linux 2.6.37-gentoo-r4 x86_64 Gentoo Base System release 2.0.2 auth_mechanisms = plain login digest-md5 cram-md5 auth_verbose = yes passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocols = none service auth-worker { unix_listener auth-worker { user = postfix } user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = postfix } ssl = no userdb { driver = passwd } --------------------------------------- With an sql config of: ------------------------- driver = mysql connect = host=mysql-1.unx.csupomona.edu host=mysql-2.unx.csupomona.edu dbname=idmgmt user=postfix password=XXXXXXX default_pass_scheme = PLAIN password_query = XXXXXXXXX ------------------------- According to the sample SQL configuration file "HA / round-robin load-balancing is supported by giving multiple host settings, like: host=sql1.host.org host=sql2.host.org". However, as far as I can tell dovecot only connects to the first listed host, and processes all queries through it, there does not appear to be any load-balancing going on. That's not necessarily a dealbreaker; however, high-availability does not appear to be working either. If I shutdown the first mysql server, dovecot starts to log connection failures: Sep 9 15:47:34 tweak dovecot: auth: Error: mysql(mysql-1.unx.csupomona.edu): Connect failed to database (idmgmt): Can't connect to MySQL server on 'mysql-1.unx.csupomona.edu' (111) - waiting for 1 seconds before retry Sep 9 15:47:39 tweak dovecot: auth: Error: mysql(mysql-1.unx.csupomona.edu): Connect failed to database (idmgmt): Can't connect to MySQL server on 'mysql-1.unx.csupomona.edu' (111) - waiting for 25 seconds before retry And postfix starts to fail authentications: Sep 9 15:47:35 tweak postfix/smtpd[5119]: warning: bender.iitsys.csupomona.edu[134.71.250.134]: SASL DIGEST-MD5 authentication failed: Connection lost to authentication server Now and again the authentication process dies: Sep 9 15:47:39 tweak dovecot: auth: Panic: file auth-request-handler.c: line 697 (auth_request_handler_flush_failures): assertion failed: (auth_request->state == AUTH_REQUEST_STATE_FINISHED) Sep 9 15:47:39 tweak dovecot: auth: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x3f71a) [0x7f25822ca71a] -> /usr/lib64/dovecot/libdovecot.so.0(+0x3f766) [0x7f25822ca766] -> /usr/lib64/dovecot/libdovecot.so.0(+0x198ca) [0x7f25822a48ca] -> dovecot/auth() [0x4137f4] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0xd4) [0x7f25822d5fe4] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x5b) [0x7f25822d6bcb] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f25822d5c48] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f25822c3de3] -> dovecot/auth(main+0x2be) [0x4179de] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f2581898bbd] -> dovecot/auth() [0x40bdc9] Sep 9 15:47:39 tweak dovecot: master: Error: service(auth): child 4154 killed with signal 6 (core dumps disabled) Requests start to pile up: Sep 9 15:51:46 tweak dovecot: auth: Warning: auth workers: Auth request was queued for 25 seconds, 45 left in queue Lookups time out: Sep 9 15:57:22 tweak dovecot: auth: Error: auth worker: Aborted request: Lookup timed out This occasionally pops up: Sep 9 15:58:38 tweak dovecot: auth: Fatal: net_connect_unix(auth-worker) failed: Resource temporarily unavailable And sometimes the auth process gets temporarily disabled: Sep 9 15:58:57 tweak dovecot: master: Error: service(auth): command startup failed, throttling Resulting in more postfix authentication failures: Sep 9 15:58:57 tweak postfix/smtpd[6531]: warning: bender.iitsys.csupomona.edu[134.71.250.134]: SASL DIGEST-MD5 authentication failed: Sep 9 15:59:08 tweak postfix/smtpd[6551]: fatal: no SASL authentication mechanisms To the point where postfix also temporarily throttles smtpd: Sep 9 15:59:21 tweak postfix/master[6526]: warning: /usr/lib64/postfix/smtpd: bad command startup -- throttling Resulting in a complete unavailability of smtp service, not just unavailability of authenticated services. I don't think all authentications fail during the scenario, but I think the majority do. Based on the network traffic, dovecot is almost continuously trying to connect to the first listed server. It sometimes connects to the second listed server, but when it does, the connection does not persist, it goes away almost immediately. Ideally, I would like no authentications to fail if one of the MySQL servers is unavailable. If a few fail just when the server dies, that would be undesirable but acceptable as long as they do not continuously fail while the server is down. Am I doing something wrong? Does the example sql config have incorrect information? We were previously running dovecot 1.2.11, we just recently upgraded to 2. In the previous version, we actually had two different passdb's configured, each one listing only one of the mysql servers. I seem to recall that was the recommendation at the time for high-availability. When that configuration did not seem to work under version 2, I found an updated recommendation to list both servers in the same passdb, which also does not appear to work correctly. I actually went back and tested the older version, and determined it seemed to work okay in the case where the server was up but the service was down, and connections were refused, but also failed a large number of authentication attempts when the server was completely down and connections were timing out. Thanks much... -- Paul B. Henson | (909) 979-6361 | http://www.csupomona.edu/~henson/ Operating Systems and Network Analyst | henson(a)csupomona.edu California State Polytechnic University | Pomona CA 91768

5 19

[Dovecot] Enable ACL for master user
by punit_j 29 Jul '12

29 Jul '12

Hi Team, I have dovecot setup and I have enabled master User login in dovecot.conf file as below :- auth_master_user_separator = * passdb { driver = passwd-file args = /etc/dovecot/passwd.masterusers master = yes pass = yes } passdb { driver = shadow } userdb { driver = passwd } and craeted file. I can login to the user but what I see after login is : - Trying 0.0.0.0... Connected to 0. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 SASL-IR SORT THREAD=REFERENCES MULTIAPPEND UNSELECT LITERAL+ IDLE CHILDREN NAMESPACE LOGIN-REFERRALS UIDPLUS LIST-EXTENDED I18NLEVEL=1 STARTTLS AUTH=PLAIN] Dovecot ready. a login user*masteruser password a OK Logged in. a list "" * * LIST (\HasNoChildren) "." "Trash" * LIST (\Unmarked) "." "INBOX" a OK List completed. However when I login as a normal user I see this : - Connected to 0. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 SASL-IR SORT THREAD=REFERENCES MULTIAPPEND UNSELECT LITERAL+ IDLE CHILDREN NAMESPACE LOGIN-REFERRALS UIDPLUS LIST-EXTENDED I18NLEVEL=1 STARTTLS AUTH=PLAIN] Dovecot ready. a login user userpassword a OK Logged in. a list "" * * LIST (\HasNoChildren) "." "Drafts" * LIST (\HasNoChildren) "." "Trash" * LIST (\HasNoChildren) "." "Sent items" * LIST (\HasNoChildren) "." "INBOX" a OK List completed. I think its because of ACL for user and it is using masterusers ACL. I looked at http://wiki2.dovecot.org/PostLoginScripting, however do we have sample login script so that I can use that script with modifications. Sorry I am new to dovecot. Thanks and Regards.

1 0

[Dovecot] lmtp log message
by Daniel L. Miller 28 Jul '12

28 Jul '12

I just tried switching to lmtp from lda. Messages are being delivered, but I now see messages similar to: Jul 26 22:59:21 bubba dovecot: lmtp(20508): Disconnect from local: Client quit (in reset) in my log for every message. Is this a normal sign-off - or does this indicate a misconfiguration? -- Daniel

4 6

[Dovecot] Problem with preserving MIME format with Dovecot IMAP and Outlook
by Peter Grimmond 28 Jul '12

28 Jul '12

Hello, I have installed Dovecot 2.0.19 on Ubuntu 12.04 LTS. I have set up a very basic IMAP configuration. I have two different laptops running Outlook (one MacOS, one Windows 7) and I have configured both to connect to the Dovecot service. My problem is, emails in MIME format that I synchronise from one Outlook client are not being correctly decoded as MIME messages on the other client. Instead, the message is rendered in plaintext, starting with the lines: This message is in MIME format. Since your mail reader does not understandthis format, some or all of this message may not be legible. --B_3426360453_22814Content-type: multipart/alternative; boundary="B_3426360453_4099" --B_3426360453_4099Content-type: text/plain; charset="ISO-8859-1"Content-transfer-encoding: quoted-printable Can anyone help me understand why this might be happening? The output from dovecot -n is: # 2.0.19: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-26-generic-pae i686 Ubuntu 12.04 LTS auth_debug = yes auth_verbose = yes disable_plaintext_auth = no mail_location = maildir:~/Maildir passdb { driver = pam } protocols = " imap" ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { driver = passwd } Thanks! PG ___ Peter Grimmond E:peter@grimmond.org M:+44 7770 963352

1 0

Re: [Dovecot] expire-tool stops on first box with nothing to expire in it
by Joseph Tam 28 Jul '12

28 Jul '12

Michael Wessel writes: > When I run for example: > > doveadm -D search -A mailbox Trash savedbefore 60d > /tmp/discard I previously observed that the "date.saved" field (tested by savedbefore) seems to be clustered around particular timestamps. Dumping out this field using doveadm -ftab fetch -A \ "mailbox date.saved" \ $ mailbox Trash OR mailbox Junk OR mailbox "Deleted Items" $ it seems the timestamps do not actually correspond to when the client moves messages to their trash folder, but rather, to the time I query for it i.e. a query instantiates this value. This timestamp is then retained. In practice, if this query is done on a periodic interval (e.g. daily), the date.saved will be roughly correct, and is usable as an age test for expungement. Is there a way to force this value to be cached when the client actually saves or moves a message? Joseph Tam <jtam.home(a)gmail.com>

2 1

[Dovecot] [PATCH] Pop3/imap ordering in courier-dovecot-migrate.pl
by Pavel Stano 28 Jul '12

28 Jul '12

Hi, current version of courier migration script courier-dovecot-migrate.pl preserve order of POP3 UIDLs, but it keep IMAP UIDs not in order which cause problems with clients using IMAP along with POP3 in some cases. Attached patch use MAIL_FETCH_POP3_ORDER feature added in version 2.0.13 to keep POP3 and IMAP UID[L]s in order. -- [ Ohodnotte kvalitu mailu: http://nicereply.com/websupport/4afafd34/ ] Pavel Stano | admin team http://WebSupport.sk *** TAKE AND ENJOY ***

2 1

Re: [Dovecot] Disable W=<vsize> in filenames
by Daniel Pieper 28 Jul '12

28 Jul '12

I implemented a plugin for dovecot and now i got a error that maildir filename has wrong W value. When i rename the filename manually and delete the W value, all is ok.

2 1

[Dovecot] Dsync replication
by list＠airstreamcomm.net 28 Jul '12

28 Jul '12

I have been loosely following discussions dsync replication, but I am wondering if this tool still in a testing phase or has it been committed as a production part of Dovecot? I would like to do some testing with the protocol to see if it's capable of handling the change rate our mail service generates, which has about 23,000 active users. We are trying to find solutions for a two site setup where mail processing is either being done at the primary data center A and fails over to a hot standby cluster at the secondary data center B, or where both data centers are doing active processing. I have been unable to find documentation on dsync replication on the wiki, so if there is any documentation available on how to setup dsync replication I would appreciate a nudge in the right direction.

4 6

[Dovecot] get recipient entry while in the zlib plugin context
by Tofig Suleymanov 28 Jul '12

28 Jul '12

Hello, is it possible to get a recipient of an email while being in the zlib plugin context ? Thank you ! -----BEGIN PGP PUBLIC KEY BLOCK----- mQINBE9zuXMBEADG8O1eZpo8gxMFcQ2Bf4K/n2MvDoH5WDnffvLQcswkigKedicz lNNQzMTsk31gH8Hkl3upEK1jd0yfAmB20puITl52Mlzm3Xe6p8JDDZdKBppIaZI+ E0sin1ff/OBn/BPsPQjvCa92kOvm8Km118Wph3mUtQM4kZlMvxWSLWye1l9SUO0b Ou8zbuL0+5YaUABz57BGfJZDMHLORaeJMu3OVty4cpVBZl1Cn6aHNBbTlXK+w6vu 8N0VoDMtoXje2XIuBYhT9e5A104Sl40IU0Uh9K1dwRyyQhqKyUIoBzNMweyWCtN/ qFn2Idp3h48Wca4g2rAnY5jNaihDEL2T/a/aL5jLoLXrzz6bO6Jm1i/NqY932juJ KupcBN6mpMGLStG6/aTw9JC2oggszg2CT8Qu7HdAPweSbbBpH16VCsDgPqfEqOht +ByUUBGjvnVA95MhifUH+ZRaz2LVxgw5s9Hnh3pKVUmx7634MY0YlnS1FTc7ys9E aQWMwwTWGd09qNPMtawwKyMRzkf/m84N8+mnOEGmzMliEPGX+1t5vZ7TBQ+DCvBL HZ4XIKXtQG2of6nbyDbwu9RNWDRyvDCzXqyeoUdqnmTqWyaDyT942uaw59DPx9A7 1+lrAT3iNdBzFnSvYoNwHJ0yCOEbSRYXZKZ9fJClZ9m6ruKEKpC9mmNjFQARAQAB tCNUb2ZpZyBTdWxleW1hbm92IDx0b2ZpZ0BmcmVlYnNkLmF6PokCPwQTAQIAKQUC T3O5cwIbLwUJB4YfgAcLCQgHAwIBBhUIAgkKCwQWAgMBAh4BAheAAAoJEOMBSob+ wYEq8v8P+gJbkfc1sO/AdmWwZDqpyfG+NWQE9nj8gAlngblexX6qowzdBdupZ/1/ BW4qJhZxhY18cVwvF8NdBCTqWDb1iZ1j3zhw69zJdbd3tfqBIRFEeZY48limbIxH 9AcknRKdD+xomjDMbhzIunkYE79Sx3yUAsbB6wqaOvspa1bz9pBEwvGonKJzMBWc 28ieHTNZGaPJurBLlX/ED7N7ZtSflDZhH8wXIWgzdiQrkBRkKrW0J74xkh2EFm9p 82KCnadnukoNGplzdQsj2bODJkpJclz3QvjsyQhwfJdGCJMjgJPoPCT4kwQkABPy IP8tpVHEsZ1UxJG/PsUIV4D1Q8Nypsecx9OrxSdjeBI6JsuuKR2/At9dB2obVNO/ fnhS5WSsNmezrFyKgi3OWcLPwFyItVcVzhzydTvgLIzwH5uJ/w3hFVv45e2ZVReH M05OZCtj4AeTo/WIf8uJCOx1eGMoyy3X7rbzzyt9gSMeuHaKrLIGPZVddI24GZND ZXnlfhZPQvbmj4fjvaQd3Ayy1F8LcziWOT5OmSm12R/GdDKqo0dy646dTKVgka8+ nRGwyMeFGDeMFvZfIURP+efos/KDU8q1KufIHWGt06sGQa8nqakJ555kdlleS9Ol fX08WdgTm4wVEXIc5uXOUm6YVZ/05TZDnHBIIsUug3M0OeGviSSvuQINBE9zuXMB EAC5GRrh23AXpIG6OE8geROcscUhnU7mORAxxfjV6OLN30DLkkmMpGEUz3yRR2ma qRMKyJYtPTpoHqA4j8l7yh+G/ModSg7LToEilGQ8EoKKbSays9s6AtmgSgcLDp8c k8AvlByrMT1lBWIK5sOGlPDbEhYiAkYQbJutQyNRuSSMACFaqKeCqSbp+mJEx2co ODQXD8c8+RA5utL2r5T6NrmTv00s62TEEkTcbHDrXE5PuyxEsa1isnDWK58RjQZE EKjwO0WOlres8PA/EagVNQ6N65auF5jV7eOpgzUu8HCtLLv+OzhVpnTxWAaqlHnd A/B/FS+EtykPq+khE8rFGAWRvYCdmTZhWJg/ku8WAfBCdi9euhNn0t+TjuCam2Rt FquuF0O5+W+8RZLGBdL/NSS66RhyKOW+iO4kVmiUvmTLCFhk0ycsCh8+VgVDtL2y aq6USlJJkZ5F40hkv2JcoNltE5A1B+jbmqT2nLm1qrEEpEhkRJUrINn3GxwrcN3V WzP2tn6fE/0IMTBOAUg8oQ3Okc6IeURINSGkuY2b3SCUSWOmiAB0zL0NaK3vfM92 mKKrceqtRX1AvmOXVVv1ipvP9E5/cwwyTfds4QUaasX3eocF5ZNrAbEX0cCKj3NV C3hGwE/nRjUIsEIp3mJfPcEvppGr1JV3rmgv1UbdF6JvSwARAQABiQREBBgBAgAP BQJPc7lzAhsuBQkHhh+AAikJEOMBSob+wYEqwV0gBBkBAgAGBQJPc7lzAAoJEOf4 16JuYcl6If0P/1ihpZipRHxionQh7454OszRNXtaUa4Wxa8HCnYu6fA5VywerkBP Xnr1Tn70AMNaKiUCc9iLC2L9LdYjXn+lgE2R69AGJGfJnFl3rcvCvCUE0AXV0wBR bdLaWOQ3U++H2U1W9l7E0GcZie8+31IYaQCRjsJsE5w9cWDIaSqEcydkfb/bvp4o Mi8Qj/yQzwOYariilKoIpsdxpwVf1fL9OAk7J/fuyzz+dCypC0kCMjS0ooATkU0s Q7soNh+ZsHNg/Nw91eXsRotwgyErSNK8seoaZzzTLKhdO7SVFta8Ko+ad+KbwiKe n4RHaCssHmf1tgKh2Do+xc9FZz9nXMgf7dCLjD6cExyvVNGjy4ybgRp+C3i3R8e4 CVrRfc6XfjljfQ/2GDH4SVBZ3RGSxW7J4iKhnO7tg4pC1fDdUqriGG6OUrDDuiix xllCdGXlABJQV015wii0LrXtWZPU+oUE/6YmBYCDCHjNRkLW5WfxNzpcSNj8Be5U mGARwmdliXILUsGYglKVSkrk6Pxr6U0WDVSViC28iIo1wvDobPlyZmH0Hw9JXMr6 Rg39j8lrKtxF6x6ElBjrJLmn2XKOXOW6gsjlxlpD/eRxi3YYBDApnTKTd+86GX7X VCLDyEc4ZXYnxEiO5GvSPeiV8LtFVaaefGi3uViMUHMy3FCWRd5Dw/rRagIP/iW9 vRPTdAKY/foYpht/VJGp4N20u25W7oohJbfm/gbPLuKPB/wvQWpt2tf1ImJB2ufG 7GEJ9uPDnEuLxV2cdp0bQOQR4DiqJxXlBeSEeQlxJXVElUztGB/z3Z/CLeCGdVf/ XoCHVdSek+VkZrOY+1VJYo1Q89Z9eAMZAcRyfUUVpcTygy3ay/7Ja+tVvmc3/kHj HUp+OgKKOv170r85MKeeJtm9gakwcuTQTrJ5fTzIUr+DRJyQuxJw5VnF0Gb0VmKq ymjV92Th6sFzoKEGWfBcqbUatlkVR7bGc8tXuk5beIZwVzUwaJiBMLSWXHWO/tFW hKCilLFJ4lewC39ykH0iRDB6dzBSHbcmThpZwG1ZTXHA+AYCd79PmeQimGwKkMGs sgeLEE09ULKdhCML2DhScW6mdM9QqvWfQyA6YMgdI9LsG9aktPOA08lb8SN30P8C 8k2/4DSlIZdwSiKPDlBtfBpA43I4c76DtRZc0knlo+drwZvCZMbu6CCs3gW0ZkZs v5qmkDopG83bysq5ll+pytRRDx2Uzmd6jOXnTE3EFKUjuOUalHIF7NJJIv9lTcon Nb4rYu3KdOkilj06jAQWbHPgk4yI+ZOr95Vu5vbLd2Ft+06Af1UhbIJ5wY2Mkh/q CDhxA3vBoOulljNRUn6iwNSSxAeWekYw+ULUapL0 =wxyH -----END PGP PUBLIC KEY BLOCK-----

2 1

[Dovecot] fts-lucene vs SEARCH HEADER
by Matthew Powell 28 Jul '12

28 Jul '12

Hello. This is in Dovecot 2.1.7 on Linux x86, but it looks as though 2.1.8 has the same issue. I'm using fts-lucene. SEARCH HEADER TO returns no results, but SEARCH TO works: B SEARCH TO TEST * SEARCH 1 2 3 4 5 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 24 26 27 28 29 31 B OK Search completed (0.000 secs). C SEARCH HEADER TO TEST * SEARCH C OK Search completed (0.000 secs). Without FTS enabled, both forms of the query work. It looks as though the header names are indexed in lower case. But Dovecot is passing the header name to the backend in upper case when SEARCH HEADER is used. I'm not sure whether this is the *right* fix, but lower-casing the header name in lucene-wrapper.cc seems to fix the problem: --- dovecot-2.1.7-clean/src/plugins/fts-lucene/lucene-wrapper.cc 2011-12-13 06:35:29.000000000 -0500 +++ dovecot-2.1.7/src/plugins/fts-lucene/lucene-wrapper.cc 2012-07-24 14:03:14.000000000 -0400 @@ -1094,7 +1094,7 @@ return false; q = lucene_get_query(index, - t_lucene_utf8_to_tchar(index, arg->hdr_field_name, FALSE), + t_lucene_utf8_to_tchar(index, t_str_lcase(arg->hdr_field_name), FALSE), arg); break; default: Now: B SEARCH TO TEST * SEARCH 1 2 3 4 5 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 24 26 27 28 29 31 B OK Search completed (0.000 secs). C SEARCH HEADER TO TEST * SEARCH 1 2 3 4 5 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 24 26 27 28 29 31 C OK Search completed (0.000 secs). -- Matthew Powell matthew(a)atom.net

2 1