October 2013 - dovecot

[Dovecot] re-sorting emails in chrono order
by voytek＠sbt.net.au 12 Oct '13

12 Oct '13

dovecot 2.1.1 I access my IMAP folders through SquirrelMail as a result of 'part copying' (probably through web client ? not sure), that is copying/moving some emails, later, the rest, I have a sub folder where I have maybe 30 emails from 2012 appear at the very end like so 2013/2012/2010/.../2006/2005/2012 how can I re-sort ? thanks Voytek

1 0

[Dovecot] Doveadm with a 2nd Instance
by Chris Lasater 11 Oct '13

11 Oct '13

Hi Guys, I am trying to use 2 instances of Dovecot on the same server so I can have a Director managing my connections, everything appears to be working, but I can not use doveadm to control my 2nd instance, but doveconf seems to work fine. [user@server logs]$ doveconf -i Director | grep logs debug_log_path = /home/user/apps/logs/director/director_debug.log info_log_path = /home/user/apps/logs/director/director_info.log log_path = /home/user/apps/logs/director/director.log info_log_path = /home/user/apps/logs/director/director_lmtp.log [user@server logs]$ doveadm -i Director log find Debug: /home/user/apps/logs/dovecot_debug.log Info: /home/user/apps/logs/dovecot_info.log Warning: /home/user/apps/logs/dovecot.log Error: /home/user/apps/logs/dovecot.log Fatal: /home/user/apps/logs/dovecot.log I have stopped and started both my instances so the config running is what is in the config file, but when I use -i Director with doveadm it uses the other instances config. [user@server logs]$ doveadm instance list path name last used running /home/user/apps/dovecot/var/run/dovecot Dovecot 2013-09-26 00:39:14 yes /home/user/apps/dovecot/var/run/director Director 2013-09-26 00:32:31 yes Any ideas, or does anyone have any experience with the Dovecot instance feature? Chris

4 6

[Dovecot] Solr issue with Zlib
by Ricardo Branco 11 Oct '13

11 Oct '13

I have found that if you the fts_solr plugin is listed after Zlib it will cause an indexing problem where any message that was compressed with Zlib is submitted to solr without being uncompressed. The order of the plugins is important. Not sure if this can be fixed but im trying to add a wiki note. -- The information contained in or attached to this email is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorised to and must not disclose, copy, distribute, or retain any message or any part of it. If you have received an email in error, please contact the sender and delete the material from any computer. The contents of this email are not for publication unless specifically stated. Furthermore, the information contained in this message, and any attachment(s) thereto, is for information purposes only and may contain the personal views and opinions of the author, which are not necessarily the views and opinions of WENN or its subsidiaries and associated companies. We make every effort to keep our network free from viruses. However, you do need to check this e-mail and any attachments to it for viruses as we can take no responsibility for any computer virus which may be transferred by way! of this e-mail. WENN Ltd: Registered Office: 35 Tileyard Studios, Tileyard Road, London, N7 9AH, England. Registered No: 4375163. Place of Registration: United Kingdom. USA Entertainment News Inc (d/b/a WENN): Registered Office: 352 7th Avenue, Suite 1105, New York, NY 10001-5657, USA The WENN name, design and related marks are trademarks of the WENN group of companies. (c) 2013 All Rights Reserved.

1 0

[Dovecot] ./configure not checking solr deps
by Ricardo Branco 11 Oct '13

11 Oct '13

It seems that when you run ./configure with --with-solr it does not check for deps libcurl and libexpat. When we compiled it silently went though and still compiled without a problem then missed installing solr. Maybe im missing something? -- The information contained in or attached to this email is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorised to and must not disclose, copy, distribute, or retain any message or any part of it. If you have received an email in error, please contact the sender and delete the material from any computer. The contents of this email are not for publication unless specifically stated. Furthermore, the information contained in this message, and any attachment(s) thereto, is for information purposes only and may contain the personal views and opinions of the author, which are not necessarily the views and opinions of WENN or its subsidiaries and associated companies. We make every effort to keep our network free from viruses. However, you do need to check this e-mail and any attachments to it for viruses as we can take no responsibility for any computer virus which may be transferred by way! of this e-mail. WENN Ltd: Registered Office: 35 Tileyard Studios, Tileyard Road, London, N7 9AH, England. Registered No: 4375163. Place of Registration: United Kingdom. USA Entertainment News Inc (d/b/a WENN): Registered Office: 352 7th Avenue, Suite 1105, New York, NY 10001-5657, USA The WENN name, design and related marks are trademarks of the WENN group of companies. (c) 2013 All Rights Reserved.

1 0

[Dovecot] Sieve and Namespace in dovecot 2.0.X
by Mauricio Tavares 11 Oct '13

11 Oct '13

Based on what I read in http://wiki2.dovecot.org/Pigeonhole/Sieve/Usage, if I have a namespace defined as tail conf.d/10-mail.conf namespace inbox { inbox = yes location = prefix = INBOX. separator =. type = private } A global script like cat /etc/dovecot/sieve/global-spam.sieve require ["fileinto", "regex"]; # Must use regex here as 'contains' may not be valid, it erroneously # moved: # X-Spam-Status: No, score=-4.0 required=8.0 tests=ALL_TRUSTED,BAYES_00, # DCC_CHECK_NEGATIVE,HTML_MESSAGE,T_TM2_M_HEADER_IN_MSG,UNTRUSTED_Relay, # XM_SPF_Neutral autolearn=disabled version=3.2.5, No # # Due to the 'YES' in BAYES, let's just make sure YES is at the # _beginning_ of X-Spam-Status, while ignoring anything past it. #if header :regex "X-Spam-Status" "^[Yy][Ee][Ss].*" { if header :matches "X-Spam-Status" "Yes*" { fileinto "INBOX.Spam"; stop; } should put spam in bob/.Spam. But, I am getting an error message stating that INBOX.Spam does not exist: Oct 11 09:57:33 mail dovecot: lda(bob(a)domain.com): Error: sieve: msgid=<0.0.0.71C.1CEC689A21CFF08.7068B7(a)ip.aidolip.us>: failed to store into mailbox 'INBOX.Spam': Mailbox doesn't exist: INBOX.Spam How come?

2 5

[Dovecot] Optimisation opportunity for IMAP searches
by Rick van Rein (OpenFortress) 11 Oct '13

11 Oct '13

Hello, I love Dovecot, but when developing a small IMAP tool, I ran into searching behaviour can easily be optimised. Please forgive a rather detailed suggestion. This was on Dovecot 1.2.15 on Debian Squeeze. My tool? It's called "midget" and retrieves documents from an IMAP box based on their mid: or cid: identifier, as per RFC 2392. I thought this would be useful to retrieve email attachments into a remote shell environment. When using Kerberos, the credentials and a strong hint of one's mail address are already present anyway, DNS SRV does the rest. If you want to see the early code, I'll be happy to share it here. The three formats of the URIs in RFC 2392 are: * mid:messageid * cid:contentid * mid:messageid/contentid Searching for a mid: with only a Message-ID part is quick and probably indexed, (HEADER Message-ID <messageid>) Searching for a cid: meant ploughing through body text to get to the attachments, so it was a slow and costly and needed checking of the outcome: (TEXT <contentid>) The quicker URI form to get to a Content-Id should be to mention it at the end of a mid: URI, and search for (HEADER Message-ID <messageid>)(TEXT <conentid>) Much to my surprise, this took about as long as the cid: query -- ploughing through lots of body text while it only needed to look through a one message bodies! As a human, I can see a straightforward improvement. However, it is more difficult to find a general solution -- but that is what I'm describing below, in the hope that it will help to improve Dovecot's search performance. 0. In general, searches are Boolean expressions composed with implied AND and explicit (NOT …) and (OR ….) constructs. 1. Using the theorems of The Morgan, push all (NOT …) constructs inside as far as possible, until they surround elementary statements. Some may have a trivial solution, such as (NOT (ALL)). Others will retain the NOT but that might be handled with a different use of the indexes. In general, this phase turns all AND and OR constructs into positive logic, without duplicating search terms. 2. Estimate the effort involved in every part of the calculation. (HEADER) is lighter than (TEXT) and headers may be further split into with / without index. Larger sets could be more work to search than smaller ones. In general, an estimate of the number of comparisons could be a good metric for work to be done. 3. For AND compositions, start with the simplest one in the AND compostition, and continue with increasingly heavier ones. Apply later conditions only on the output of the previous conditions, either after collecting all of it or everytime something is found. Whatever is optimal -- bulk operations are more efficient, but may collect large sets to handle. 4. For OR compositions, either start a thread for each part, or switch focus and collect sorted streams into one collectively sorted stream. I've been doing this stuff all through my PhD thesis work, so it come easy to me :-) But this optimisation does not appear to be implemented yet, so I thought I'd suggest it. I hope this is useful to Dovecot! Best wishes, Rick van Rein

2 2

[Dovecot] Questio about replication
by Patrick Westenberg 11 Oct '13

11 Oct '13

Hi everyone, this article (http://blog.dovecot.org/2012/02/dovecot-clustering-with-dsync-based.html) describes the situation for director/NFS-based clusters and SSH-based clusters. Right now I'm running the first setup with one proxy/director and two backends (all 2.2.x) mounting an NFS-share but I'd like to get rid of NFS (it's slow) and, if one backend is down, the proxy still sends the user to that backend. The article says: "High-availability non-NFS setup One possibility is to use Dovecot proxies, which know which servers are down. Instead of directing users to those servers, it would direct them to replica servers." My question is, if I can still use my proxy/direcot setup but every backend having it's local (or iSCSI-backed) ext4 storage and not using the SSH-based replication method? Regards Patrick

2 1

[Dovecot] SSL with startssl.com certificates
by Dan Langille 11 Oct '13

11 Oct '13

I'm using Dovecot 2.2.5. I'm setting up and new IMAPS server for personal use (i.e. only me). I have success with self-signed certificates but not with others (e.g. StartSSL.com) With StartSSL certs: I've been able to connect and test commands via: openssl s_client -connect imaps.unixathome.org:993 Can you configure your iPhone or Macbook to access the above? Authentication isn't the issue. Connection is the issue. I've been able to get Thunderbird to connect and access my mail. However, I've been unable to get my iPhone or my Mac configured to use the same IMAP server. On the iPhone, adding the new Mail account causes the Settings app to crash on a persistently consistent basis when adding the new account. The crash occurs when connecting to the IMAPS server. Configuration never completes. I suspect the problem is SSL because in both cases (iPhone and Mac), I see these messages I see in the logs: *** /var/log/debug.log *** Sep 13 11:50:32 imaps dovecot: imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read client certificate A [166.137.84.11] Sep 13 11:50:45 imaps dovecot: auth: Debug: auth client connected (pid=31647) Sep 13 11:50:45 imaps dovecot: imap-login: Debug: SSL: where=0x10, ret=1: before/accept initialization [166.137.84.11] Sep 13 11:50:45 imaps dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: before/accept initialization [166.137.84.11] Sep 13 11:50:45 imaps dovecot: imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv2/v3 read client hello A [166.137.84.11] Sep 13 11:50:45 imaps dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read client hello A [166.137.84.11] Sep 13 11:50:45 imaps dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write server hello A [166.137.84.11] Sep 13 11:50:45 imaps dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write certificate A [166.137.84.11] Sep 13 11:50:45 imaps dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write server done A [166.137.84.11] Sep 13 11:50:45 imaps dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 flush data [166.137.84.11] Sep 13 11:50:45 imaps dovecot: imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read client certificate A [166.137.84.11] *** /var/log/maillog *** Sep 13 11:50:46 imaps dovecot: imap-login: Warning: SSL failed: where=0x2002: SSLv3 read client certificate A [166.137.84.11] Sep 13 11:50:46 imaps dovecot: imap-login: Disconnected (no auth attempts in 1 secs): user=<>, rip=166.137.84.11, lip=199.233.228.197, TLS handshaking: Disconnected, session=<a7AJd0LmWwCmiVQL> /usr/local/etc/ssl/imaps.unixathome.org.crt contains only the cert issued by StartSSL /usr/local/etc/ssl/imaps.unixathome.org.nopassword.key contains a no-password key generated by myself. Output of doveconf -n: # 2.2.5: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 9.1-RELEASE-p6 amd64 auth_debug = yes auth_verbose = yes first_valid_gid = 1001 first_valid_uid = 1001 mail_debug = yes mail_location = maildir:~/Maildir mail_privileged_group = mail passdb { args = scheme=BLF-CRYPT /var/db/dovecot.users driver = passwd-file } protocols = imap service imap-login { inet_listener imap { port = 0 } inet_listener imaps { address = 199.233.228.197 } } ssl = required ssl_cert = </usr/local/etc/ssl/imaps.unixathome.org.crt ssl_key = </usr/local/etc/ssl/imaps.unixathome.org.nopassword.key userdb { args = /var/db/dovecot.users driver = passwd-file } verbose_proctitle = yes verbose_ssl = yes protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep } -- Dan Langille - http://langille.org/

10 53

[Dovecot] cached message size errors
by Charles Sprickman 10 Oct '13

10 Oct '13

Hello, We recently moved from courier to dovecot, and I'm seeing a handful of errors that prevent people from retrieving email. Below is a snippet of the log sequence I see when this happens: Oct 10 13:44:24 mbox dovecot: imap(xxx(a)bway.net): Error: Cached message size smaller than expected (1759 < 1830) Oct 10 13:44:24 mbox dovecot: imap(xxx(a)bway.net): Error: Maildir filename has wrong S value, renamed the file from /home/vpopmail/domains/bway.net/1/xxx/Maildir/cur/1381381552.91972.xena.bway.net,S=1759:2,b to /home/vpopmail/domains/bway.net/1/xxx/Maildir/cur/1381381552.91972.xena.bway.net,S=1830:2,b Oct 10 13:44:24 mbox dovecot: imap(xxx(a)bway.net): Error: Corrupted index cache file /home/vpopmail/domains/bway.net/1/xxx/Maildir/dovecot.index.cache: Broken physical size for mail UID 447401 Oct 10 13:44:24 mbox dovecot: imap(xxx(a)bway.net): Error: read(/home/vpopmail/domains/bway.net/1/xxx/Maildir/cur/1381381552.91972.xen… failed: Invalid argument Oct 10 13:44:24 mbox dovecot: imap(xxx(a)bway.net): Disconnected: Internal error occurred. Refer to server log for more information. [2013-10-10 13:44:24] in=784 out=3017 Basically it looks like dovecot detects a mismatch in the file size vs. the file size embedded in the maildir filename, tries to fix it and then has issues reading the fixed file. Removing the file in question allows the user to retrieve email. The "invalid argument" error seems especially strange, as I'm able to view the file by hand with no problems. Blowing away the indexes and letting them get recreated seems to not help at all. This was an old vpopmail setup with both maildrop and vdelivermail doing the final delivery both now and prior to the migration to dovecot. For a quick fix, is there a way to have dovecot ignore the error and present the rest of the mailbox? Longer term fix, I'm not even sure where to start I've used dovecot elsewhere for quite some time (with maildirs) and I've never had an issue like this. 'doveconf -n' below... Thanks, Charles -- Charles Sprickman NetEng/SysAdmin Bway.net - New York's Best Internet www.bway.net spork(a)bway.net - 212.655.9344 # 2.2.5: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 8.4-RELEASE-p1 amd64 auth_default_realm = bway.net auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes base_dir = /var/run/dovecot/ default_process_limit = 300 disable_plaintext_auth = no first_valid_uid = 89 instance_name = dovecot1 last_valid_uid = 90 listen = 127.0.0.1,216.220.96.26,216.220.96.25 login_greeting = Dovecot ready - bway.net. mail_plugins = " quota" mailbox_list_index = yes namespace inbox { hidden = no inbox = yes list = yes location = mailbox Drafts { special_use = \Drafts } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { special_use = \Junk } mailbox Trash { special_use = \Trash } prefix = INBOX. separator = . subscriptions = yes type = private } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = maildir:User quota quota_rule = Inbox.Trash:storage=+100M quota_rule2 = Inbox.Spam:storage=+100M } pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %v-%u protocols = imap pop3 ssl_cert = </usr/local/etc/ssl/bwaynet.pem ssl_key = </usr/local/etc/ssl/bwaynet.key userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol imap { mail_max_userip_connections = 40 mail_plugins = " quota imap_quota" } protocol pop3 { mail_max_userip_connections = 20 }

1 1

[Dovecot] Maildir and home location
by Florian Lindner 10 Oct '13

10 Oct '13

Hello! My current mail setup looks like that: MTA is postfix, MDA is maildrop. Each system user configures mail delivery using the .mailfilter of maildrop to arbitrary maildirs in his home. For each maildir a POP3/IMAP account can be configured, currently I use courier-imap/-pop and think about replacing it by dovecot. AFAIK with dovecot each account has a home and maildir assigned which should not be equal neither should a home dir be used for mulitple accounts. My problem is where to put these homedirs? System users have full SSH access on the machine. I would really prefer to put them somewhere where the users could delete them (if they need to reset or delete their account). Optimal would be in the maildir itself, but... What are your suggestions? Regards, Florian

1 0