May 2013 - dovecot - dovecot.org

[Dovecot] feature request: pipe for custom quota dict queries
by Jeff Gustafson 03 May '13

03 May '13

Just a followup to my previous post. I appears that a pipe for quota queries via dict is not supported right now. Because of the way we group things we need more flexibility. I'd like to propose that dovecot should support quota queries just like userdb dict queries. My expertise is not C, but I might try to poke around and see if I can make something work. ...Jeff

1 0

[Dovecot] sieve: deleteheader sometimes fails [bug]
by Daniel Piddock 03 May '13

03 May '13

Hi, We're using Dovecot 2.1.16 with Pigeonhole 0.3.4. Mail is delivered by exim4 to the dovecot LDA. Sometimes users are unable to delete headers. I've narrowed the problem down to preceding the deleteheader command with a query against the body. A simple example: """ require ["body", "editheader", "fileinto"]; if body :contains "!TEST!" { fileinto "INBOX.Trash"; stop; } deleteheader "Subject"; addheader "Subject" "Testing"; """ Putting the "deleteheader" above the "if body" causes it to be deleted correctly. Although I've used Subject in this example it happens for other headers. Cheers, Dan

2 5

[Dovecot] dovecot 1.2.5 Fatal: setgid(5000(vmail)) Operation not permitted
by Mosh Mage 03 May '13

03 May '13

This is the error I'm getting (whenever i recieve and email): *deliver() : Fatal: setgid(5000(vmail)) failed with euid=5000(vmail), gid=8(mail), egid=8(mail): Operation not permitted (This binary should probably be called with process group set to 5000(vmail) instead of 8(mail)) * I am at loss. I have tried "everything" i could think of (besides purging the machine and do it all over again, which is not an option -- unless it is). Can anyone help me fix this? # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 3.2.13-grsec-xxxx-grs-ipv6-64-vps x86_64 Debian 6.0.6 ext3 log_timestamp: %Y-%m-%d %H:%M:%S disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login verbose_proctitle: yes mail_uid: vmail mail_gid: vmail mail_location: maildir:/var/zpanel/vmail/%d/%n mail_debug: yes mbox_write_locks: fcntl dotlock auth default: mechanisms: plain login verbose: yes debug: yes debug_passwords: yes passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 432 user: vmail group: vmail plugin: trash: /etc/dovecot/dovecot-trash.conf # /etc/dovecot/dovecot-sql.conf driver = mysql connect = host=localhost dbname=zpanel_postfix user=USERNAME password=AWESOMEPASSWORD default_pass_scheme = PLAIN password_query = \ SELECT name, domain, password \ FROM mailbox WHERE name = '%n' AND domain = '%d' user_query = \ SELECT maildir \ FROM mailbox WHERE name = '%n' AND active = '1' AND domain = '%d' #This is my ls -la drwxrwsrwx 5 vmail vmail 4096 May 2 18:39 vmail #and this is inside vmail/ drwx--S--- 3 vmail vmail 4096 May 2 18:39 asfaltotejo.com drwx--S--- 3 vmail vmail 4096 May 2 12:35 gestaoconteudos.pt drwx--S--- 17 vmail vmail 4096 May 2 15:34 tribanet.com #/var/run/dovecot srw-rw---- 1 vmail vmail 0 May 2 19:46 auth-master srw------- 1 root root 0 May 2 19:46 auth-worker.6564 srwxrwxrwx 1 root root 0 May 2 19:46 dict-server lrwxrwxrwx 1 root root 25 May 2 19:46 dovecot.conf -> /etc/dovecot/dovecot.conf drwxr-x--- 2 root dovecot 4096 May 2 19:46 login

1 0

[Dovecot] CATENATE allows zero parts
by Mike Abbott 03 May '13

03 May '13

I'm pleased to see that dovecot-2.2 includes support for RFCs 4467 and 4469 (URLAUTH and CATENATE). I have begun testing these features (in dovecot-2.2.1) and comparing their functionality against Apple's implementation. So far I have discovered a few inconsistencies. I will report each of these, and any more that I may find, in separate threads. The first issue is that using CATENATE with no message parts works but, IMO, shouldn't: a2 append inbox catenate () a2 OK [APPENDUID 1366726248 9] Append completed. For comparison this is how Apple's enhanced dovecot in OS X Server handles this request: a2 append inbox catenate () a2 BAD Invalid arguments. According to RFC 4469 section 3 CATENATE requires "one or more message parts." This is formalized in section 5: append-data =/ "CATENATE" SP "(" cat-part *(SP cat-part) ")" and in RFC 3501 section 9 which states that "[ABNF] rules MUST be followed strictly." I believe that permitting zero cat-parts is contrary to the RFC, although I can understand permitting it under the general doctrine of "servers should be liberal in what they accept, and clients strict in what they send." Is accepting zero cat-parts an intentional exception to the RFC or an oversight?

2 2

[Dovecot] CATENATE doesn't support literal+ url
by Mike Abbott 03 May '13

03 May '13

Dovecot-2.2.1 does not appear to support URLs specified via non-synchronizing literals (RFC 2088 LITERAL+), and also does not read and discard the literal+ input after reporting the error. This results in the literal+ input being interpreted as IMAP commands, which could alter the user's mail store in unexpected ways and/or lead to the abrupt disconnection of the client for exceeding the maximum number of bad IMAP commands. Example using dovecot-2.2.1 (note that the text nonsync literal includes the invisible CRLFs: "foobarCRLFbarbazCRLF"): a3 append inbox catenate (url {9+} /some-url text {16+} foobar barbaz ) a4 noop a3 BAD Error in IMAP command APPEND: Invalid arguments. foobar BAD Error in IMAP command : Unknown command. barbaz BAD Error in IMAP command : Unknown command. ) BAD Error in IMAP command : Unknown command. a4 OK NOOP completed. Example using Apple's enhanced dovecot in OS X Server: a3 append inbox catenate (url {9+} /some-url text {16+} foobar barbaz ) a4 noop a3 NO [BADURL /some-url] missing or invalid uid a4 OK NOOP completed. I believe the RFCs indicate that nonsync literal URLs should be permitted. RFC 4469 section 5: append-data =/ "CATENATE" SP "(" cat-part *(SP cat-part) ")" cat-part = text-literal / url url = "URL" SP astring RFC 3501 section 9: astring = 1*ASTRING-CHAR / string string = quoted / literal RFC 2088 section 4: literal ::= "{" number ["+"] "}" CRLF *CHAR8

2 4

[Dovecot] Tuning!
by Rafael VOlpe TI 02 May '13

02 May '13

Hi Buddies, I have 2 servers running dovecot + postfix for pop/imap users. The users mailbox are in a Nfs storage. The load of server is ranging at 3.0 to 15.0. I really dont know what happened.. I read a lot about tuning of dovecot, and the changes are applied, how you can see. The Hardware of server is really good! The host have 8GB of mem and 2 sockets with 2 cores anyone. The server receives 900-1000 user connections in pop and imap. How i can aprimmorate this processes? Some errors are displayed on the console randomly when the load rises. Example: May 02 13:57:42 pop3(user(a)domain.com): Error: Timeout (180s) while waiting for lock for transaction log file //var/vmail/ domain.com/user//dovecot.index.log May 02 13:57:42 pop3(user(a)domain.com): Error: Couldn't init INBOX: Internal error occurred. Refer to server log for more information. [2013-05-02 13:54:40] May 02 13:57:42 pop3(user(a)domain.com): Info: Mailbox init failed top=0/0, retr=0/0, del=0/0, size=0 I really appreciate any suggestion! Thanks and Regards, Rafael Volpe Dovecot conf: # 2.0.19: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-40-generic x86_64 Ubuntu 12.04.2 LTS auth_mechanisms = plain login auth_verbose = yes debug_log_path = /var/log/dovecot-debug.log disable_plaintext_auth = no dotlock_use_excl = no first_valid_gid = 125 first_valid_uid = 125 last_valid_gid = 125 last_valid_uid = 125 log_path = /var/log/dovecot.log mail_debug = yes mail_fsync = always mail_location = maildir:/%Lh/:INDEX=/%Lh/ mail_nfs_index = yes mail_nfs_storage = yes mmap_disable = yes passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocols = " imap pop3" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } service_count = 0 } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } ssl_cert = </etc/postfix/ssl/wildcard.domain.com.crt ssl_key = </etc/postfix/ssl/wildcard.domain.com.key userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } verbose_proctitle = yes protocol imap { imap_idle_notify_interval = 2 mins mail_max_userip_connections = 150 } protocol pop3 { pop3_lock_session = no pop3_uidl_format = %08Xu%08Xv } Postfix conf: alias_database = hash:/etc/aliases alias_maps = hash:/etc/aliases append_dot_mydomain = no biff = no bounce_queue_lifetime = 1d broken_sasl_auth_clients = yes config_directory = /etc/postfix content_filter = smtp-amavis:[127.0.0.1]:10024 inet_interfaces = all mailbox_size_limit = 0 maximal_queue_lifetime = 1d message_size_limit = 20240000 myhostname = myname.mydomain.com mynetworks = 127.0.0.0/8 myorigin = /etc/mailname policy-spf_time_limit = 3600s readme_directory = no recipient_bcc_maps = mysql:/etc/postfix/mysql_bcc.cf recipient_delimiter = + relay_domains = proxy:mysql:/etc/postfix/mysql_relay_domains_maps.cf relayhost = sender_bcc_maps = mysql:/etc/postfix/mysql_bcc.cf smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_banner = Welcome to $myhostname smtpd_client_restrictions = permit_mynetworks, permit_sasl_authenticated, check_client_access mysql:/etc/postfix/mysql_access.cf, reject_unknown_client, reject_unknown_client_hostname, reject_unauth_pipelining, reject_rbl_client bl.spamcop.net, reject_rbl_client zen.spamhaus.org, reject_rbl_client b.barracudacentral.org smtpd_end_of_data_restrictions = check_policy_service inet:127.0.0.1:10031 smtpd_helo_required = yes smtpd_helo_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_non_fqdn_helo_hostname, reject_invalid_helo_hostname, reject_invalid_hostname smtpd_recipient_restrictions = reject_unknown_sender_domain, reject_unknown_recipient_domain, reject_non_fqdn_sender, reject_non_fqdn_recipient, reject_unlisted_recipient, check_policy_service inet:127.0.0.1:10031, permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, check_policy_service unix:private/policy-spf smtpd_reject_unlisted_sender = yes smtpd_sasl_auth_enable = yes smtpd_sasl_local_domain = $mydomain smtpd_sasl_path = private/auth smtpd_sasl_security_options = noanonymous smtpd_sasl_type = dovecot smtpd_sender_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_pipelining, reject_unauth_destination smtpd_timeout = 30 smtpd_tls_cert_file = /etc/postfix/ssl/wildcard.domain.com.crt smtpd_tls_key_file = /etc/postfix/ssl/wildcard.domain.com.key smtpd_tls_security_level = may smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache transport_maps = mysql:/etc/postfix/mysql_transport.cf vacation_destination_recipient_limit = 1 virtual_alias_maps = mysql:/etc/postfix/mysql_virtual_alias_maps.cf virtual_gid_maps = static:125 virtual_mailbox_base = /var/vmail virtual_mailbox_domains = mysql:/etc/postfix/mysql_virtual_domains_maps.cf virtual_mailbox_limit = 51200000 virtual_mailbox_maps = mysql:/etc/postfix/mysql_virtual_mailbox_maps.cf virtual_minimum_uid = 125 virtual_transport = virtual virtual_uid_maps = static:125

3 2

[Dovecot] Dovecot vs MBox
by Gregory Sloop 02 May '13

02 May '13

I'm still in the "what's wrong" stage of figuring out what's going on. But I've got a mail user who isn't getting new messages. Postfix accepts it and drops it in the users mbox. [This is verified. If I tail the Mbox, I can see the new messages.] "Mail" will see the messages too. But dovecot doesn't seem to know they exist. I don't think the MBox is corrupt, as I've tossed the first few messages in hopes that it would then read the rest, but no luck. Any ideas where to look next, what I might do to force dovecot to forget message ID's etc - that might force it to read the whole mailbox file again? [Or a pointer as to where it might be most productive to poke next?] TIA -Greg -- Gregory Sloop, Principal: Sloop Network & Computer Consulting 503.251.0452 x121 Voice | 503.251.0452 Fax www.sloop.net mailto:gregs@sloop.net

5 7

[Dovecot] CATENATE allows empty messages
by Mike Abbott 02 May '13

02 May '13

Dovecot-2.2.1 allows empty messages to be APPENDed when using CATENATE: b1 append inbox catenate (text {0+} ) b1 OK [APPENDUID 1366726248 12] Append completed. Contrast this with regular APPEND: b2 append inbox {0+} b2 NO Can't save a zero byte message. Note that zero-size literals are OK but zero-size messages are not. So while "b1" above should fail like "b2", this should continue to succeed: b3 append inbox catenate (text {0+} text {8+} foobar ) b3 OK [APPENDUID 1366726248 13] Append completed.

3 6

[Dovecot] Slow DNS warnings (proxy/auth)
by Christian Balzer 02 May '13

02 May '13

Hello, I've just finished transiting our proxies from perdition to dovecot (2.1.7-7 Debian). Yesterday 12 messages (all within the same second) like this caught my attention: --- Apr 25 17:19:09 pp11 dovecot: auth: Warning: proxy(redacted(a)gol.com,xx.xx.xx.xx,<26hUEivbfQBlMrMS>): DNS lookup for mb04.dentaku.gol.com took 5.002 s --- Now this machine at that time was handling a load of about 2 logins per second, about 20% of what it previously handled with perdition w/o a hiccup. It also runs a local caching nameserver and the A record for the mailbox server in question was most definitely cached at the time (verified via TTL). The machine in question was very bored and certainly capable of handling hundreds if not thousands of DNS queries per second at that moment. In short, I can't see any reason how the lookup could have taken so long, so my guess is there are some issues with the dns-helper (locking, stepping on each others feet, not being spawned fast enough) causing this. Some general remarks, dovecot as proxy feels "heavier" than perdition. In the CPU area that's probably a more subjective impression, because all the little helper processes make it clear what's going on where. Though the "config" process being rather active is something that perdition definitely doesn't do, it reads the config once at start time and that's it. All the IPC and central processes of course also make dovecot rather file handle hungry. Memory wise it's about 35% bigger than perdition and that's not subjective at all. ^o^ About one MB per proxy process/connection for dovecot in my case. Caveat emptor. ^o^ Regards, Christian -- Christian Balzer Network/Systems Engineer chibi(a)gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/

2 1

Re: [Dovecot] XFS vs EXT4 for mail storage
by Reindl Harald 02 May '13

02 May '13

Am 02.05.2013 17:31, schrieb Charles Marcus: > On 2013-05-02 10:45 AM, Reindl Harald <h.reindl(a)thelounge.net> wrote: >> "man mount" is very generic and doe snot cover any option >> there is also no "man mount.ntfs" while "mount.ntfs" command exists > > ? > > man mount, at least mine, shows first the FILESYSTEM INDEPENDENT MOUNT OPTIONS, then as you scroll down, you'll get > to all of the additional filesystem specific options, ie, for xfs... > > So, not sure what you mean by 'does not cover any option'... typo - meant "does not cover ALL options" someone has to write the manpages, there are millions of options all over the software-world which did it not make into manpages google for "xfs delaylog" leads to http://xfs.org/index.php/XFS_FAQ http://xfs.org/index.php/XFS_FAQ#Q:_I_want_to_tune_my_XFS_filesystems_for_.… For mount options, the only thing that will change metadata performance considerably are the logbsize and delaylog mount options so how can it be redhat-specific if it is mentioned upstream?

1 0