December 2014 - dovecot

LMTP BUG
by Peter Hodur 12 Dec '14

12 Dec '14

Hello, just setting up mail hub with Postfix & Dovecot I have found BUG in LMTP implementation. * assumptions - latest stable Postix & Dovecot, - Postfix delivers via Dovecot LMTP (virtual_transport = lmtp:127.0.0.1:24), - Dovecot userdb & passwd lookups are made via custom checkpassword interface, - each user (email account) has different UID/GID * description LMTP process must run as root to be able to deliver msgs to each user. It drops privileges temporarily when delivering and then restoring effective UID/GID to root (saved one). The problem is, when an attempt with more than ONE recipients is made. More than one recipient within single session. It seems, that Dovecot LMTP restores root privileges not between each recipient delivery attempt but after whole transaction. The only solution is throttle Postfix to send single message with multiple recipients as many small transactions - all with only ONE rcpt. lmtp_destination_recipient_limit=1 makes this magic. Otherwise you can find: Dec 12 03:30:36 vm dovecot: lmtp(3580, info(a)xxx.com) Fatal: setgid(48672 from userdb lookup) failed with euid=33001, gid=43570, egid=43570: Operation not permitted (This binary should probably be called with process group set to 4867 2 instead of 43570) Can someone confirm that this is an error/bug? Thanks Pete

1 0

[Corrected] Can't get shared public folders working
by Randall Gellens 12 Dec '14

12 Dec '14

I'm trying to use a shared public namespace. My mail client says it exists, and I can telnet in and SELECT it, but sieve scripts can't write to it. The configuration in /etc/dovecot/conf.d/10-mail.conf: # Shared namespace for Foo stuff namespace { type = public separator = '/' prefix = "#Foo/" location = maildir:/local/mnt/mail/shared:INDEXPVT=/local/mnt/mail/%n/shared:LAYOUT=fs # Use the default namespace for saving subscriptions. #subscriptions = no # List the shared/ namespace only if there are visible shared mailboxes. #list = children } I created the location, owned by the vmail user: $ ls -ld /local/mnt/mail/shared drwxr-xr-x 4 vmail vmail 4096 Dec 11 12:38 /local/mnt/mail/shared I created the subfolders, verified that they are there. Also, I see that Dovecot created a dovecot.mailbox.log file: $ ls -l /local/mnt/mail/shared total 12 -rw-r--r-- 1 vmail vmail 96 Dec 11 12:38 dovecot.mailbox.log drwxr-xr-x 3 vmail vmail 4096 Dec 11 12:37 Foo-Bugs drwxr-xr-x 3 vmail vmail 4096 Dec 11 12:38 Foo-Patches The dovecot.mailbox.log file is empty: $ more /local/mnt/mail/shared/dovecot.mailbox.log The intermediate folder is empty: $ ls -l /local/mnt/mail/shared/Foo-Bugs/ total 4 drwxr-xr-x 5 vmail vmail 4096 Dec 11 12:36 Foo-Bugs-New The mailbox has some folders in it: $ ls -l /local/mnt/mail/shared/Foo-Bugs/Foo-Bugs-New/ total 12 drwxr-xr-x 2 vmail vmail 4096 Dec 11 12:36 cur drwxr-xr-x 2 vmail vmail 4096 Dec 11 12:36 new drwxr-xr-x 2 vmail vmail 4096 Dec 11 12:36 tmp They seem to be empty: $ ls -l /local/mnt/mail/shared/Foo-Bugs/Foo-Bugs-New/* /local/mnt/mail/shared/Foo-Bugs/Foo-Bugs-New/cur: The Sieve log insists the mailbox doesn't exist: $ more /local/mnt/home/Foo-bugs/.dovecot.sieve.log sieve: info: started log at Dec 11 15:35:20. error: msgid=<p06240614d0afdab4a086(a)[99.111.97.136]>: failed to store into mailbox '#Foo/Foo-Bugs/Foo-Bugs-New': Mailbox doesn't exist: #Foo/Foo-Bugs/Foo-Bugs-New. info: msgid=<p06240614d0afdab4a086(a)[99.111.97.136]>: stored mail into mailbox 'INBOX'. But if I telnet into the server, the mailbox shows up: a1 list "" * * LIST (\HasNoChildren \Junk) "/" Junk * LIST (\Noselect \HasChildren) "/" #Foo * LIST (\Noselect \HasChildren) "/" #Foo/Foo-Patches * LIST (\HasNoChildren) "/" #Foo/Foo-Patches/Foo-Patches-New * LIST (\Noselect \HasChildren) "/" #Foo/Foo-Bugs * LIST (\HasNoChildren) "/" #Foo/Foo-Bugs/Foo-Bugs-New * LIST (\HasNoChildren) "/" INBOX a1 OK List completed. And I can SELECT it: a2 select "#Foo/Foo-Bugs/Foo-Bugs-New" * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 0 EXISTS * 0 RECENT * OK [UIDVALIDITY 1418343553] UIDs valid * OK [UIDNEXT 1] Predicted next UID * OK [NOMODSEQ] No permanent modsequences a2 OK [READ-WRITE] Select completed (0.006 secs). So why can't the Sieve script store into it? -- Randall Gellens Opinions are personal; facts are suspect; I speak for myself only -------------- Randomly selected tag: --------------- I must have a prodigious quantity of mind; it takes me as much as a week sometimes to make it up. --Mark Twain, _The Innocents Abroad_

1 0

Can't get shared public folders working
by Randall Gellens 12 Dec '14

12 Dec '14

I'm trying to use a shared public namespace. Clients say it exists, but sieve scripts can't write to it. The configuration in /etc/dovecot/conf.d/10-mail.conf: # Shared namespace for Foo stuff namespace { type = public separator = '/' prefix = "#Foo/" location = maildir:/local/mnt/mail/shared:INDEXPVT=/local/mnt/mail/%n/shared:LAYOUT=fs # Use the default namespace for saving subscriptions. #subscriptions = no # List the shared/ namespace only if there are visible shared mailboxes. #list = children } I created the location, owned by the vmail user: $ ls -ld /local/mnt/mail/shared drwxr-xr-x 4 vmail vmail 4096 Dec 11 12:38 /local/mnt/mail/shared I created the subfolders, verified that they are there. Also, I see that Dovecot created a dovecot.mailbox.log file: $ ls -l /local/mnt/mail/shared total 12 -rw-r--r-- 1 vmail vmail 96 Dec 11 12:38 dovecot.mailbox.log drwxr-xr-x 3 vmail vmail 4096 Dec 11 12:37 Foo-Bugs drwxr-xr-x 3 vmail vmail 4096 Dec 11 12:38 Foo-Patches The dovecot.mailbox.log file is empty: $ more /local/mnt/mail/shared/dovecot.mailbox.log The intermediate folder is empty: $ ls -l /local/mnt/mail/shared/Foo-Bugs/ total 4 drwxr-xr-x 5 vmail vmail 4096 Dec 11 12:36 Foo-Bugs-New The mailbox has some folders in it: $ ls -l /local/mnt/mail/shared/Foo-Bugs/Foo-Bugs-New/ total 12 drwxr-xr-x 2 vmail vmail 4096 Dec 11 12:36 cur drwxr-xr-x 2 vmail vmail 4096 Dec 11 12:36 new drwxr-xr-x 2 vmail vmail 4096 Dec 11 12:36 tmp They seem to be empty: $ ls -l /local/mnt/mail/shared/Foo-Bugs/Foo-Bugs-New/* /local/mnt/mail/shared/Foo-Bugs/Foo-Bugs-New/cur: The Sieve log insists the mailbox doesn't exist: $ more /local/mnt/home/Foo-bugs/.dovecot.sieve.log sieve: info: started log at Dec 11 15:35:20. error: msgid=<p06240614d0afdab4a086(a)[99.111.97.136]>: failed to store into mailbox '#Foo/Q popper-Bugs/Foo-Bugs-New': Mailbox doesn't exist: #Foo/Foo-Bugs/Foo-Bugs-New. info: msgid=<p06240614d0afdab4a086(a)[99.111.97.136]>: stored mail into mailbox 'INBOX'. But if I telnet into the server, the mailbox shows up: a1 list "" * * LIST (\HasNoChildren \Junk) "/" Junk * LIST (\Noselect \HasChildren) "/" #Foo * LIST (\Noselect \HasChildren) "/" #Foo/Foo-Patches * LIST (\HasNoChildren) "/" #Foo/Foo-Patches/Foo-Patches-New * LIST (\Noselect \HasChildren) "/" #Foo/Foo-Bugs * LIST (\HasNoChildren) "/" #Foo/Foo-Bugs/Foo-Bugs-New * LIST (\HasNoChildren) "/" INBOX a1 OK List completed. And I can SELECT it: a2 select "#Foo/Foo-Bugs/Foo-Bugs-New" * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 0 EXISTS * 0 RECENT * OK [UIDVALIDITY 1418343553] UIDs valid * OK [UIDNEXT 1] Predicted next UID * OK [NOMODSEQ] No permanent modsequences a2 OK [READ-WRITE] Select completed (0.006 secs). So why can't the Sieve script store into it? -- Randall Gellens Opinions are personal; facts are suspect; I speak for myself only -------------- Randomly selected tag: --------------- Broad-mindedness: The result of flattening high-mindedness out.

1 0

mdbox backup strategy
by Jiří František 11 Dec '14

11 Dec '14

Hi, now we backup maildir with rdiff-backup every single day. Backup takes almost 20hours. I would like to switch to mdbox, but how to acomplish possibility of restore emails from any date what I want? Now if I need to restore mails from the day before yesterday I put the right date to parametrs of rdiff-backup restore command and I get what I want. But if I use doveadm backup I have backup only from last run. Because we have tens of TB email I couldn't do full backup every single day. Is it possible to restore state of mailbox from backup run before the last run if I use doveadm? I don't want do full backup every day, because backup storage is not unlimited. Thank you Jiri

1 0

dovecot.index.log files: what are they?
by Thomas Klausner 11 Dec '14

11 Dec '14

Hi! I have lots of these files: /home/wiz/Mail/my-folder-name/cur/.imap/1238738125.13533_23713.danbala:2,S/dovecot.index.log What are they for? Why are they here? Can I remove them? Thomas

2 4

dovecot & Apple Mail & maildir & lots of Mail
by Thomas Klausner 11 Dec '14

11 Dec '14

Hi! I have dovecot 2.1.12 running on a mail server, and recently configured Apple Mail to connect to it using secure IMAP, for the first time. At the beginning it just showed the inbox and everything was fine, but then I wanted to look at some of my folders and found the 'subscribe' menu. When I opened it, Apple Mail went to discover what mailboxes there are, and that's where my trouble began. It seemed to make good progress for some time (though slow -- over days), and now lists the mailboxes from starting with letters a to d on the left hand side (filling all the visible space, so there might be more). However, whenever Apple Mail gets focus, the cursor becomes a spinning rainbow circle and I can't interact with it. I tried deleting the mail account and setting it up again. The inbox was shown again immediately and Apple Mail was usable, but the first time the cursor moved over the mailboxes on the left hand side, it froze again. There were two imap processes on the server, one rather idle, the other eating CPU for about two minutes, then idling, and some time later it disappeared too. My Mail directory is 31G with about 180 directories, each containing mails in maildir format. Has anyone had similar issues? Is my maildir too big for dovecot too handle? How can I debug this? Thanks, Thomas

5 6

Quota issue with mailbox.
by Grant Pasley 11 Dec '14

11 Dec '14

good morning, i am experiencing an issue with a quot limits with a new install of dovecot with mysql integration. quotas are set to 0 (unlimited) and show as such in the mysql database as well as via the postfixadmin frontend. i use imapsync to transfer mail from another imap server to this one, the total user mailbox size is 18gb. dovecot lda tells me mailbox is full even though i have set mailbox to unlimited. *the the error is:* Dec 9 19:17:01 sentinel dovecot: lda(grant(a)xxxxxxx.com): msgid=<e8b2c897c11cafda2c2ab93a92beb681(a)localhost.localdomain>: rejected: Quota exceeded (mailbox for user is full) *[root@sentinel dovecot]# grep -v '^ *$#.*$\?$' dovecot-mysql.conf* driver = mysql connect = host=localhost dbname=postfix user=postfix password=xxxxxxxx default_pass_scheme = MD5-CRYPT password_query = SELECT username as user, password, concat('/home/vmail/', maildir) as userdb_home, concat('maildir:/home/vmail/', maildir) as userdb_mail, 101 as userdb_uid, 12 as userdb_gid FROM mailbox WHERE username = '%u' AND active = '1' user_query = SELECT concat('/home/vmail/', maildir) as home, concat('maildir:/home/vmail/', maildir) as mail, 101 AS uid, 12 AS gid, CONCAT('*:messages=10000:bytes=', quota) as quota_rule FROM mailbox WHERE username = '%u' AND active = '1' *[root@sentinel dovecot]# dovecot -n* *# 2.0.9: /etc/dovecot/dovecot.conf* *# OS: Linux 2.6.32-504.1.3.el6.x86_64 x86_64 CentOS release 6.6 (Final) ext4* auth_mechanisms = plain login dict { quotadict = mysql:/etc/dovecot/dovecot-dict-quota.conf } first_valid_gid = 12 first_valid_uid = 101 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lmtp_save_to_detail_mailbox = yes mail_location = maildir:/home/vmail/%d/%n managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date passdb { args = /etc/dovecot/dovecot-mysql.conf driver = sql } plugin { acl = vfile:/etc/dovecot/acls quota = dict:user::proxy::quotadict sieve = ~/dovecot.sieve sieve_before = /home/sieve/globalfilter.sieve sieve_dir = ~/sieve sieve_max_script_size = 1M trash = /etc/dovecot/trash.conf } protocols = imap pop3 lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = mail mode = 0666 user = vmail } } service dict { unix_listener dict { group = mail mode = 0666 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service imap { vsz_limit = 256 M } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } ssl_cert = </etc/pki/tls/certs/xxxxxxxxxxxxxxxx.crt ssl_key = </etc/pki/tls/private/xxxxxxxxxxxxxxxxx.key userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-mysql.conf driver = sql } protocol lda { mail_plugins = quota sieve postmaster_address = postmaster@xxxxxxxxxxxxxx } protocol imap { imap_client_workarounds = delay-newmail mail_plugins = quota imap_quota trash } protocol lmtp { mail_plugins = sieve } protocol pop3 { mail_plugins = quota pop3_client_workarounds = outlook-no-nuls oe-ns-eoh } protocol sieve { managesieve_implementation_string = Dovecot Pigeonhole managesieve_max_compile_errors = 5 managesieve_max_line_length = 65536 } --- This email has been checked for viruses by Avast antivirus software. http://www.avast.com

2 2

Re: dovecot.index.log files: what are they?
by Joseph Tam 10 Dec '14

10 Dec '14

Thomas Klausner writes: >> RTFM: http://wiki2.dovecot.org/IndexFiles > > Thanks, but I had read this. > > I still don't know what they are good for, why they stay there for > days and if I can remove them. After all, they are not the caches, but > some "transaction logs" (I don't know what this is). Re-read the page, all the way to the bottom. Interpreting the last section: The transaction log records recent changes to a mailbox, and together with the index, gives the current state of the mailbox. This is an I/O optimization so that mailbox states changes are quickly updated without having to compare an entire index file with the last (in-memory?) index. For large mailboxes, this can save a lot of I/O. It may persist because updates to the indices are not done immediately (again, possibly to reduce I/O), so the current mailbox state is reflected by the index file *and* deltas in the transaction log. I suspect that you could remove the transaction logs (if you do not use dbox) and dovecot should rebuild them (or more likely, rebuild the indices). Joseph Tam <jtam.home(a)gmail.com>

1 0

devoid mailbox status for mail reloaded from a tape backup
by Stephen Lidie 10 Dec '14

10 Dec '14

dovecot --version 2.2.10 I've only be able to status a mailbox for currently existing users, is it possible to status a mailbox reloaded from tape to an alternate location? The user may or may not exist at this time, but I still need to inspect the mailbox ... thank you, Steve

3 9

Renaming directories with subdirectories over IMAP
by Зинин Дмитрий Андреевич 10 Dec '14

10 Dec '14

Hi, I¹ve a problem with renaming directories with subdirectories. In root directory I create directory named ³lvl1² with subdirectory ³lvl2² When I rename directory lvl1 to lvl1-new I get: 1. /lvl1-new with subdir lvl2 but I can use only /lvl1-new and not /lvl1-new/lvl2 2. /lvl1 with subdir lvl2 but I can use only /lvl1/lvl2 and not /lvl1 Is it a feature or a bug? If bug is there a fix? PS: CentOS 6.6 and Dovecot 2.0.9-7 from official repository with all updates. Best regards, Dmitriy

2 1