April 2012 - dovecot - dovecot.org

[Dovecot] distributed mdbox
by James Devine 09 Apr '12

09 Apr '12

Anyone know how to setup dovecot with mdbox so that it can be used through shared storage from multiple hosts? I've setup a gluster volume and am sharing it between 2 test clients. I'm using postfix/dovecot LDA for delivery and I'm using postal to send mail between 40 users. In doing this, I'm seeing these errors in the logs Mar 21 09:36:29 test-gluster-client2 dovecot: lda(testuser34): Error: Fixed index file /mnt/testuser34/mdbox/storage/dovecot.map.index: messages_count 272 -> 271 Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: Log synchronization error at seq=4,offset=3768 for /mnt/testuser28/mdbox/storage/dovecot.map.index: Append with UID 516, but next_uid = 517 Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: Log synchronization error at seq=4,offset=4220 for /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update for invalid uid=517 Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: Log synchronization error at seq=4,offset=5088 for /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update for invalid uid=517 Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Warning: fscking index file /mnt/testuser28/mdbox/storage/dovecot.map.index Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser34): Warning: fscking index file /mnt/testuser34/mdbox/storage/dovecot.map.index This is my dovecot config currently: jdevine@test-gluster-client2:~> dovecot -n # 2.0.13: /etc/dovecot/dovecot.conf # OS: Linux 3.0.0-13-server x86_64 Ubuntu 11.10 lock_method = dotlock mail_fsync = always mail_location = mdbox:~/mdbox mail_nfs_index = yes mail_nfs_storage = yes mmap_disable = yes passdb { driver = pam } protocols = " imap" ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { driver = passwd }

5 19

[Dovecot] Using a namespace for providing access to mail snapshots for user based on-demand restoration of email backups
by Charles Marcus 09 Apr '12

09 Apr '12

Hi all, I'm planning on implementing this in my new upcoming dovecot instance, and would like to hear thoughts on how best to accomplish this. We will be paying Timo's support company to do the work, but obviously, the less work in the form of coding he has to do to get this working (I'm hoping it won't be a lot), the more money it will save us... ;) First - I currently use rsnapshot to backup emails, so that is the use-case I'm most interested in getting working. It is rsync based, and like other rsync based backup programs it uses hardlinks to save storage space - so you can have a *lot* of backups (going back months, or even years), where each snapshot only adds a little more to the total disk space being used. The snapshots are stored with the following filesystem layout: /path/to/snapshotsdir/hourly.0 ... /path/to/snapshotsdir/hourly.4 /path/to/snapshotsdir/daily.0 ... /path/to/snapshotsdir/daily.7 /path/to/snapshotsdir/weekly.0 ... /path/to/snapshotsdir/weekly.4 /path/to/snapshotsdir/monthly.0 ... /path/to/snapshotsdir/monthly.12 /path/to/snapshotsdir/yearly.0 ... /path/to/snapshotsdir/yearly.5 The 'names' (hourly, daily, weekly, monthly, yearly) are arbitrary (this is a bit confusing to people new to rsnapshot), and would *not* be used for displaying the mail folders to the users - it is the Date/Time stamps of each of the snapshot dirs above that would be used to display the folder names under the 'Time Machine' namespace. This is, I imagine, the part that will need some actual coding by Timo to get working - maybe just some new config variables added to the namespace code for mapping the date/time stamps of the directories to user friendly folder names in the namespace. That said, I'd like to design this and have it coded such that it will work with almost any type of backup storage that stores snapshots as date/time stamped directories like this (there must be others, right?). Also, it goes without saying that this code will be (if Timo is ok with it) part of the core dovecot code going forward, so anyone else will be able to benefit from it. What I'm envisioning is something like this... 1. Define a namespace - for this example we'll call it 'Time Machine' 2. Under this namespace, each user will see their, and *only* their snapshots So, each user would see something like this: My Mail Account Inbox Drafts Templates Sent Time Machine (sorted above user created folders if possible) -4/3/12, 8:00am (first subfolder) Inbox Drafts etc... (all other folders and sub-folders shown here) +4/3/12, 12:00pm (first subfolder) etc... Other User Folders ... Or even better, I'm thinking some magical code that can group them by Date, like: -4/3/12 (first subfolder) -8:00am (next sub-folder) Inbox Drafts Etc... (all folders and sub-folders shown here) +12:00pm +4:00pm +8:00pm +4/4/12 etc... Comments? Suggestions? Flames? -- Best regards, Charles

3 4

Re: [Dovecot] Using a namespace for providing access to mail snapshots for user based on-demand restoration of email backups
by Joseph Tam 09 Apr '12

09 Apr '12

A timely topic as I was just mulling over ways to provide this to my users. Charles Marcus wrote: > The snapshots are stored with the following filesystem layout: > > /path/to/snapshotsdir/hourly.0 > ... This is familiar to NetApp users. > The 'names' (hourly, daily, weekly, monthly, yearly) are arbitrary (this > is a bit confusing to people new to rsnapshot), and would *not* be used > for displaying the mail folders to the users - it is the Date/Time > stamps of each of the snapshot dirs above that would be used to display > the folder names under the 'Time Machine' namespace. This is, I imagine, > the part that will need some actual coding by Timo to get working - > maybe just some new config variables added to the namespace code for > mapping the date/time stamps of the directories to user friendly folder > names in the namespace. > > That said, I'd like to design this and have it coded such that it will > work with almost any type of backup storage that stores snapshots as > date/time stamped directories like this (there must be others, right?). One idea is to take this complexity entirely out of dovecot and create a synthetic filesystem using hard or soft links (as rsnapshot has done) and create your own, with whatever weird and wonderful naming scheme you want. /path/to/TimeMachine/<friendlylabel>/<user> -> /path/to/snapshotsdir/<snaplabel>/<user> namespace { prefix = TimeMachine location = maildir:/path/to/TimeMachine:INDEX=MEMORY ... } This might not be very scalable depending on how big your userbase is. I would probably define memory indices for this namespace, and take the performance hit on the assumption that access will be a once-in-a- while thing. On-disk indices will probably get out of date with each snapshot rollover, if if you have a lot of snapshots/mailboxes, it could consume a non-trivial amount of space without a lot of benefits. Or you could run a cron script to rename or remove old indices, but that seems more trouble than it's worth. One other consideration (at least for me) is if the INBOX and personal mail folders are stored in two separate FS's. It would be nice to fuse the two sets of backups under the same namespace, but I don't know how the namespace prefix matching works and whether you can define hierarchical namespaces like namespace { prefix = backup/inbox location = mbox:/path/to/inbox-snapdir/%u ... } namespace { prefix = backup/mail location = mbox:/path/to/mail-snapdir/%u ... } The above can also be accomplished with a synthetic filesystem. Joseph Tam <jtam.home(a)gmail.com>

2 1

[Dovecot] 2.1.3: Overly lax FETCH parsing
by Michael M Slusarz 09 Apr '12

09 Apr '12

These commands both work in Dovecot (as of 2.1.3): 1 UID FETCH 1:* () (CHANGEDSINCE 1) * 1 FETCH (UID 30097 MODSEQ (19554)) [...] 1 OK Fetch completed. 2 FETCH 1:* () (CHANGEDSINCE 1) * 2 FETCH (MODSEQ (19554)) [...] 2 OK Fetch completed. However, RFC 3501 indicates that an empty FETCH attribute list is incorrect: fetch = "FETCH" SP sequence-set SP ("ALL" / "FULL" / "FAST" / fetch-att / "(" fetch-att *(SP fetch-att) ")") fetch-att = "ENVELOPE" / "FLAGS" / "INTERNALDATE" / "RFC822" [".HEADER" / ".SIZE" / ".TEXT"] / "BODY" ["STRUCTURE"] / "UID" / "BODY" section ["<" number "." nz-number ">"] / "BODY.PEEK" section ["<" number "." nz-number ">"] In other words, fetch-att is not allowed to be empty. (RFC 4466 does not alter this requirement when using FETCH modifiers). While useful that Dovecot is more liberal about what it receives, 3501 seems pretty clear that incorrect FETCH parameters must return a BAD. I can verify that the above commands fail on Cyrus. I don't disagree that the way Dovecot handles this is correct - or at least is the way that I wish the RFCs were written. Since this seems a bit redundant, but is necessary under the ABNF: 1 UID FETCH 1:* UID (CHANGEDSINCE 1) -or- 1 UID FETCH 1:* (UID) (CHANGEDSINCE 1) michael

2 1

[Dovecot] lmtp sometimes fails to deliver a message to all recipients
by Artur Zaprzała 09 Apr '12

09 Apr '12

Dovecot version: 2.1.3 lmtp config: protocol lmtp { mail_plugins = zlib acl sieve } Running strace on lmtp reveals that when lmtp delivers a message to multiple recipients it saves the message to a file in the first mailbox and hard links that file to other mailboxes. But when in the meantime of delivery that file is deleted, lmtp fails to deliver the message to remaining mailboxes. For example a message for 46 recipients is delivered by lmtp to the first 12 only (fortunately postfix retries delivery for remaining recipients) and for the rest lmtp logs the following error messages: lmtp(3344, foo@domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: msgid=unspecified: failed to store into mailbox 'INBOX': Message was expunged (guid) lmtp(3344, foo@domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: script /vmail/domain/foo/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /vmail/domain/foo/.dovecot.sieve.log may reveal additional details) I'm seeing this error for about 10 messages per hour. Best regards, Artur Zaprzała

2 3

[Dovecot] Problem with polish characters and sieve fileinto
by l.chelchowski＠eurocar.pl 06 Apr '12

06 Apr '12

Hi, I have problems with sieve fileinto and polish characters in folder names: # rule:[ąćęłńóśźżĄĆĘŁŃÓŚŹŻ] elsif anyof (header :contains "Subject" "ąćęłńóśźżĄĆĘŁŃÓŚŹŻ") { fileinto "INBOX/&AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-"; } dovecot log: Apr 5 15:17:42 mailer dovecot: lmtp(54203, test(a)test1.com) rmHVJHabfU+70wAA9hAjDg: sieve: msgid=<4F7D9B68.4030206(a)test2.com>: stored mail into mailbox 'INBOX' Apr 5 15:17:42 mailer dovecot: lmtp(54203, test(a)test1.com) Error: rmHVJHabfU+70wAA9hAjDg: sieve: execution of script /home/vmail/test1.com/test//.dovecot.sieve failed, but implicit keep was successful (user logfile /home/vmail/test1.com/test//.dovecot.sieve.log may reveal additional details) Apr 5 15:17:42 mailer postfix/lmtp[54202]: 941B1E2681F: to=<test(a)test1.com>, relay=m.test1.com[private/dovecot-lmtp], delay=0.04, delays=0/0.01/0/0.03, dsn=2.0.0, status=sent (250 2.0.0 <test(a)test1.com> rmHVJHabfU+70wAA9hAjDg Saved) dovecot.sieve.log: sieve: info: started log at Apr 05 15:17:42. error: msgid=<4F7D9B68.4030206(a)test2.com>: failed to store into mailbox 'INBOX/&AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-' (INBOX/&-AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-): Mailbox doesn't exist: INBOX.&-AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-. Thanks for help. -- Pozdrawiam Łukasz

3 3

[Dovecot] Courier to Dovecot migration script issue
by xnasx 06 Apr '12

06 Apr '12

Hi! Forgive my bad English. I trying to migrate to Dovecot 2.0.18 from Courier 4.10. I using a script 'courier-dovecot-migrate.pl' from oficial wiki - it works without errors and creating dovecot-uidlist, etc. . I running Dovecot on a different port: 11149. When I connecting to port 11149 from Thunderbird, all my message headers are reloading. Please help me to solve this issue. Best regards, Victor

3 3

[Dovecot] Username bug in '%%' variables selects
by Victor 06 Apr '12

06 Apr '12

Hi! Forgive my bad English. I trying to set up shared folders, but I can't. I shared INBOX of 'victor(a)domain.com' for 'victor(a)testmail.domain.com' There is my debugging results: doveadm acl debug -u victor(a)testmail.domain.com INBOX.shared.victor(a)domain.com. doveadm(victor(a)testmail.domain.com): Error: Couldn't create namespace 'INBOX.shared.' for user victor@domain: userdb didn't return a home directory, but mail_location used it (%h): maildir:%h That's I seen in logs: auth-worker: Debug: sql(victor@domain): SELECT CONCAT('/var/spool/mail/', SUBSTRING_INDEX(email, '@', -1), '/', SUBSTRING_INDEX(email, '@', 1)) as home from `users` WHERE email='victor@domain' What happened with my username? 'victor@domain' is not correct value. My config: namespace { inbox = yes location = prefix = INBOX. separator = . type = private } namespace { hidden = no list = yes location = maildir:%%h prefix = INBOX.shared.%%u. separator = . subscriptions = no type = shared } password_query = \ SELECT email as username, password \ FROM users WHERE email = '%u' user_query = SELECT CONCAT('/var/spool/mail/', SUBSTRING_INDEX(email, '@', -1), '/', SUBSTRING_INDEX(email, '@', 1)) as home from `users` WHERE email='%u' Please, help me! -- Best regards, Victor

1 1

[Dovecot] dovecot: auth: Error: ldap(...) failed: No such object
by Bruce Hammond 06 Apr '12

06 Apr '12

I just installed the 2.1.3 version of dovecot. When I telnet to my dovecot linux server on the imap port (143) and try to login I get the error "No [UNAVAILABLE] Temporary authentication failure" on the telnet display and dovecot: auth: Error: ldap(bhammond_test(a)131.103.212.38,128.121.121.193): ldap_search(base=uid=bhammond_test(a)131.103.212.38,ou=users,dc=aspect filter=(&(objectClass=posixAccount)(productFeature=mail))) failed: No such object in the /var/log/maillog. I am not familiar with ldap and I haven't been able to find anything using google. Anyone have any ideas with the problem? Thanks. Here is my configuration: # 2.1.3: /usr/local/packages/dovecot2/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-274.3.1.el5.secure i686 Red Hat Enterprise Linux Server release 5.7 (Tikanga) auth_mechanisms = plain login auth_verbose = yes default_login_user = dovecot disable_plaintext_auth = no listen = * mail_location = maildir:~/mail passdb { args = /etc/devel/passdb.conf driver = ldap } pop3_lock_session = yes protocols = imap pop3 service auth { user = dovecot } service imap-login { process_limit = 128 process_min_avail = 30 service_count = 0 user = dovecot vsz_limit = 64 M } service pop3-login { process_limit = 128 process_min_avail = 30 service_count = 0 user = dovecot vsz_limit = 64 M } shutdown_clients = no ssl_cert = </etc/ssl.cert ssl_cipher_list = ALL:!LOW!SSLv2 ssl_key = </etc/ssl.key userdb { args = /usr/local/packages/dovecot2/etc/userdb.conf driver = ldap } protocol imap { imap_max_line_length = 64 k } protocol pop3 { pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s pop3_uidl_format = %08Xv%08Xu }

1 0

[Dovecot] create mailboxes in /var/mail via doveadm
by Radim Kolar 05 Apr '12

05 Apr '12

what usedid is used by doveadm for creating mailboxes? can it be setup to run as dovecot:mail? cant create mailbox from command line ponto# ls -ld /var/mail drwxrwxr-x 8 root mail 24 Apr 5 20:35 /var/mail ponto# doveadm mailbox create -u jboss INBOX doveadm(jboss): Error: Can't create mailbox INBOX: Permission denied mail_location = maildir:%h/Maildir:INBOX=/var/mail/%u

1 0