January 2012 - dovecot

[Dovecot] A namespace error on 2.1rc5
by Bill Ryder 30 Jan '12

30 Jan '12

Hello all, I'm not sure if this is a bug. It's probably just an upgrade note. In summary I had no namespace section in my 2.0.17 config. When trying out 2.1rc5 no user could login because of a namespace error. 2.1rc5 adds a default namespace clause which broke my logins (It was noted in the changelog) I seemed to fix it by just putting this in the config file: namespace inbox { inbox = yes } Long story: I've been recently testing dovecot against cyrus to decide where we should go for our next mail server(s) I loaded up the mail server with mail delivered via postfix all on dovecot 2.0.15 (I've since moved to 2.0.17) I have three dovecot directors, two backends on the same NFS mail store. With dovecot 2.0.xx the tester works fine (it't just a script which logins in and emulates thunderbird when a user is idle - without using IDLE so the client asks for mail every few minutes). When I moved to 2.1rc5 I got namespace errors and the user can not login. The server said: dovecot-error.log-20120128:Jan 27 13:37:59 imap(ethab01): Error: user ethab01: Initialization failed: namespace configuration error: inbox=yes namespace missing The client says: * BYE Internal error occurred. Refer to server log for more information. The session looks like 0.000000 192.168.121.37 -> 192.168.121.2 TCP 33213 > imap [SYN] Seq=0 Win=5840 Len=0 MSS=1460 TSV=1056649457 TSER=0 WS=5 0.000036 192.168.121.2 -> 192.168.121.37 TCP imap > 33213 [SYN, ACK] Seq=0 Ack=1 Win=5792 Len=0 MSS=1460 TSV=3264407631 TSER=1056649457 WS=7 0.000187 192.168.121.37 -> 192.168.121.2 TCP 33213 > imap [ACK] Seq=1 Ack=1 Win=5856 Len=0 TSV=1056649458 TSER=3264407631 0.006338 192.168.121.2 -> 192.168.121.37 IMAP Response: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Dovecot ready. 0.006889 192.168.121.37 -> 192.168.121.2 TCP 33213 > imap [ACK] Seq=1 Ack=124 Win=5856 Len=0 TSV=1056649465 TSER=3264407637 0.006973 192.168.121.37 -> 192.168.121.2 IMAP Request: I ID ("x-originating-ip" "192.168.114.249" "x-originating-port" "49403" "x-connected-ip" "192.168.121.37" "x-connected-port" "143") 0.006980 192.168.121.2 -> 192.168.121.37 TCP imap > 33213 [ACK] Seq=124 Ack=178 Win=6912 Len=0 TSV=3264407638 TSER=1056649465 0.007086 192.168.121.2 -> 192.168.121.37 IMAP Response: * ID NIL 0.018471 192.168.121.2 -> 192.168.121.37 IMAP Response: * BYE Internal error occurred. Refer to server log for more information. (interestingly the tshark output strips out the user name and password which is convenient but which may mean there's not enough information?) I rolled back to 2.0.17 and it was fine again. It's the same config files for both, same maildirs etc etc. All I did was change the dovecot version from 2.0.17 to 2.1rc5 However I see from the changelog that 2.1rc5 added a default namespace inbox: diff doveconf-n.2.0.17 doveconf-n.2.1-rc5 1c1 < # 2.0.17 (684381041dc4+): /etc/dovecot/dovecot.conf --- > # 2.1.rc5: /etc/dovecot/dovecot.conf 20a21,39 > namespace inbox { > location = > mailbox Drafts { > special_use = \Drafts > } > mailbox Junk { > special_use = \Junk > } > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > special_use = \Trash > } > prefix = > } We had this section commented out in 2.0.x so there was no namespace inbox anywhere. ============== doveconf -n for 2.0.17 (for the backends) # 2.0.17 (684381041dc4+): /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-131.6.1.el6.x86_64 x86_64 Scientific Linux release 6.1 (Carbon) nfs auth_mechanisms = plain login auth_username_format = %n auth_verbose = yes debug_log_path = /var/log/dovecot/dovecot-debug.log disable_plaintext_auth = no first_valid_uid = 200 info_log_path = /var/log/dovecot/dovecot-info.log log_path = /var/log/dovecot/dovecot-error.log mail_debug = yes mail_fsync = always mail_gid = vmail mail_location = maildir:/vol/dt_mailstore1/spool/%n:INDEX=/var/indexes/%n mail_nfs_storage = yes mail_plugins = " fts fts_solr mail_log notify quota" mail_uid = vmail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { driver = pam } plugin { autocreate = Trash autocreate2 = Drafts autocreate3 = Sent autocreate4 = Templates autosubscribe = Trash autosubscribe2 = Drafts autosubscribe3 = Sent autosubscribe4 = Templates fts = solr fts_solr = break-imap-search debug url=http://dovecot-solr1.wetafx.co.nz:8080/solr/ mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { group = vmail user = vmail } } service lmtp { inet_listener lmtp { address = 192.168.121.2 127.0.0.1 port = 24 } process_min_avail = 20 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { args = uid=vmail gid=vmail home=/vol/dt_mailstore1/%n/%u driver = passwd } protocol lmtp { mail_plugins = sieve } protocol lda { mail_plugins = " fts fts_solr mail_log notify quota sieve" } protocol imap { mail_plugins = " fts fts_solr mail_log notify quota imap_quota" } protocol sieve { managesieve_notify_capability = managesieve_sieve_capability = } ===== doveconf for 2.1rc5 # 2.1.rc5: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-131.6.1.el6.x86_64 x86_64 Scientific Linux release 6.1 (Carbon) nfs auth_mechanisms = plain login auth_username_format = %n auth_verbose = yes debug_log_path = /var/log/dovecot/dovecot-debug.log disable_plaintext_auth = no first_valid_uid = 200 info_log_path = /var/log/dovecot/dovecot-info.log log_path = /var/log/dovecot/dovecot-error.log mail_debug = yes mail_fsync = always mail_gid = vmail mail_location = maildir:/vol/dt_mailstore1/spool/%n:INDEX=/var/indexes/%n mail_nfs_storage = yes mail_plugins = " fts fts_solr mail_log notify quota" mail_uid = vmail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { autocreate = Trash autocreate2 = Drafts autocreate3 = Sent autocreate4 = Templates autosubscribe = Trash autosubscribe2 = Drafts autosubscribe3 = Sent autosubscribe4 = Templates fts = solr fts_solr = break-imap-search debug url=http://dovecot-solr1.wetafx.co.nz:8080/solr/ mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { group = vmail user = vmail } } service lmtp { inet_listener lmtp { address = 192.168.121.2 127.0.0.1 port = 24 } process_min_avail = 20 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { args = uid=vmail gid=vmail home=/vol/dt_mailstore1/%n/%u driver = passwd } protocol lmtp { mail_plugins = sieve } protocol lda { mail_plugins = " fts fts_solr mail_log notify quota sieve" } protocol imap { mail_plugins = " fts fts_solr mail_log notify quota imap_quota" } protocol sieve { managesieve_notify_capability = managesieve_sieve_capability = }

1 0

[Dovecot] 2.1.rc3 (1a722c7676bb): Panic: file ostream.c: line 173 (o_stream_sendv): assertion failed: (stream->stream_errno != 0)
by Pascal Volk 30 Jan '12

30 Jan '12

Hi Timo, today some imap processes are crashed. Regards, Pascal -- The trapper recommends today: f007ba11.1201322(a)localdomain.org

1 1

[Dovecot] imap-login process_limit reached
by Don Buchholz 29 Jan '12

29 Jan '12

I've been having some problems with IMAP user connections to the Dovecot (v2.0.8) server. The following message is being logged. Jan 16 10:51:36 postal dovecot: master: Warning: service(imap-login): process_limit reached, client connections are being dropped The server is running Red Hat Enterprise Linux release 4 (update 6). Dovecot is v2.0.8. We have only 29 user accounts in /etc/dovecot/users. There were 196 "dovecot/imap" processes and 6 other dovecot processes, for a total of 202 "dovecot" processes, listed in the 'ps aux' output when problems were being experienced. Stopping and restarting the Dovecot system fixes the problem -- for a while. The 'doveconf -n' output is attached. I have not set any "process_limit" values, and I don't think I'm getting anywhere close to the 1024 default, so I'm pretty confused as to what might be wrong. Any suggestions on what to do next are appreciated. Thanks, - Don ------------------------------------------------------------------------ # 2.0.8: /etc/dovecot/dovecot.conf # OS: Linux 2.6.9-67.0.1.ELsmp i686 Red Hat Enterprise Linux WS release 4 (Nahant Update 6) ext3 auth_username_format = %Lu auth_verbose = yes auth_verbose_passwords = sha1 login_log_format_elements = user=<%u> service=%Us method=%m rip=%r lip=%l mpid=%e %c login_trusted_networks = xx.xx.xx.0/24 mail_location = maildir:/Virtual_Users/%u/Mail passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } passdb { driver = pam } ssl_ca = </etc/dovecot/ssl/iMove_2011_CAcert.pem ssl_cert = </etc/dovecot/ssl/postal_cert.pem ssl_key = </etc/dovecot/ssl/postal_key+req.pem ssl_key_password = ********** ssl_parameters_regenerate = 12 userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } userdb { driver = passwd } verbose_proctitle = yes protocol imap { ssl_cert = </etc/dovecot/ssl/imap_cert.pem ssl_key = </etc/dovecot/ssl/imap_key+req.pem ssl_key_password = </etc/dovecot/ssl/imap.pw } protocol pop3 { ssl_cert = </etc/dovecot/ssl/pop3_cert.pem ssl_key = </etc/dovecot/ssl/pop3_key+req.pem ssl_key_password = </etc/dovecot/ssl/pop3.pw }

5 16

[Dovecot] XFS Developer Takes Shots At Btrfs, EXT4
by Adrian Minta 29 Jan '12

29 Jan '12

Nice article about XFS improvements: http://tinyurl.com/7pvr9ju

2 2

[Dovecot] 2.1.rc1 (056934abd2ef): virtual plugin mailbox search pattern
by e-frog 29 Jan '12

29 Jan '12

Hello Timo, With dovecot 2.1.rc1 (056934abd2ef) there seems to be something wrong with virtual plugin mailbox search patterns. I'm using a virtual mailbox 'unread' with the following dovecot-virtual file $ cat dovecot-virtual * unseen For testing propose I created the following folders with each containing one unread message INBOX, INBOX/level1 and INBOX/level1/level2 2.1.rc1 (056934abd2ef) 1 LIST "" "*" * LIST (\HasChildren) "/" "INBOX" * LIST (\HasChildren) "/" "INBOX/level1" * LIST (\HasNoChildren) "/" "INBOX/level1/level2" * LIST (\HasChildren) "/" "virtual" * LIST (\HasNoChildren) "/" "virtual/unread" 1 OK List completed. 2 STATUS "INBOX" (UNSEEN) * STATUS "INBOX" (UNSEEN 1) 2 OK Status completed. 3 STATUS "INBOX/level1" (UNSEEN) * STATUS "INBOX/level1" (UNSEEN 1) 3 OK Status completed. 4 STATUS "INBOX/level1/level2" (UNSEEN) * STATUS "INBOX/level1/level2" (UNSEEN 1) 4 OK Status completed. 5 STATUS "virtual/unread" (UNSEEN) * STATUS "virtual/unread" (UNSEEN 1) 5 OK Status completed. Result: virtual/unread shows only 1 unseen message. Further tests showed it's the one from INBOX. The mails from the deeper levels are not found. Downgrading to 2.0.16 restores the correct behavior: 1 LIST "" "*" * LIST (\HasChildren) "/" "INBOX" * LIST (\HasChildren) "/" "INBOX/level1" * LIST (\HasNoChildren) "/" "INBOX/level1/level2" * LIST (\HasChildren) "/" "virtual" * LIST (\HasNoChildren) "/" "virtual/unread" 1 OK List completed. 2 STATUS "INBOX" (UNSEEN) * STATUS "INBOX" (UNSEEN 1) 2 OK Status completed. 3 STATUS "INBOX/level1" (UNSEEN) * STATUS "INBOX/level1" (UNSEEN 1) 3 OK Status completed. 4 STATUS "INBOX/level1/level2" (UNSEEN) * STATUS "INBOX/level1/level2" (UNSEEN 1) 4 OK Status completed. 5 STATUS "virtual/unread" (UNSEEN) * STATUS "virtual/unread" (UNSEEN 3) 5 OK Status completed. Result: virtual/unread shows 3 unseen messages as it should The namespace configuration is as following namespace { hidden = no inbox = yes list = yes location = prefix = separator = / subscriptions = yes type = private } namespace { location = virtual:~/virtual prefix = virtual/ separator = / subscriptions = no type = private } I've also tried this with location = virtual:~/virtual:LAYOUT=maildir++ leading to the same result. Thanks, e-frog

2 3

[Dovecot] Migration to multi-dbox and SiS
by Ronald MacDonald 29 Jan '12

29 Jan '12

Dear list, A huge thank-you first of all for all the work that's gone into Dovecot itself. I'm rebuilding a mail server next week and so, taking the rare opportunity to re-consider all the options I've had running over the past couple of years. Around the time of the last re-build (2010), there had been some discussion on single instance storage, which was quite new on Dovecot around then. I chickened out of setting it up though. Now with it having been in the wild for a couple of years, I wonder, how have people found SiS to behave? Additionally, though there was talk of the prospect of it being merged with 2.x am I right in thinking it's not yet in the main project? Couldn't find any 2.x changelogs that could confirm this. With best wishes, Ronald.

2 1

[Dovecot] Attribute Cache flush errors on FreeBSD 8.2
by Phil Kernick 29 Jan '12

29 Jan '12

I'm running dovecot 2.0.16 on FreeBSD 8.2 with the mail spool and indexes on an NFS server. Lines like the following keep appearing in syslog for access to each mailbox: Error: nfs_flush_attr_cache_fd_locked: fchown(/home/philk/Mail/Deleted) failed: Bad file descriptor This is coming from nfs-workarounds.c line 210, which tracing back seems to be coming from the call to mbox_lock on lib-storage/index/mbox/mbox-lock.c line 774. I have /home mounted with options acregmin=0,acregmax=0,acdirmin=0,acdirmax=0 (as FreeBSD doesn't have a noac option), but it throws the same error either way. The output of dovecot -n is below. Phil. # 2.0.16: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 8.2-RELEASE-p3 i386 auth_mechanisms = plain login auth_username_format = %Lu disable_plaintext_auth = no first_valid_gid = 1000 first_valid_uid = 1000 listen = *, [::] mail_fsync = always mail_location = mbox:~/Mail/:INBOX=/var/mail/%u mail_nfs_index = yes mail_nfs_storage = yes mail_privileged_group = mail mmap_disable = yes passdb { args = session=yes dovecot driver = pam } protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = root } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { args = blocking=yes driver = passwd } verbose_proctitle = yes protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep } protocol pop3 { pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv } protocol lda { postmaster_address = postmaster(a)example.com sendmail_path = /usr/sbin/sendmail }

2 1

[Dovecot] 2.0.17: Index lost -> SAVEDON lost as well?
by Ralf Hildebrandt 29 Jan '12

29 Jan '12

Today I encoundered these errors: Jan 9 08:30:06 mail dovecot: lmtp(31174, backup(a)backup.invalid) Error: Log synchronization error at seq=858,offset=44672 for /home/mailboxname/mdbox/storage/dovecot.map.index: Append with UID 282388, but next_uid = 282389 Jan 9 08:30:06 mail dovecot: lmtp(31819, backup(a)backup.invalid) Error: Log synchronization error at seq=858,offset=44672 for /home/mailboxname/mdbox/storage/dovecot.map.index: Append with UID 282388, but next_uid = 282389 Jan 9 08:30:06 mail dovecot: lmtp(32148, backup(a)backup.invalid) Error: Log synchronization error at seq=858,offset=44672 for /home/mailboxname/mdbox/storage/dovecot.map.index: Append with UID 282388, but next_uid = 282389 After that, the SAVEDON date for all mails was reset to today: mail:~# doveadm search -u backup(a)backup.invalid mailbox INBOX SAVEDON 2012-01-09 | wc -l 75650 mail:~# doveadm search -u backup(a)backup.invalid mailbox INBOX SAVEDON 2012-01-08 | wc -l 0 mail:~# doveadm search -u backup(a)backup.invalid mailbox INBOX SAVEDON 2012-01-07 | wc -l 0 Before, I was running this: vorgestern=`date -d "-2 day" +"%Y-%m-%d"` doveadm expunge -u backup(a)backup.invalid mailbox INBOX SAVEDBEFORE $vorgestern doveadm purge -u backup(a)backup.invalid Is there a way of restoring the SAVEDON info? -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | http://www.charite.de

2 4

[Dovecot] compressed mboxes very slow
by kjonca＠o2.pl 29 Jan '12

29 Jan '12

I have some archive mails in gzipped mboxes. I could use them with dovecot 1.x without problems. But recently I have installed dovecot 2.0.12, and they are slow. very slow. Creating index files takes about 10 minutes for ~20M file with 560 messages for bzipped mbox, for gzipped is little better but still unusable :( Stracing dovecot process shows that every ~ 20 messages it rereads complete mbox file. Am I doing something wrong? KJ -- http://modnebzdury.wordpress.com/2009/10/01/niewiarygodny-list-prof-majewsk…

4 13

[Dovecot] maildir vs mdbox
by Jean-Daniel Beaubien 29 Jan '12

29 Jan '12

Hi, I am planning on running on test between maildir and mdbox to see which is a better fit for my use case. And I'm just looking for general advice/recommendation. I will post any results I obtain here. Important question: I have multiple users hitting the same email account at the same time. Can be a problem with mdbox? (either via thunderbird or with custom webmail apps). I remember having huge issues with mbox a decade ago because of this. Maildir fixed this... will mdbox reintroduce this problem? This is a very important point for me. Here is my use case: - Ubuntu server (any specific recommandations on FS to use?) - Standard PC hardware (core i5 or i7, few gigs of ram, hdds at first, probably ssd afterwards, nothing very fancy) - Serving only a hand full of email accounts but some of the accounst have over 3 millions emails in them (with individual mail folders having 100k+ emails) - Will use latest dovecot (2.1 when it comes out) - fts-lucene or fts-solr? -jd

2 5