February 2011 - dovecot

[Dovecot] Problem with messages with long lines
by ZMA - Emiliano Mariño 17 Feb '11

17 Feb '11

Hello dear mailing list readers, Recently I am experimenting a problem trying to retrieve messages over a POP3 protocol. Everything goes right until the POP3 client tries to retrieve a message that contains one or more long lines lines as long as 3000 characters or even more. The client (Outlook 2003 and 2007) stops on that message until timeout. I have opened a connection manually to see what happens, and when I issue the RETR command on one of these messages, dovecot informs the message size and then nothing else. No message sending. Any other message works fine; I get the message size and next the message itself. I have upgraded to version 2.0.1 but I still get the same behavior. No error is reported on the server side. Logs indicate that everything is just fine. Any advice on how to solve this would be very appreciated. Here is the dovecot 2.0.1 configuration: ############################################################################ ####### # 2.0.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.23.1-42.fc8 i686 Fedora release 8 (Werewolf) auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@& auth_verbose = yes disable_plaintext_auth = no listen = * login_greeting = Dovecot DA ready. mail_access_groups = mail mail_location = maildir:~/Maildir passdb { args = username_format=%n /etc/virtual/%d/passwd driver = passwd-file } passdb { driver = shadow } service auth { user = root } ssl_cert = </etc/httpd/conf/ssl.crt/server.crt ssl_cipher_list = ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:-LOW:-SSLv2:-EXP ssl_key = </etc/httpd/conf/ssl.key/server.key userdb { args = username_format=%n /etc/virtual/%d/passwd driver = passwd-file } userdb { driver = passwd } verbose_proctitle = yes protocol pop3 { pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv } ############################################################################ ####### Thanks a lot for your time Kind regards, cid:image001.png@01CB7784.62522490 Emiliano Mariño Technical Analyst Senior <mailto:%20mkt4@zma.com.ar> tech7(a)zma.com.ar Larrea 1011 piso 8º | C1117ABE | Ciudad A. de Buenos Aires - ARGENTINA Tel: (5411) 4825-1602 | Fax: (5411) 4825-7692 <http://www.zma.com.ar/> www.zma.com.ar

4 3

[Dovecot] Logout fails after copy to non existing folder (mutt)
by Micha Krause 17 Feb '11

17 Feb '11

Hi, I have a Problem with mutt, and copying mails to a new folder. If I copy to a non existing folder, and try to logout, dovecot doesnt accept the logout. I was able to reproduce the problem with the following imap session: C: 1 login user pass S: 1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS ACL RIGHTS=texk] Logged in C: 2 select INBOX S: 2 OK [READ-WRITE] Select completed. C: 3 UID COPY 82806 "INBOX.test" S: 3 NO [TRYCREATE] Mailbox doesn't exist: INBOX.test C: 4 LOGOUT S: <nothing> Micha Krause

2 1

[Dovecot] Dovecot + Exim + Maildir quota problem
by Matteo Tinazzi 17 Feb '11

17 Feb '11

Hi all i'm a new user of this list, i got a server with DirectAdmin which has exim as MUA, dovecot as POP/IMAP server, recently i added the quota to exim and pointed the maildirsize file in the same location if the dovecot's one so both services can interact with it. This has been done to let exim update the quotas while doing localdelivery while dovecot can show quotas through roundcube and update them when users delete mails. The issue that i noticed is that some users get wrong quota informations... after a little investigation i think it could be a concurrent access to this file so dovecot tries to remove the deleted mails fomr maildirsize and exim tries to put new ones inside. In a specific case i got a user with a quota of 50M seeing that 75% of his quota filled while the real weight of the mail files is about 4% ... any help would be very appreciated

1 1

[Dovecot] Evolution IMAP with Dovecot as proxy
by Antonio Perez-Aranda 17 Feb '11

17 Feb '11

We have some users who use Evolution as mail client. During migration process we are run Dovecot as proxy of Courier-imap until any user get migrated depend of ldap attribute. At present, the problem is that with Thunderbird and other clients, The client run as normal. But Evolution can't get new messages or Suscription list. Capabilities on Courier and Dovecot are differents, but when a connection is proxied to Courier, Courier print the Capabilitites. Any body has any idea of this? -- Antonio Pérez-Aranda Alcaide aperezaranda(a)yaco.es Yaco Sistemas S.L. http://www.yaco.es/ C/ Rioja 5, 41001 Sevilla Teléfono +34 954 50 00 57 Fax +34 954 50 09 29

3 11

[Dovecot] Dovecot 2.0.9/1.2.11: crash in driver-pgsql.c
by dovecot＠excellency.fr 17 Feb '11

17 Feb '11

Hello, I finally took time to isolate and reproduce a long time Dovecot crash that is bugging me since at least Dovecot 1.0.x. It's still there in Dovecot 2.0.9, which I just tested. Here is the backtrace from my logs: Feb 16 16:56:57 dovecot: auth: Error: pgsql: Query timed out, aborting Feb 16 17:25:16 dovecot: auth: Panic: file driver-pgsql.c: line 349 (result_finish): assertion failed: (db->io == NULL) Feb 16 17:25:16 dovecot: auth: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0 [0x7fdbfe6d4272] -> /usr/lib/dovecot/libdovecot.so.0 [0x7fdbfe6d42d6] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7fdbfe6d46c8] -> dovecot/auth [0x42f555] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0xdd) [0x7fdbfe6df9bd] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x5e) [0x7fdbfe6e05be] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7fdbfe6df618] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fdbfe6cd883] -> dovecot/auth(main+0x2d4) [0x419474] -> /lib/libc.so.6(__libc_start_main+0xe6) [0x7fdbfc4f91a6] -> dovecot/auth [0x40da09] Feb 16 17:26:35 dovecot: master: Error: service(auth): child 140263 killed with signal 6 (core not dumped) Both my userdb and passdb use a PostgreSQL database. The bug happens when the PostgreSQL host suddenly becomes unavailable. I can easily reproduce it by launching Dovecot with my host being up, and then null routing its IP address (to simulate an outage): route add -host 10.0.0.2 reject (10.0.0.2 being the IP address of the PostgreSQL host.) >From that point, if I try to authenticate (using a POP3 connection, for instance), dovecot-auth will crash one or two minutes later. In Dovecot 2.0.9, despite the crash, authentication will resume once the PostgreSQL is back up (which can be simulated with: route delete -host 10.0.0.2 reject). In Dovecot 1.x however (latest version I tested is 1.2.11), it will stay in this state forever (logs are flooded with Password query failed: Not connected to database.) Core dump is available on http://paste.excellency.fr/dovecot.core # dovecot -n # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.37 x86_64 Debian 5.0.6 auth_mechanisms = plain login auth_username_chars = disable_plaintext_auth = no first_valid_gid = 2000 first_valid_uid = 2000 listen = *, [::] log_timestamp = "%Y-%m-%d %H:%M:%S " mail_fsync = always mail_location = maildir:~/ mail_max_userip_connections = 100 mail_nfs_index = yes mail_nfs_storage = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mmap_disable = yes passdb { args = /etc/dovecot/dovecot-pgsql.conf driver = sql } plugin { quota = maildir sieve = filter.sieve sieve_max_actions = 64 sieve_max_redirects = 100 } protocols = imap pop3 service auth { unix_listener auth-client { mode = 0666 } } service imap-login { service_count = 0 } service imap { process_limit = 1024 } service managesieve-login { service_count = 0 } service pop3-login { service_count = 0 } service pop3 { process_limit = 1024 } ssl_cert = </etc/ssl/certs/my.pem ssl_key = </etc/ssl/private/my.key userdb { args = /etc/dovecot/dovecot-pgsql.conf driver = sql } protocol imap { imap_client_workarounds = delay-newmail mail_plugins = quota imap_quota } protocol pop3 { mail_plugins = quota } # cat /etc/dovecot/dovecot-pgsql.conf driver = pgsql connect = host=10.0.0.2 dbname=foobar user=dovecot password=foobar default_pass_scheme = CRYPT password_query = SELECT password FROM mailbox WHERE domain = '%d' AND local_part = '%n' user_query = SELECT home, uid, gid, system_user, quota_rule FROM mailbox WHERE domain = '%d' AND local_part = '%n' -- Cyril

3 2

[Dovecot] Cannot subscribe to shared folders
by Lukas Haase 17 Feb '11

17 Feb '11

Hi, I am unable to subscribe to shared folders with Thunderbird although everything seems to be correct (dovecot 1.2). What does not work? I click "Subscribe..." in Thunderbird. The shared mailboxes are listed, all with [ ] (no checkbox). Therefore I mark them [x], click "subscribe" and "OK". However, the folders do not appear in the folder tree. Furthermore, when I again click "Subscribe..." the folders do NOT have the mark [x] any more. This happens on different computers with *new* profiles. My configuration is the following (maybe that is the problem?!): namespace private { separator = . prefix =. inbox = no list = no hidden = yes location = maildir:~/Maildir-root subscriptions = yes } namespace private { separator = . prefix = INBOX. inbox = yes subscriptions = yes } namespace public { separator = . prefix = Shared. location = maildir:/var/mail/shared:INDEX=~/Maildir/shared:LAYOUT=fs subscriptions = no } The folder /var/mail/shared has appropriate permissions/form: * Owner: $user:mailuser (all users are member of "mailuser") * All files 0660, all folders 0770 * There exists a dovecot-shared in every folder * There exists a dovecot-acl in each folder where each user has the permissions "lrwstie" I switched on "mail_debug = yes" but no messages. However, the file ~/Maildir-root/subscriptions contains the subscribed folders: Shared.Group1 Shared.Group2 Shared.Group1/Test Furthermore, I switched on rawlog. Everything seems to be OK as well: .in-file: 2 subscribe "Shared.Group1" 3 lsub "" "INBOX.*" 5 lsub "" "Shared.*" 6 list "" "INBOX" .out-file: 2 OK Subscribe completed. [...] I am pretty sure it already worked. But it stopped working. Does anybody have a hint what could be the problem? Regards, Luke

2 4

[Dovecot] dsync conversion error
by Frank Bonnet 17 Feb '11

17 Feb '11

Hello When converting mailboxes I get that kind of error messages : dsync(xxx): Error: Next message unexpectedly lost from mbox file /user/xxx/mail/Sent at 9580036 (cached) the dsync command I run is dsync -u xxx mirror mbox:/user/xxx/mail:INBOX=/var/mail/xxx Dovecot mail_location configuration is : mail_location = mbox:~/MailSpace/Mailboxes:DIRNAME=Mbox-Messages:INDEX=~/Index:CONTROL=~/Kontroll Note that ONLY happen on users's "Sent" folder, never on any other folder Thanks for any advices.

2 2

[Dovecot] Some questions about mdbox
by Maria Arrea 16 Feb '11

16 Feb '11

Hello I am starting testing with dovecot 2.0.9. In my //etc/dovecot/dovecot.conf/ I have the following line: /mail_location = mdbox:/mydomain.org/%2.26Hn/%2.200Hn/%n:INDEX=/mydomain.org/indexes/%2.26Hn… These are my questions (a long mail): *1º How should I restore from backup with mdbox & separate indexes?* I have read in dovecot list archive about restoring the full user mailbox to a temp location and then run something like: /doveadm import mdbox:/tmp/restored_mailbox/ "" savedsince 2011-01-01 / In my schema indexes and mail stores are in separated paths. What (and where!) should I copy to the temp location and how do I restore it? *2º Disaster recovery: indexes & mail store mismatch* When creating backups, indexes and mail store are in different paths. My backup takes several hours, so indexes are copied BEFORE mail stores. Is the backup consistent? What would happens if I lose indexes? Can I still recover the messages? Reading the doc it seems I would lose all the message flags and all the messages would be in INBOX, is this right? *3º Performance tuning with mdbox* I have read in dovecot archive about using mdbox_rotate_size & mdbox_rotate_interval to balance between a lot of smaill files (like Maildir) or really big files (a "variant" of mbox). I have tested with a big vaule of mdbox_rotate_size (100 MB) and is a complete disaster for backup:if a user delete a message, then a nightly cronjob purges the mailbox (refcount=0) a new mdbox .m file is created!. So deleting a 1 k message from the mailbox of an user is a new 99 MB file to backup. Any hint about this? I am also using ext4 (rw,noatime,nodiratime,commit=15,data=writeback) and I have read about mdbox_preallocate_space. I will try it, any hint about this setup? This is not still in production, so I have no idea about IOPs and user concurrency, but I have several thousand users. *4º Compression with mdbox* I am reading about compression in mdbox (zlib/bzlib). I am starting with zlib, has anyone experience with bzlib? How much load is bzlib against zlib and how much delay adds to user experience (and space savings vs zlib???) ? *5º Recover a message with refcount=0* Is there any way to recover a message with refcount=0 without using a backup? I am thinking about the following plan: /1º a user gets her mail via pop3 (or delete a message via imap and empties trash) 2º the message is still stored in my mail server, but has refcount=0 3º nightly backup is performed 4º nightly cronjob purges the mailbox (deleting messaged with refcount=0)/ Using this schema the backup is bigger, but If I can recover files with refcount=0 from backup we can give our users a safeguard against accidentaly deleted pop3 mail. Thank you very much for your responsed and sorry for my (awful) english! Maria

3 2

Re: [Dovecot] Some questions about mdbox
by Maria Arrea 16 Feb '11

16 Feb '11

Hello everybody I have found that with bzip compression level 6 I have a better balance between compression and decompression speed with dovecot+mdbox, but still too slow compared with zlib. Does anybody have any other hint about my original questions (see below) Regards Maria Arrea ----- Original Message ----- From: Maria Arrea Sent: 02/15/11 09:16 PM To: dovecot(a)dovecot.org Subject: [Dovecot] Some questions about mdbox Hello I am starting testing with dovecot 2.0.9. In my //etc/dovecot/dovecot.conf/ I have the following line: /mail_location = mdbox:/mydomain.org/%2.26Hn/%2.200Hn/%n:INDEX=/mydomain.org/indexes/%2.26Hn… These are my questions (a long mail): *1º How should I restore from backup with mdbox & separate indexes?* I have read in dovecot list archive about restoring the full user mailbox to a temp location and then run something like: /doveadm import mdbox:/tmp/restored_mailbox/ "" savedsince 2011-01-01 / In my schema indexes and mail stores are in separated paths. What (and where!) should I copy to the temp location and how do I restore it? *2º Disaster recovery: indexes & mail store mismatch* When creating backups, indexes and mail store are in different paths. My backup takes several hours, so indexes are copied BEFORE mail stores. Is the backup consistent? What would happens if I lose indexes? Can I still recover the messages? Reading the doc it seems I would lose all the message flags and all the messages would be in INBOX, is this right? *3º Performance tuning with mdbox* I have read in dovecot archive about using mdbox_rotate_size & mdbox_rotate_interval to balance between a lot of smaill files (like Maildir) or really big files (a "variant" of mbox). I have tested with a big vaule of mdbox_rotate_size (100 MB) and is a complete disaster for backup:if a user delete a message, then a nightly cronjob purges the mailbox (refcount=0) a new mdbox .m file is created!. So deleting a 1 k message from the mailbox of an user is a new 99 MB file to backup. Any hint about this? I am also using ext4 (rw,noatime,nodiratime,commit=15,data=writeback) and I have read about mdbox_preallocate_space. I will try it, any hint about this setup? This is not still in production, so I have no idea about IOPs and user concurrency, but I have several thousand users. *4º Compression with mdbox* I am reading about compression in mdbox (zlib/bzlib). I am starting with zlib, has anyone experience with bzlib? How much load is bzlib against zlib and how much delay adds to user experience (and space savings vs zlib???) ? *5º Recover a message with refcount=0* Is there any way to recover a message with refcount=0 without using a backup? I am thinking about the following plan: /1º a user gets her mail via pop3 (or delete a message via imap and empties trash) 2º the message is still stored in my mail server, but has refcount=0 3º nightly backup is performed 4º nightly cronjob purges the mailbox (deleting messaged with refcount=0)/ Using this schema the backup is bigger, but If I can recover files with refcount=0 from backup we can give our users a safeguard against accidentaly deleted pop3 mail. Thank you very much for your responsed and sorry for my (awful) english! Maria

1 0

[Dovecot] Auth processes/login processes count
by Thomas Hummel 16 Feb '11

16 Feb '11

Hello Timo, I'm running dovecot-1.2.12/Maildir/LDAP-passdb/LDAP-prefetched-userdb and I'm experiencing "random" latency (which even seem to make squirrelmail timeout sometimes) at login without noticing particular load on the server. LDAP backends are running fine as far as I know so I was investigating maybe mistuned limits : Dovecot is configured (for ~3000 users) with . each login processed in its own process : #login_process_per_connection = yes #login_max_connections = 256 . counts as : #login_processes_count = 3 login_max_processes_count = 1024 auth default { [...] count = 5 } I've got several questions : 1) how can I count more than login_max_processes_count processes ? # ps aux | egrep -i '(pop3|imap)-login' | wc -l # 1031 2) could be my auth process count the bottleneck ? is there a rule of thumb to deduce it from the login process count ? 3) on dovecot login/master/auth design, is my understanding correct ? - dovecot-1.2.x : . passdb and userdb lookup are done by the auth process, on behalf of either the login process which wants to know if the client's password is correct or of the master process which wants to check if it was actually correct and which wants to have the userdb info. . client connection fd is sent to the master which creates the mail process and transfers the fd to it. - dovecot-2.x : . client connection fd does not go through the master anymore but through a socket in the login/ directory . it's the mail process which ask the auth process for the userdb lookup 4) with LDAP backend using {CRYPT} passwords dans with plaintext auth enabled in dovecot, is the clear password crypted before comparing it to the LDAP crypted password or does dovecot try to simply bind to LDAP with the clear password ? Thanks -- Thomas Hummel | Institut Pasteur <hummel(a)pasteur.fr> | Groupe Exploitation et Infrastructure

2 1