May 2009 - dovecot - dovecot.org

[Dovecot] dovecot weird bug
by Fernando Bertasso 11 May '09

11 May '09

Hi, We have a weird bug happening in our dovecot installation. When I send html emails from many clients and compress them, there are no problem. They are opened and correctly formated. When I send an email from Squirrelmail and compress it, dovecot exits with no errors (there are no coredumps or seg fault - dovecot just exits). But the weird is, it just happens when squirrelmail opens an email (compressed) that was sent by squirrelmail (all other clients can open it and squirrel can open all other emails sent by other clients). (Try with this message - compressing too - it occurs with me). Best Regards, Fernando Dovecot version: 1.2.rc3 Dovecot Details: # 1.2.rc3: /usr/local/dovecot/etc/dovecot.conf # OS: Linux 2.6.18-128.el5 i686 Red Hat Enterprise Linux Server release 5.3 (Tikanga) base_dir: /usr/local/dovecot/var/run/dovecot/ protocols: imap pop3 ssl: no disable_plaintext_auth: no login_dir: /usr/local/dovecot/var/run/dovecot/login login_executable(default): /usr/local/dovecot/libexec/dovecot/imap-login login_executable(imap): /usr/local/dovecot/libexec/dovecot/imap-login login_executable(pop3): /usr/local/dovecot/libexec/dovecot/pop3-login login_greeting: Server ready. login_process_per_connection: no login_processes_count: 1 mail_max_userip_connections: 1024 first_valid_gid: 500 mail_uid: 505 mail_gid: 506 mail_location: maildir:~/Maildir mail_debug: yes mail_max_keyword_length: 128 fsync_disable: yes maildir_copy_preserve_filename: yes mail_drop_priv_before_exec: yes mail_executable(default): /usr/local/dovecot/libexec/dovecot/gdbhelper /usr/local/dovecot/libexec/dovecot/imap mail_executable(imap): /usr/local/dovecot/libexec/dovecot/gdbhelper /usr/local/dovecot/libexec/dovecot/imap mail_executable(pop3): /usr/local/dovecot/libexec/dovecot/pop3 mail_process_size: 512 mail_plugins(default): quota imap_quota zlib mail_plugins(imap): quota imap_quota zlib mail_plugins(pop3): quota zlib mail_plugin_dir(default): /usr/local/dovecot/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/dovecot/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/dovecot/lib/dovecot/pop3 pop3_no_flag_updates(default): no pop3_no_flag_updates(imap): no pop3_no_flag_updates(pop3): yes namespace: type: private separator: . prefix: INBOX. inbox: yes list: yes subscriptions: yes auth default: mechanisms: login plain cache_size: 1024 cache_negative_ttl: 0 worker_max_count: 100 passdb: driver: sql args: /usr/local/dovecot/etc/dovecot-sql.conf userdb: driver: prefetch plugin: quota: maildir Imap commands: 127.000.000.001.54022-127.000.000.001.00143: A009 UID FETCH 1 (FLAGS UID RFC822.SIZE INTERNALDATE BODY.PEEK[HEADER.FIELDS (Date To Cc From Subject X-Priority Importance Priority Content-Type X-UOLHost-Filter)]) 127.000.000.001.00143-127.000.000.001.54022: * 1 FETCH (UID 1 FLAGS (\Seen) RFC822.SIZE 1241 INTERNALDATE "10-May-2009 16:37:39 -0300" BODY[HEADER.FIELDS (DAT E TO CC FROM SUBJECT X-PRIORITY IMPORTANCE PRIORITY CONTENT-TYPE X-UOLHOST-FILTER)] {264} Date: Sun, 10 May 2009 16:37:38 -0300 (BRT) Subject: a From: namba234(a)emailnovocluster.com To: "Namba 234" <namba234(a)emailnovocluster.com> Content-Type: multipart/alternative;boundary="----=_20090510163738_24160" X-Priority: 3 (Normal) Importance: Normal ) A009 OK Fetch completed. 127.000.000.001.54026-127.000.000.001.00143: A002 SELECT "INBOX" 127.000.000.001.00143-127.000.000.001.54026: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 1 EXISTS * 0 RECENT * OK [UIDVALIDITY 1241984317] UIDs valid * OK [UIDNEXT 2] Predicted next UID * OK [HIGHESTMODSEQ 1] A002 OK [READ-WRITE] Select completed. 127.000.000.001.54026-127.000.000.001.00143: A003 EXPUNGE 127.000.000.001.00143-127.000.000.001.54026: A003 OK Expunge completed. 127.000.000.001.54026-127.000.000.001.00143: A004 UID FETCH 1 BODY[2] 127.000.000.001.00143-127.000.000.001.54026: * 1 FETCH (UID 1 BODY[2] {7} 127.000.000.001.54026-127.000.000.001.00143: A005 UID FETCH 1 BODY[] (dovecot finish the process). Here is a copy of the message (uncompressed - opens / compressed - hangs). Return-Path: <namba234(a)emailnovocluster.com> X-Original-To: namba234(a)emailnovocluster.com Delivered-To: namba234(a)emailnovocluster.com Received: from webmail.emailnovocluster.com (localhost.localdomain [127.0.0.1]) by maildev.localdomain (Postfix) with ESMTP id B45ACFE146 for <namba234(a)emailnovocluster.com>; Sun, 10 May 2009 16:37:38 -0300 (BRT) Received: from 189.62.202.143 (SquirrelMail authenticated user namba234(a)emailnovocluster.com) by webmail.emailnovocluster.com with HTTP; Sun, 10 May 2009 16:37:38 -0300 (BRT) Message-ID: < c0b80d709ef3dc829fc06395baca944d.squirrel(a)webmail.emailnovocluster.com> Date: Sun, 10 May 2009 16:37:38 -0300 (BRT) Subject: a From: namba234(a)emailnovocluster.com To: "Namba 234" <namba234(a)emailnovocluster.com> User-Agent: SquirrelMail/1.4.17 MIME-Version: 1.0 Content-Type: multipart/alternative;boundary="----=_20090510163738_24160" X-Priority: 3 (Normal) Importance: Normal ------=_20090510163738_24160 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 8bit a ------=_20090510163738_24160 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: 8bit a<br /> ------=_20090510163738_24160--

2 1

[Dovecot] NTLM configuration
by Cédric Laruelle 11 May '09

11 May '09

Hi again everybody ! Im still stuck with the dovecot ntlm authentication I configured dovecot to use winbind, and I would like winbind to authenticate against samba (samba, winbind and dovecot are running on the same box). Here is the log I have (192.168.0.1 is the server box, 192.168.0.254 the client box) dovecot: May 06 14:52:37 Info: auth(default): new auth connection: pid=25828 dovecot: May 06 14:52:38 Info: auth(default): client in: AUTH 1 NTLM service=imap secured lip=192.168.0.1 rip=192.168.0.254 lport=143 rport=1084 dovecot: May 06 14:52:38 Info: auth(default): client out: CONT 1 dovecot: May 06 14:52:38 Info: auth(default): client in: CONT 1 TlRMTVNTUAABAAAAB4IIogAAAAAAAAAAAAAAAAAAAAAFASgKAAAADw== dovecot: May 06 14:52:38 Info: auth(default): client out: CONT 1 TlRMTVNTUAACAAAADgAOADAAAAAFgomizPYc4ALWKQgAAAAAAAAAAIAAgAA+AAAAQQBMAFYA TlRMTVNTUAACAAAADgAOADAAAAAFgomizPYc4ALWKQgAAAAAAAAAAIAAgAA+QQBS AFUATQACAA4AQQBMAFYAQQBSAFUATQABABAASQBOAFQARQBSAE4AQQBMAAQAHgBhAGkAZABlAHIA ZABvAG4AbgBlAHIALgBjAG8AbQADADAAaQBuAHQAZQByAG4AYQBsAC4AYQBpAGQAZQByAGQAbwBu AG4AZQByAC4AYwBvAG0AAAAAAA== dovecot: May 06 14:52:38 Info: auth(default): client in: CONT 1 TlRMTVNTUAADAAAAGAAYAF4AAAAYABgAdgAAAAAAAABIAAAABgAGAEgAAAAQABAATgAAAAAAAACO AAAABYKIogUBKAoAAAAPZgBmAHMAQQBMAFYAQQBSAFUATQAzABXRN5WNNwAgAAAAAAAAAAAAAAAA AAAAALm1ePVxjdOF1UPe8A/e1D6H0+jlJYQPUA== dovecot: May 06 14:52:38 Info: auth(default): winbind(?,192.168.0.254): user not authenticated: NT_STATUS_NO_LOGON_SERVERS dovecot: May 06 14:52:40 Info: auth(default): client out: FAIL 1 Please help, I really need to set this up and it begins to drive me really crazy Cédric Laruelle

3 5

[Dovecot] dovecot weird bug
by Fernando Bertasso 11 May '09

11 May '09

I found the bug, there are two errors: 1) Squirrelmail is wrongly formating multipart/alternative messages. Look here: ------=_20090510163738_24160 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: 8bit a<br /> *<------ HERE* ------=_20090510163738_24160-- All other clients insert an '\n' here, but squirrelmail. 2) When this king of message (above) is compressed (in order to use zlib plugin), dovecot dies. Best Regards, Fernando

1 0

[Dovecot] Dovecot not accepting connections
by Jim 11 May '09

11 May '09

Hi, new user here, trying to track down an issue. Sometimes, after starting dovecot, I'll attempt to check mail and will not be able to get a connection to my server. I'll kill dovecot, restart it, and usually it will begin working. When I first ran into this, I first tried auth_debug=yes, restarted dovecot, and had no problems. I'd turn set auth_debug=no, restart dovecot, and would be unable to check mail. After a time I determined that setting the debug wasn't really effecting anything, it was just hit or miss if I'd be able to get a connection. Once I was able to get a connection, things were fine until dovecot started again. Nothing unusual in the logs, if it works or not, I'll just see this at startup: Mar 25 21:34:48 mail-web-server dovecot[664]: dovecot v1.1.13 starting up (core dumps disabled) Mar 25 21:34:48 mail-web-server dovecot[664]: auth-worker(default): mysql: Connected to localhost (postfix) If debug is on and connections aren't being accepted, I never see anything in the logs. Below is my config. Running on OS X 10.5.6. Any info on troubleshooting would be appreciated. mail-web-server:~ wright$ sudo /usr/local/sbin/dovecot -n Password: # 1.1.13: /usr/local/etc/dovecot.conf # OS: Darwin 9.6.0 i386 hfs protocols: imap imaps pop3 pop3s ssl_disable: yes disable_plaintext_auth: no login_dir: /usr/local/var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login login_user: _dovecot login_max_processes_count: 64 max_mail_processes: 128 first_valid_uid: 27 last_valid_uid: 27 first_valid_gid: 27 last_valid_gid: 27 mail_privileged_group: postfix mail_uid: 27 mail_gid: 27 mail_location: maildir:/var/mail/vhosts/%d/%n mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 namespace: type: private inbox: yes list: yes subscriptions: yes auth default: mechanisms: plain login cram-md5 passdb: driver: sql args: /usr/local/etc/dovecot-mysql.conf userdb: driver: sql args: /usr/local/etc/dovecot-mysql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix

3 11

[Dovecot] Degeneration of CPU Performance
by Thiago Monaco Papageorgiou 11 May '09

11 May '09

Hello everybody! We have a interesting issue about dovecot behavior here. First, the scenario: We have 2 server running with the same load, one with our old pop3 solution (out of date) and other with Dovecot. We realized that dovecot are comsuming more CPU, and this consumption is growing day by day. When we starts dovecot, it runs between 40%-45% of CPU consumption and our old solution runs on 30-35%. This is quite acceptable, so no problem here. The problem is one day after it jumps to 45%-55% of cpu comsunption while the old pop3 solution runs on the same CPU consumption of one day before (30%-35). I attached a graph with this information. In this graph the green line is dovecot CPU consumption and the blue line is the old solution. In this graph we restart dovecot at monday morning. This restart is noticed as a big fall of CPU consumption(green line) in the graph. After that, it stay between 40%-45%, one day after it runs on 45%-55% and next day, it reaches 50%-60%! You can notice that the blue line (old pop3 solution) runs with the same behavior, so we have the same load. We have other monitoring informations that confirm it. I looked for some bugs about this, but I didn't find anything. The question is: Is there any dovecot problem or wrong configuration that could cause this degeneration of CPU performance? Any sugestion about what can be the cause of this? I attached dovecot -n output. Two important information: we store the mailboxes in NFS and we are only using pop3 protocol. Thanks in advance! -- Thiago Monaco Papageorgiou <thiago.monaco(a)corp.terra.com.br> ________________________ Terra Networks Brasil S/A Tel: +55 (51) 3284-4274 # 1.1.6: /usr/local/dovecot/etc/dovecot.conf # OS: Linux 2.6.9-78.0.1.ELsmp i686 CentOS release 4.7 (Final) syslog_facility: local2 protocols: pop3 listen: *:1110 ssl_disable: yes disable_plaintext_auth: no login_dir: /usr/local/dovecot/var/run/dovecot/login login_executable: /usr/local/dovecot/libexec/dovecot/pop3-login login_user: nobody login_log_format_elements: user=<%u> idperm=<%i> command=%t method=%m rip=%r lip=%l %c login_process_per_connection: no login_process_size: 32 login_processes_count: 100 login_max_processes_count: 500 login_max_connections: 100 mail_uid: popmgr mail_gid: popmgr mail_cache_min_mail_count: 100 mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes lock_method: dotlock maildir_copy_preserve_filename: yes mail_executable: /usr/local/dovecot/libexec/dovecot/pop3 mail_plugins: hashdir mail_plugin_dir: /usr/local/dovecot/lib/dovecot/pop3 mail_log_prefix: %Us: user %u (%e): pop3_uidl_format: %f pop3_client_workarounds: outlook-no-nuls oe-ns-eoh auth default: mechanisms: plain trrproxy_v1 worker_max_count: 64 passdb: driver: trrpam args: extra_fields=short_name,quota_caixa,mail_folders_limit,id_perm,mail_imap %s_dovecot userdb: driver: prefetch plugin: quota: trrquota uidlist_create_form: 1

6 8

[Dovecot] Unit testing
by Timo Sirainen 10 May '09

10 May '09

Unit testing is slowly going forward in Dovecot. In v2.0 tree all the unit tests are now run when you do "make check". They're also run on "make distcheck", which is what I use when building releases or nightly snapshots. So as soon as I start building nightly v2.0 snapshots I should start getting mails every night if the unit tests fail. And I can no longer make a release if a unit test fails. (Unit tests never get run manually - I once fixed a unit test that had been failing for months.) Besides that, I started trying to add unit tests to lib-index. After some hours of refactoring the code I finally managed to get mail-transaction-log-view and mail-index-transaction to a condition where unit tests could be added to them without too much pain. I didn't really add that many unit tests there yet though. I'll probably slowly continue adding more tests and refactoring the code more to allow more unit tests, but if anyone else finds they have nothing better to do, now's a good time to add more tests :) Especially lib/, lib-mail/ and lib-imap/ should already be easily testable and they're missing a lot of tests.

1 0

[Dovecot] dovecot slowing down
by P.A 08 May '09

08 May '09

Hi, not really a dovecot issue but, im running dovecot 1.0.10 with no issues for awhile now J. However I ran into an issue today where postfix didn't have enough processes to handle incoming smtp connections. This for whatever reason was causing pop/imap (dovecot) to take a long time to respond on the appropriate port. After looking at server and finding nothing wrong I stopped postfix which got rid of the delay on port 110/143. I then turned on postfix again and saw the delay for port 25/110/143. What I then did was adjust/increase the default_process_limit in postfix which took care of the problem. Question is why would not enough processes in postfix slow other services down like dovecot 110/143? There was plenty of bandwith/mem/cpu left.

1 0

[Dovecot] Problem with IMAP (rename and delete non existent folder).
by Lenthir 08 May '09

08 May '09

I don't know if this problem exists only in my version of dovecot, or in all... To reproduce BUG do this (use gdb): telnet localhost 143 o login addr(a)email.sth verystrong_pass o delete "kj-test-folder" o rename "kj-test-folder" o rename "kj-test-folder" "other-folder" SIGABRT Of course kj-test-folder is not exist. BT: #31 0xb7da0022 in free () from /lib/tls/libc.so.6 #32 0x0813313a in pool_system_free (pool=0x8169060, mem=0x8146161) at mempool-system.c:92 #33 0x080de02f in mailbox_list_clear_error (list=0x81885e8) at mailbox-list.c:762 #34 0x080dd5ae in mailbox_list_get_mailbox_name_status (list=0x81885e8, name=0x81a4258 "other-folder", status=0xbfe9cae0) at mailbox-list.c:394 My version of Dovecot is 1.2beta1. Do you know anything about this bug? Please help. -- Konrad

2 5

[Dovecot] Problem with IMAP (rename and delete non existent folder).
by Lenthir 08 May '09

08 May '09

I don't know if this problem exists only in my version of dovecot, or in all... To reproduce BUG do this (use gdb): telnet localhost 143 o login addr(a)email.sth verystrong_pass o delete "kj-test-folder" o rename "kj-test-folder" o rename "kj-test-folder" "other-folder" SIGABRT Of course kj-test-folder is not exist. BT: #31 0xb7da0022 in free () from /lib/tls/libc.so.6 #32 0x0813313a in pool_system_free (pool=0x8169060, mem=0x8146161) at mempool-system.c:92 #33 0x080de02f in mailbox_list_clear_error (list=0x81885e8) at mailbox-list.c:762 #34 0x080dd5ae in mailbox_list_get_mailbox_name_status (list=0x81885e8, name=0x81a4258 "other-folder", status=0xbfe9cae0) at mailbox-list.c:394 My version of Dovecot is 1.2beta1. Do you know anything about this bug? Please help. -- Konrad

3 6

[Dovecot] mailutil?
by Timo Sirainen 08 May '09

08 May '09

dbox format needs at least two tools: 1. Expunging only marks the message having refcount=0. The actual disk space is freed only after calling a tool pointing to the user's dbox dir. 2. Ability to force index resync/rebuild in case it doesn't figure out itself that there's something wrong. I'm starting to think about building a more generic mailutil tool, maybe something similar to what UW-IMAP has. So two questions: - What should the tool be called? Probably not mailutil to avoid confusion. doveutil? dovemail? dovetool? mailtool? - Is there any functionality you'd like it to do?

10 15