- dovecot - dovecot.org

[Dovecot] Significant performance problems
by Chris Hobbs 07 Oct '10

07 Oct '10

Hi all, I'm sure my issues are a result of misconfiguration, but I'm hoping someone can point me in the right direction. I'm getting pressure to move us back to GroupWise, which I desperately want to avoid :-/ We're running dovecot 1.2.9 on Ubuntu 10.4 LTS+postfix. The server is a VM with 1 vCPU and 4GB of RAM. We serve about 10,000 users with anywhere from 500-1000 logged in at any one time. Messages are stored in Maildir format on two NFS servers (one for staff, the other for students). Today I implemented the "High performance" setup described here: http://wiki.dovecot.org/NFS (mainly moving indexes off of NFS, since I'm only using the one server). I also added imapproxy to our webmail client server (SOGo). The vast majority of our users come in over the web. We currently see load averages spiking into the 20-30 range. When this happens, service crawls to a near standstill, and ultimately the SOGo client starts crashing out. I'm wondering if anything jumps out at anybody here - feel free to mock if/when you find an obvious configuration problem. I just want it to work :-) dovecot -n # 1.2.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-25-server x86_64 Ubuntu 10.04.1 LTS log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s managesieve listen(default): * listen(imap): * listen(pop3): * listen(managesieve): *:2000 ssl_cert_file: /etc/dovecot/certs/mail_nhusd_k12_ca_us.crt ssl_key_file: /etc/dovecot/certs/mail_nhusd_k12_ca_us.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_executable(managesieve): /usr/lib/dovecot/managesieve-login login_process_per_connection: no login_process_size: 512 login_processes_count: 20 login_max_processes_count: 3000 login_max_connections: 64 max_mail_processes: 2048 mail_max_userip_connections(default): 20 mail_max_userip_connections(imap): 20 mail_max_userip_connections(pop3): 10 mail_max_userip_connections(managesieve): 10 mail_access_groups: staffmailusers mail_privileged_group: dovecot mail_uid: mail mail_gid: 502 mail_location: maildir:~/Maildir:INDEX=/var/indexes/%u mail_nfs_storage: yes mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_executable(managesieve): /usr/lib/dovecot/managesieve mail_plugins(default): acl imap_acl quota imap_quota expire mail_plugins(imap): acl imap_acl quota imap_quota expire mail_plugins(pop3): mail_plugins(managesieve): mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 mail_plugin_dir(managesieve): /usr/lib/dovecot/modules/managesieve namespace: type: private separator: / inbox: yes list: yes subscriptions: yes namespace: type: shared separator: / prefix: shared/%%u/ location: maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u list: children lda: deliver_log_format: %$ -- FROM=%f SUBJECT=%s mail_plugins: cmusieve acl expire log_path: info_log_path: syslog_facility: mail postmaster_address:postmaster@nhusd.k12.ca.us hostname: mail.nhusd.k12.ca.us auth_socket_path: /var/run/dovecot/auth-master auth default: passdb: driver: pam passdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf userdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf socket: type: listen master: path: /var/run/dovecot/auth-master mode: 384 plugin: quota: maildir:User quota quota_rule: *:storage=9G quota_rule2: Trash:storage=200M acl: vfile acl_shared_dict:file:/home/staff/dovecot/shared-mailboxes expire: Trash 7 Trash/* 7 Spam 30 expire_dict: proxy::expire sieve: ~/.dovecot.sieve sieve_dir: ~/sieve sieve_extensions: +imapflags dict: expire: mysql:/etc/dovecot/dovecot-dict-expire.conf -- Chris Hobbs Director, Technology New Haven Unified School District -- This message was scanned by ESVA and is believed to be clean.

7 11

[Dovecot] And again... too many open files
by Joseba Torre 07 Oct '10

07 Oct '10

Hi! Two days ago I upgraded to 1.2.15, and again I the warning "Warning: fd limit 1024 is lower than what Dovecot can use under full load" reappeared. It's a bit weird, because my actual limits are higher. From /etc/security/limits.conf * soft nofile 4096 * hard nofile 8192 root soft nofile 4096 root hard nofile 8192 dovecot soft nofile 4096 dovecot hard nofile 8192 I added the * lines to avoid problems if a new user was involved, and finally supposed it was a bug in the detection process. Today, as expected, dovecot was not responding with the famous "Too many open files" error. Has something changed? Is there some way to know which is the maximum number of files that can be open? Is there some parameter I can change to avoid this? Here's my dovecot -n # 1.2.15: /usr/local/etc/dovecot.conf # OS: Linux 2.6.18-164.11.1.el5 x86_64 Red Hat Enterprise Linux Server release 5.4 (Tikanga) syslog_facility: local1 protocols: imap imaps pop3 pop3s listen(default): *:143 listen(imap): *:143 listen(pop3): *:110 ssl_listen(default): *:993 ssl_listen(imap): *:993 ssl_listen(pop3): *:995 ssl_cert_file: /usr/share/ssl/certs/imapd.pem ssl_key_file: /usr/share/ssl/certs/imapd.pem disable_plaintext_auth: no login_dir: /usr/local/var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login login_process_per_connection: no login_process_size: 256 login_processes_count: 16 login_max_processes_count: 256 login_max_connections: 512 max_mail_processes: 2048 first_valid_uid: 100 mail_location: maildir:~/Maildir:INDEX=/var/dovecot/%u:CONTROL=/var/dovecot nas/%u/control mail_nfs_storage: yes lock_method: dotlock mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 pop3_uidl_format(default): %08Xu%08Xv pop3_uidl_format(imap): %08Xu%08Xv pop3_uidl_format(pop3): %v.%u auth default: cache_size: 1000 cache_ttl: 6000 master_user_separator: * debug: yes passdb: driver: passwd-file args: /usr/local/etc/bloqueados deny: yes passdb: driver: ldap args: /usr/local/etc/dovecot-ldap.conf passdb: driver: passwd-file args: /usr/local/etc/shadow passdb: driver: pam passdb: driver: passwd-file args: /usr/local/etc/dovecot-master.conf master: yes userdb: driver: prefetch userdb: driver: passwd-file args: /usr/local/etc/passwd userdb: driver: ldap args: /usr/local/etc/dovecot-ldap.conf socket: type: listen master: path: /var/run/dovecot/auth-master mode: 384 plugin: quota: fs -- Joseba Torre. Vicegerencia de TICs, área de Explotación

3 6

[Dovecot] dovecot-1.2.12 core
by Thomas Hummel 07 Oct '10

07 Oct '10

Hi Timo, just to report that . dovecot-1.2.12 which ran fine for several month . on FreeBSD 7.2-STABLE (/usr/obj/usr/src/sys/GENERIC amd64) . using Maildir (mailboxes, control and indexes on NFS) has crashed this morning on our server. I found nothing special in the logfile. Attached are dovecot -n output and a gdb backtrace of the core file. Thanks -- Thomas Hummel | Institut Pasteur <hummel(a)pasteur.fr> | Pôle informatique - systèmes et réseau

1 0

[Dovecot] Panic: file index-search-result.c .. assertion failed: (next_uid == mail->uid)
by Lutz Preßler 07 Oct '10

07 Oct '10

Hello, with 2.0.5-0~auto+9 (6ea01abfaea9), haven't seen it earlier (but of course index files have changed): Oct 7 09:40:09 host dovecot: imap(user): Panic: file index-search-result.c: line 87 (search_result_update_search): assertion failed: (next_uid == mail->uid) Oct 7 09:40:09 host dovecot: imap(user): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0 [0x7f964c5b7842] -> /usr/lib/ dovecot/libdovecot.so.0 [0x7f964c5b78aa] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f964c5b7c53] -> /usr/lib/dovecot/libdove cot-storage.so.0(index_search_result_update_flags+0x2d8) [0x7f964c83e4d8] -> /usr/lib/dovecot/modules/lib20_virtual_plugin.so [0x7f9 64a9c4b92] -> /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_storage_sync_init+0x871) [0x7f964a9c5561] -> /usr/lib/dovecot /libdovecot-storage.so.0(mailbox_sync_init+0x31) [0x7f964c8262d1] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x27) [0x 7f964c826a87] -> dovecot/imap(cmd_select_full+0x264) [0x40d964] -> dovecot/imap [0x41047c] -> dovecot/imap [0x41052d] -> dovecot/ima p(client_handle_input+0x45) [0x4106a5] -> dovecot/imap(client_input+0x62) [0x4110d2] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_han dler_run+0xcb) [0x7f964c5c293b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x18) [0x7f964c5c1a58] -> /usr/lib/dovecot/libdoveco t.so.0(master_service_run+0x13) [0x7f964c5b0f93] -> dovecot/imap(main+0x281) [0x419391] -> /lib/libc.so.6(__libc_start_main+0xe6) [0x7f964be3b1a6] -> dovecot/imap [0x408c79] when SELECTing virtual folder "sent" (.sent/dovecot-virtual is INBOX header return-path lutz(a)pressler.de ), IMAP session is terminated. After moving .sent/dovecot.index[.log], everything works fine again. I can send you the broken(?) index files, if appropriate. Lutz -- Lutz Preßler <Lutz.Pressler(a)SerNet.DE> http://www.SerNet.DE/ SerNet Service Network GmbH, Bahnhofsallee 1b, D-37081 Göttingen Tel.: +49-551-370000-2, FAX: +49-551-370000-9 AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen

1 0

Re: [Dovecot] dovecot-lda segfaults
by Andre 07 Oct '10

07 Oct '10

Il giorno 07/ott/2010, alle ore 09.06, Andre ha scritto: > Hi to all. > Since version 2.0.3 (now I am running 2.0.5) I have experienced segfaults on mail delivery, either with dovecot-lda and lmtp. > > At least I succeded in having a core dump, here is the backtrace: > > #0 0x00007f1cca350160 in maildir_sync_set_new_msgs_count () from /usr/lib/dovecot/libdovecot-storage.so.0 > #1 0x00007f1cca34df0a in maildir_transaction_save_commit_pre () from /usr/lib/dovecot/libdovecot-storage.so.0 > #2 0x00007f1cca3491c3 in ?? () from /usr/lib/dovecot/libdovecot-storage.so.0 > #3 0x00007f1cca382ef7 in mail_index_transaction_commit_full () from /usr/lib/dovecot/libdovecot-storage.so.0 > #4 0x00007f1cca348e4b in index_transaction_commit () from /usr/lib/dovecot/libdovecot-storage.so.0 > #5 0x00007f1cc8cccc8e in ?? () from /usr/lib/dovecot/modules/lib20_zlib_plugin.so > #6 0x00007f1cc8edaa31 in ?? () from /usr/lib/dovecot/modules/lib10_quota_plugin.so > #7 0x00007f1cca3281bd in mailbox_transaction_commit_get_changes () from /usr/lib/dovecot/libdovecot-storage.so.0 > #8 0x00007f1cca3281fe in mailbox_transaction_commit () from /usr/lib/dovecot/libdovecot-storage.so.0 > #9 0x00007f1cc8461fd7 in ?? () from /usr/lib/dovecot/libdovecot-sieve.so.0 > #10 0x00007f1cc8459597 in ?? () from /usr/lib/dovecot/libdovecot-sieve.so.0 > #11 0x00007f1cc845992b in sieve_result_execute () from /usr/lib/dovecot/libdovecot-sieve.so.0 > #12 0x00007f1cc846973d in sieve_execute () from /usr/lib/dovecot/libdovecot-sieve.so.0 > #13 0x00007f1cc869f83e in ?? () from /usr/lib/dovecot/modules/lib90_sieve_plugin.so > #14 0x00007f1cca0f4084 in mail_deliver () from /usr/lib/dovecot/libdovecot-lda.so.0 > #15 0x0000000000402cf0 in main () > > I have tried to disable quota and zlib, but I suppose it is not related to those plugins. > What else can I try??? > Andre > I’m so sorry I have not seen the previous thread "[Dovecot] 2.0.5: deliver crashing”. I’ll try that patch too and report to the list. Should it work even for lmtp (that is my favourite delivery method, for its scalability)? Andre

1 0

[Dovecot] dovecot-lda segfaults
by Andre 07 Oct '10

07 Oct '10

Hi to all. Since version 2.0.3 (now I am running 2.0.5) I have experienced segfaults on mail delivery, either with dovecot-lda and lmtp. At least I succeded in having a core dump, here is the backtrace: #0 0x00007f1cca350160 in maildir_sync_set_new_msgs_count () from /usr/lib/dovecot/libdovecot-storage.so.0 #1 0x00007f1cca34df0a in maildir_transaction_save_commit_pre () from /usr/lib/dovecot/libdovecot-storage.so.0 #2 0x00007f1cca3491c3 in ?? () from /usr/lib/dovecot/libdovecot-storage.so.0 #3 0x00007f1cca382ef7 in mail_index_transaction_commit_full () from /usr/lib/dovecot/libdovecot-storage.so.0 #4 0x00007f1cca348e4b in index_transaction_commit () from /usr/lib/dovecot/libdovecot-storage.so.0 #5 0x00007f1cc8cccc8e in ?? () from /usr/lib/dovecot/modules/lib20_zlib_plugin.so #6 0x00007f1cc8edaa31 in ?? () from /usr/lib/dovecot/modules/lib10_quota_plugin.so #7 0x00007f1cca3281bd in mailbox_transaction_commit_get_changes () from /usr/lib/dovecot/libdovecot-storage.so.0 #8 0x00007f1cca3281fe in mailbox_transaction_commit () from /usr/lib/dovecot/libdovecot-storage.so.0 #9 0x00007f1cc8461fd7 in ?? () from /usr/lib/dovecot/libdovecot-sieve.so.0 #10 0x00007f1cc8459597 in ?? () from /usr/lib/dovecot/libdovecot-sieve.so.0 #11 0x00007f1cc845992b in sieve_result_execute () from /usr/lib/dovecot/libdovecot-sieve.so.0 #12 0x00007f1cc846973d in sieve_execute () from /usr/lib/dovecot/libdovecot-sieve.so.0 #13 0x00007f1cc869f83e in ?? () from /usr/lib/dovecot/modules/lib90_sieve_plugin.so #14 0x00007f1cca0f4084 in mail_deliver () from /usr/lib/dovecot/libdovecot-lda.so.0 #15 0x0000000000402cf0 in main () I have tried to disable quota and zlib, but I suppose it is not related to those plugins. What else can I try??? Andre

2 1

[Dovecot] probelm with users logging in
by donovan jeffrey j 07 Oct '10

07 Oct '10

Greetings running dovecot for my first time on an OSX 10.6 server. after a reboot, several users are not able to Authenticate. here is a sample Oct 5 20:22:32 hsmail dovecot[2142]: auth(default): od(juser,127.0.0.1): auth mech=CRAM-MD5 Oct 5 20:22:32 hsmail dovecot[2142]: auth(default): od(juser,127.0.0.1): username=juser, challenge=<6757250558559635.1286324552(a)hsmail.beth.k12.pa.us>, response=26b23b30128bba4f9ba3aca261ec764e Oct 5 20:22:32 hsmail dovecot[2142]: auth(default): od(juser,127.0.0.1): found user in local table: user=juser Oct 5 20:22:34 hsmail dovecot[2142]: auth(default): client out: FAIL 41 user=juser Oct 5 20:22:39 hsmail dovecot[2142]: imap-login: Aborted login (auth failed, 1 attempts): user=<juser>, method=CRAM-MD5, rip=127.0.0.1, lip=127.0.0.1, secured these users were working fine, then poof no access. can someone guide me on how to troubleshoot the dovecot portion. i do have many other users that are fine. -j

2 2

[Dovecot] 2.0.5: deliver crashing
by Ralf Hildebrandt 07 Oct '10

07 Oct '10

>From my log: Oct 6 00:10:36 postamt kernel: [2353838.557216] deliver[2799]: segfault at 48 ip b77dd649 sp bfcf3b48 error 6 in libdovecot-storage.so.0.0.0[b778e000+b8000] Oct 6 00:14:39 postamt kernel: [2354082.014722] deliver[6399]: segfault at 48 ip b7779649 sp bf9ca028 error 6 in libdovecot-storage.so.0.0.0[b772a000+b8000] Oct 6 00:28:12 postamt kernel: [2354894.551201] deliver[13972]: segfault at 48 ip b77a4649 sp bfaa65a8 error 6 in libdovecot-storage.so.0.0.0[b7755000+b8000] Oct 6 00:31:27 postamt kernel: [2355089.844614] deliver[15649]: segfault at 48 ip b77ff649 sp bfee32e8 error 6 in libdovecot-storage.so.0.0.0[b77b0000+b8000] Oct 6 00:31:27 postamt postfix/local[6082]: A396A5CB89: to=<username(a)postamt.charite.de>, relay=local, delay=1.1, delays=0.01/0/0/1, dsn=5.3.0, status=bounced (Command died with status 255: "/usr/bin/maildrop") Please don't ask, I'm still running deliver from /etc/maildroprc, because maildrop is handling vacation message and such. -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | http://www.charite.de

3 4

[Dovecot] User authentication fields size limit
by Andre Nathan 06 Oct '10

06 Oct '10

Hello I'm using authentication via SQL, and I'm storing the full mail_location in the database. This is a long string because we're using multiple partitions and hashing to create this path. It looks like this: # doveadm user andre=sneakymustard.com userdb: andre=sneakymustard.com password : $1$ixW87UeD$fLhORaJS0c5wNFmWqfJdI1 mail : mdbox:/var/spool/imap/partition7/11/andre=sneakymustard.com:INDEX=/var/lib/… quota_rule: *:bytes=1024M Depending on the user's email, this string can be even longer. What I've noticed is that it is truncated when it reaches 128 bytes. Could this size limit be increased? Thanks, Andre

2 3

[Dovecot] WITHIN searching broken?
by Michael M. Slusarz 06 Oct '10

06 Oct '10

Using 2.0.5, I see this: [select INBOX] 1 search SINCE 5-Oct-2010 * SEARCH 16 17 18 19 20 21 1 OK Search completed (0.001 secs). 2 search YOUNGER 86400 * SEARCH 2 OK Search completed (0.000 secs). michael

2 1