- dovecot - dovecot.org

Re: [Dovecot] severe performance problem (mail cache related?)
by Bill Boebel 16 Feb '07

16 Feb '07

On Fri, February 16, 2007 6:39 am, Timo Sirainen <tss(a)iki.fi> said: > On Fri, 2007-02-16 at 03:29 -0800, Dan Price wrote: >> Well there you go, I'm a dope. I think I set mmap_disable=yes because I >> was seeing strange things happening where there would be thousands >> (millions?) of madvise(DONTNEED) calls-- seemingly forever in a loop. I >> have not yet tracked that problem down-- next time I see it, I will >> make sure to get to the bottom of it. > > Actually the only place where madvise(MADV_DONTNEED) is called is in the > file-cache code which is used only with mmap_disable=yes. I'm not sure if this is related or not, but we set mmap_disable=yes last week and since then have seen what appears to be an occasional infinite loops. I have been trying to track down some more details before posting to the list, but here is what I have so far. There is no error logged in the dovecot log when this occurs, and it only stops by "kill -9" on the process... # top -c PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND 7832 virtual 25 0 580 496 376 R 71.5 0.0 70:59 0 imap [user(a)example.com 192.168.1.14] # lsof -p 7832 COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME imap 7832 virtual cwd DIR 147,0 600 2372995 /data1/mail/ex/example.com/us/user imap 7832 virtual rtd DIR 3,3 4096 2 / imap 7832 virtual txt REG 3,3 412996 789553 /usr/libexec/dovecot/imap imap 7832 virtual mem REG 3,3 33856 771355 /usr/lib/dovecot/lib10_quota_plugin.so imap 7832 virtual mem REG 3,3 7720 771364 /usr/lib/dovecot/imap/lib11_imap_quota_plugin.so imap 7832 virtual mem REG 3,3 106912 278626 /lib/ld-2.3.2.so imap 7832 virtual mem REG 3,3 14868 278653 /lib/libdl-2.3.2.so imap 7832 virtual mem REG 3,3 1571824 278633 /lib/tls/libc-2.3.2.so imap 7832 virtual 0u IPv4 -1140164370 TCP vip.mail17a.r4.iad.mlsrvr.com:imap->director2.r2.iad.emailsrvr.com:56195 (ESTABLISHED) imap 7832 virtual 1u IPv4 -1140164370 TCP vip.mail17a.r4.iad.mlsrvr.com:imap->director2.r2.iad.emailsrvr.com:56195 (ESTABLISHED) imap 7832 virtual 2w FIFO 0,5 3154802927 pipe imap 7832 virtual 3r CHR 1,9 599160 /dev/urandom imap 7832 virtual 4r FIFO 0,5 3154802928 pipe imap 7832 virtual 5w FIFO 0,5 3154802928 pipe imap 7832 virtual 6r FIFO 0,5 3154802929 pipe imap 7832 virtual 7w FIFO 0,5 3154802929 pipe imap 7832 virtual 8u REG 22,1 11112 6602923 /var/index/data1/mail/ex/example.com/us/user/.Elementos enviados/dovecot.index imap 7832 virtual 9u REG 22,1 50824 6602903 /var/index/data1/mail/ex/example.com/us/user/.Elementos enviados/dovecot.index.log imap 7832 virtual 10u REG 147,0 3074 6346008 /data1/mail/ex/example.com/us/user/maildirsize imap 7832 virtual 11w REG 147,0 905650 7062984 /data1/mail/ex/example.com/us/user/.Elementos enviados/tmp/1171483586.P7832Q0M770594.mail17a.r4.iad.mlsrvr.com # strace -p 7832 read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable) read(0, 0x96ca1f0, 3992) = -1 EAGAIN (Resource temporarily unavailable)

2 2

[Dovecot] imap-login crash with RC19
by Nicolas STRANSKY 16 Feb '07

16 Feb '07

Hi Timo, Using RC19, I've have the following crash. If there was a core file, I've got no idea where it's gone... Jan 25 10:35:10 rouge dovecot: imap-login: file client.c: line 528 (client_unref): assertion failed: (client->refcount > 0) Jan 25 10:35:10 rouge dovecot: child 25498 (login) killed with signal 6 Best regards, -- Nico On réalise qu'une femme est de la dynamite quand on la laisse tomber. -+- Marcel Pagnol -+-

2 4

[Dovecot] Wiki to doc/ in tarball
by Timo Sirainen 16 Feb '07

16 Feb '07

So I finally started building a script that does it. Anyone interested in helping with it so that it produces readable output? First step is getting raw wiki pages. I'll do this directly by getting them from the filesystem. You can get them from here: http://dovecot.org/tmp/wikipages.tar.gz Next there's http://dovecot.org/tmp/wiki2txt.pl which is my horrible Perl script that kind of works now to produce almost readable text files. Its current output is in http://dovecot.org/tmp/wikitxt.tar.gz One thing I'm wondering about is how to make the directory layout work. For example there is "UserDatabase" and "UserDatabase/Static" pages. Currently I'm placing the latter into "_UserDatabase/Static" but I don't think that's the best solution. I was thinking about adding some "include" and "exclude" lists. If a page is in excluce list it's skipped. If it's in neither list it prints a warning but still includes it. So this makes it easy to find out the new pages and figure out if they should be included in the documentation.

3 2

[Dovecot] Index Prob
by Kevin Richter 16 Feb '07

16 Feb '07

Hi, i have a problem with rc19 (sorry, i have no time at the moment to uptime) and homedirs and maildirs mounted via NFS on Solaris10. At the beginning of the following log file everything is ok. At the end it cannot access the index... and just deleted the whole mailbox! It was empty... so it was no problem. Any ideas? Thanks. Bye, Kevin dovecot: Feb 15 21:17:52 Info: auth(default): client in: AUTH 1 PLAIN service=POP3 secured lip=147.x.x.x rip=89.x.x.x resp=<hidden> dovecot: Feb 15 21:17:52 Info: auth(default): client out: CONT 1 dovecot: Feb 15 21:17:52 Info: auth(default): client in: CONT<hidden> dovecot: Feb 15 21:17:52 Info: auth(default): client out: OK 1 user=ssal0041 dovecot: Feb 15 21:17:52 Info: auth(default): master in: REQUEST 111 2754 1 dovecot: Feb 15 21:17:52 Info: auth(default): master out: USER 111 ssal0041 system_user=ssal0041 uid=3092 gid=1100 home=/home/ssal0041 dovecot: Feb 15 21:17:52 Info: pop3-login: Login: user=<ssal0041>, method=PLAIN, rip=89.x.x.x, lip=147.x.x.x, TLS dovecot: Feb 15 21:17:52 Info: POP3(ssal0041): Effective uid=3092, gid=1100 dovecot: Feb 15 21:17:52 Info: POP3(ssal0041): mbox: data=/home/ssal0041/mail/:INBOX=/var/mail/ssal0041 dovecot: Feb 15 21:17:52 Info: POP3(ssal0041): mbox: root=/home/ssal0041/mail, index=/home/ssal0041/mail, inbox=/var/mail/ssal0041 dovecot: Feb 15 21:17:55 Error: POP3(ssal0041): Timeout while waiting for shared lock for index file /home/ssal0041/mail/.imap/INBOX/dovecot.index dovecot: Feb 15 21:19:52 Error: POP3(ssal0041): Timeout while waiting for shared lock for index file /home/ssal0041/mail/.imap/INBOX/dovecot.index dovecot: Feb 15 21:19:55 Error: POP3(ssal0041): Timeout while waiting for shared lock for index file /home/ssal0041/mail/.imap/INBOX/dovecot.index dovecot: Feb 15 21:19:55 Error: POP3(ssal0041): Couldn't open INBOX: Internal error occurred. Refer to server log for more information. [2007-02-15 21:15:55] dovecot: Feb 15 21:19:55 Info: POP3(ssal0041): No INBOX for user top=0/0, retr=0/0, del=0/0, size=0 dovecot: Feb 15 21:21:29 Info: POP3(ssal0041): Disconnected top=0/0, retr=0/0, del=0/0, size=0 dovecot: Feb 15 21:21:29 Info: POP3(ssal0041): Disconnected top=0/0, retr=0/0, del=0/0, size=0

2 2

[Dovecot] Son't have IMAPS
by Mário Gamito 16 Feb '07

16 Feb '07

Hi, I've installed dovecot and i'm pretty sure it has SSL support built in. But i don't have IMAPS. I get all sort of errors. Thunderbird tells: "The connection was refused". If i disable imaps, imap works just fine. If i enable it, none of them work. The certificates in /etc/ssl/certs and /etc/ssl/private are the same ? Any help would be appreciated. Warm Regards, Mário Gamito ## ## SSL settings ## # IP or host address where to listen in for SSL connections. Defaults # to above if not specified. ssl_listen = *:943 # Disable SSL/TLS support. #ssl_disable = no # PEM encoded X.509 SSL/TLS certificate and private key. They're opened before # dropping root privileges, so keep the key file unreadable by anyone but # root. Included doc/mkcert.sh can be used to easily generate self-signed # certificate, just make sure to update the domains in dovecot-openssl.cnf ssl_cert_file = /etc/ssl/certs/dovecot.pem ssl_key_file = /etc/ssl/private/dovecot.pem # If key file is password protected, give the password here. Alternatively # give it when starting dovecot with -p parameter. #ssl_key_password = theedge # File containing trusted SSL certificate authorities. Usually not needed. # The CAfile should contain the CA-certificate(s) followed by the matching # CRL(s). CRL checking is new in dovecot .rc1 #ssl_ca_file = # Request client to send a certificate. #ssl_verify_client_cert = no # How often to regenerate the SSL parameters file. Generation is quite CPU # intensive operation. The value is in hours, 0 disables regeneration # entirely. #ssl_parameters_regenerate = 168 # SSL ciphers to use #ssl_cipher_list = ALL:!LOW # Show protocol level SSL errors. #verbose_ssl = no ## ## Login processes ## # Directory where authentication process places authentication UNIX sockets # which login needs to be able to connect to. The sockets are created when # running as root, so you don't have to worry about permissions. Note that # everything in this directory is deleted when Dovecot is started. login_dir = /var/run/dovecot/login

2 3

[Dovecot] Dovecot forward problems
by Venilton Junior 16 Feb '07

16 Feb '07

Hi Folks, I setted up a mail server based on this how-to http://wanderingbarque.com/howtos/mailserver/mailserver.html All things are working thing as you can see in my server log. Off course, that I've suppressed some informations. postfix/smtpd[16483]: connect from unknown[myip] postfix/smtpd[16483]: 62C9034002: client=unknown[myip] postfix/cleanup[16777]: 62C9034002: message-id=<20070215182523.62C9034002@mysmtpserver> postfix/qmgr[16457]: 62C9034002: from=<me@domain>, size=360, nrcpt=1 (queue active) dovecot: auth(default_with_listener): master in: USER 1 user@domain service=deliver dovecot: auth(default_with_listener): ldap(user@domain): user search: base=dc=example,dc=com scope=subtree filter=(&(objectClass=VirtualMailAccount)(mail=user@domain)(accountActiv e=TRUE)(delete=FALSE)) fields=mail,homeDirectory dovecot: auth(default_with_listener): master out: USER 1 user@domain uid=202 gid=202 deliver(user@domain): msgid=<20070215182523.62C9034002@mysmtpserver>: saved mail to INBOX postfix/pipe[16785]: 62C9034002: to=<user@domain>, relay=dovecot, delay=12, status=sent (domain) postfix/qmgr[16457]: 62C9034002: removed This log is showing that a message was delivered to an user at my mailserver. But, at this time, I've tried to test forwarding features of postfix+dovecot+ldap. I've setted up this user entry (forwardActive=TRUE) in my ldap server. Here follows some entries that I put at my main.cf # Accounts accounts_server_host = $ldap_server_host accounts_search_base = $ldap_search_base accounts_query_filter = (&(&(objectClass=VirtualMailAccount)(mail=%s))(forwardActive=FALSE)(acco untActive=TRUE)(delete=FALSE)) accounts_result_attribute = mailbox accounts_cache = no virtual_transport = dovecot virtual_mailbox_base = /var/lib/imap/vmail/domains virtual_mailbox_maps = ldap:accounts virtual_mailbox_domains = ldap:domains virtual_minimum_uid = 202 virtual_uid_maps = static:202 virtual_gid_maps = static:202 virtual_alias_maps = ldap:aliases, ldap:virtualforward home_mailbox = Maildir/ # aliases aliases_server_host = $ldap_server_host aliases_search_base = $ldap_search_base aliases_query_filter = (&(&(objectClass=VirtualMailAlias)(mail=%s))(forwardActive=FALSE)(accoun tActive=TRUE)(delete=FALSE)) aliases_result_attribute = maildrop aliases_cache = no aliases_bind_dn = $ldap_bind_dn aliases_bind_pw = $ldap_bind_pw accountsmap_server_host = $ldap_server_host accountsmap_search_base = $ldap_search_base accountsmap_query_filter = (&(&(objectClass=VirtualMailAccount)(mail=%s))(forwardActive=FALSE)(acco untActive=TRUE)(delete=FALSE)) accountsmap_result_attribute = mail accountsmap_cache = no # Domains domains_server_host = $ldap_server_host domains_search_base = $ldap_search_base domains_query_filter = (&(objectClass=VirtualDomain)(vd=%s)(accountActive=TRUE)(delete=FALSE)) domains_result_attribute = vd domains_bind = no domains_scope = one # VirtualForward virtualforward_server_host = $ldap_server_host virtualforward_search_base = $ldap_search_base virtualforward_query_filter = (&(&(objectClass=VirtualMailAccount)(mail=%s))(forwardActive=TRUE)(accou ntActive=TRUE)(delete=FALSE)) virtualforward_result_attribute = maildrop virtualforward_cache = no virtualforward_bind_dn = $ldap_bind_dn virtualforward_bind_pw = $ldap_bind_pw Any guesses or misconfiguration? I don't have an idea of how can I debug to get the error. Looking forward to get some clues and helps :P Best Regards Venilton C. Junior

2 1

[Dovecot] I keep getting password mismatches
by Mark 16 Feb '07

16 Feb '07

Hey, This is the debug information: auth(default): client in: AUTH 1 PLAIN service=IMAP secured lip=127.0.0.1 rip=127.0.0.1 resp=AG1hcmsAbWFyaw==auth(default): passwd(mark,127.0.0.1): password mismatchauth(default): client out: FAIL 1 user=markimap-login: Disconnected: user=<mark>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured The strange thing is that i can login with ssh using my username and password but i can`t do anything related to dovecot.. i`m using a default .conf file with auth method set as plain and the supported protocols are set to pop and imap.. anyone that could help me with this? Thanx in favor. _________________________________________________________________ Probeer Live.com: je eigen persoonlijke opstartpagina met alleen de dingen die jij belangrijk vindt op één plek. http://www.live.com/getstarted

3 2

[Dovecot] Is it 1.0 yet?
by Marc Perkel 15 Feb '07

15 Feb '07

OK - my last main bug was fixed a few releases ago. I know that there's no end to getting things right but there comes a point when you need to call it done. No one trusts a 1.0 to be perfect but Dovecot is already better that most 2.0 versions of other software. I vote (as if my vote counted) to call it done, declare victory, and move on to 1.01, 1.02 etc. No one is going to trus it till it gets to 1.02 so I say lets bless it and move on.

15 22

[Dovecot] 1.0.rc22 released
by Timo Sirainen 15 Feb '07

15 Feb '07

http://dovecot.org/releases/dovecot-1.0.rc22.tar.gz http://dovecot.org/releases/dovecot-1.0.rc22.tar.gz.sig Found another bad bug in rc19 changes. Wonder why my imaptest catched the bug only in CVS HEAD but not in branch_1_0 even though both had it. Anyway, now the imaptest runs nicely for both, and I'm again optimistic that the bug count is low enough for v1.0 to be released soon :) BTW. My previous benchmarks which showed how mmap_disable=yes and fsync_disable=yes give huge performance benefits are pretty XFS-specific. I did some quick tests with ext3 and the differences were quite small with it. I'll probably do another proper run of tests later. + pop3: Commit the transaction even if client didn't QUIT so cached data gets saved. - Fixed another indexing bug in rc19 and later which caused transactions to be skipped in some situations, causing all kinds of problems. - mail_log_max_lines_per_sec was a bit broken and caused crashes with dovecot -a - BSD filesystem quota was counted wrong. Patch by Manuel Bouyer - LIST: If namespace has a prefix and inbox=no, don't list prefix.inbox if it happens to exist when listing for %.

8 22

[Dovecot] Sieve Vacation virtual support?
by Charles Marcus 15 Feb '07

15 Feb '07

Hello, I'm wanting to enable vacation messages for my users on a new dovecot server that I'm playing with, but want to know if the Dovecot LDA+Sieve vacation message function http://wiki.dovecot.org/LDA/Sieve will work with virtual users? For security reasons, I only want to use virtual domains/users. Thanks... -- Best regards, Charles

2 4