- dovecot - dovecot.org

[Dovecot] unkillable imap process(es) with high CPU-usage
by Arno Wald 15 Feb '09

15 Feb '09

Hello, I am having a problem with my dovecot-daemon. It is forking one or more (I saw up to perhaps 8 of them) imap processes under my user name. These processes are consuming a lot of CPU time and are not killable: > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 8616 arno 20 0 2900 1600 1204 R 98 0.2 1196:38 imap Stopping dovecot does not quit these processes. Killing them (even "kill -9" as root) is not possible. The only solution to get rid of them is to reboot. I have found a mailing list post of one who seemed to have the same problem. He solved it by upgrading to version 1.1. But in my case this did not help. Is this a known problem? What could I check or do? My system: very up to date debian/sid (sidux) kernel 2.6.27-8.slh.1-sidux-686 CPU: Intel(R) Core(TM)2 CPU 4300 @ 1.80GHz Filesystem: ext3fs tried dovecot from sid: 1:1.0.15-2.3 tried dovecot from experimental: 1:1.1.2-3 My dovecot.conf is the original debian configuration with only one line changed into: protocols = imaps # dovecot -n # 1.1.2: /etc/dovecot/dovecot.conf log_timestamp: %Y-%m-%d %H:%M:%S protocols: imaps login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login mail_privileged_group: mail auth default: passdb: driver: pam userdb: driver: passwd I am using dovecot locally on my system with me as the only user. As client I am using thunderbird alias icedove 2.0.0.17-1. Icedove retrieves the Mails from another imap server and sorts them into Maildir-folders in dovecot. I do not know when the imap-process are going mad. It happens after the system (and the mail-client) is up for a while. Do you need more information? Thanks, Arno

10 35

Re: [Dovecot] unkillable imap process(es) with high CPU-usage
by Robert Schetterer 15 Feb '09

15 Feb '09

just again to the list wasnt in reply Robert Schetterer schrieb: > Hi Gene , small tips depending suse > > Gene Steinberg schrieb: >> On Feb 14, 2009, at 9:57 AM, agent59624285 wrote: >> >>> I have this EXACT same problem after upgrading to SuSE 11.1, which >>> uses this >>> exact kernel version!! >>> >>> After reading this, I was excited to think that if I killed the nfsserver >>> daemon (which I had running for no good reason), that it would sort my >>> problem.... >>> >>> Sure enough, my computer - which up to now had been going unresponsive >>> every >>> 24 hours - was running fine for 72 hours and then BOOM... it happened >>> again. >>> >>> Just wanted to let people know that it seems that at the minute, the >>> dovecot >>> that ships with SuSE and the kernel they are using in 11.1 exhibit this >>> problem. ---my reply > > Suse kernels are not like vanilla kernels, > they are deeply patched an mostly have stuff from the next kernel > version allready in > you might use yast for upgrading latest suse kernel it might fixed allready > a short look gives > http://download.opensuse.org/repositories/home:/tristanhoffmann/openSUSE_11… > gives dovecot 1.1.11 which is latest > also loadable by yast or just download rpm and upgrade wtih rpm -u > > standart rep for mail stuff is > http://download.opensuse.org/repositories/server:/mail/openSUSE_11.1/ > but here dovecot is not at latest > but it shouldnt be any big problem downloading src and > prepare the spec with latest src dovecot > and compile by yourself > which is the best way and gives you the suse splitted packs > too and the advantage to see compile stuff > for sure you need some others libs etc cause of dependencies > befor but they are mostly downloadable also by yast > > be aware that suse sometimes splits up funktions > in own packs like > dovecot-autocreate11, dovecot-antispam11 > etc this musnt be the same by versions compile from home suse users > > also help full send a mail to the build service, > normally it takes only days and the newest version gets compiled > > > use http://software.opensuse.org/search > if looking for newer versions of software ----- > >>> Gino >>> >> This is sounding similar to the problem I have with my setup: >> >> 1. High CPU usage. >> >> 2. Can't kill IMAP. >> >> 3. Server becomes unresponsive. >> >> I'm using CentOS 5.2 64-bit version with the latest cPanel. >> >> So what am I missing, other than the problem nobody else is having is >> clearly something they ARE having? >> >> Peace, >> Gene > > -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria

1 0

[Dovecot] digest-md5 problem
by Nigel Kukard 15 Feb '09

15 Feb '09

Info: v1.1.11 OS = Linux 32bit, custom build CPU Arch = i686 Filesystem = NFSv4 Problem: Digest-md5 not working. Tested plain, login and cram-md5 with pop3, imap and smtp/sasl fine. Using digest-md5 doesn't authenticate with the below error.... *** Feb 15 12:46:45 s03-400-134 dovecot: auth(default): client in: AUTH^I1^IDIGEST-MD5^Iservice=pop3^Ilip=10.0.2.134^Irip=10.0.2.1^Ilport=110^Irport=41579^Iresp=<hidden> Feb 15 12:46:45 s03-400-134 dovecot: auth(default): client out: CONT^I1^IcmVhbG09IiIsbm9uY2U9IktvMkFoK0lyeVhHRWtmY1k5TjM1TXc9PSIscW9wPSJhdXRoIixjaGFyc2V0PSJ1dGYtOCIsYWxnb3JpdGhtPSJtZDUtc2VzcyI= Feb 15 12:46:45 s03-400-134 dovecot: auth(default): client in: CONT<hidden> Feb 15 12:46:47 s03-400-134 dovecot: auth(default): client out: FAIL^I1 Feb 15 12:46:47 s03-400-134 dovecot: pop3-login: Disconnected (auth failed, 1 attempts): method=DIGEST-MD5, rip=10.0.2.1, lip=10.0.2.134 *** The decoded reply is ... authzid="USERNAME_HERE",charset=utf-8,cnonce="32d669a1334164b3362bff5f809acf12",digest-uri="pop/SERVER_URL_HERE",nc=00000001,nonce="rDWjRuMzuvbr5TLsinsE0A==",qop=auth,realm="",response=5c7c2ff3284f350fc62c5a2a80d2be64,username="USERNAME_HERE" On the same database cyrus imap & sasl is working 100%. All passwords are in plaintext format. Maybe I"m doing something wrong? Below my dovecot config... # 1.1.11: /etc/dovecot/dovecot.conf # OS: Linux 2.6.27.10 i686 none protocols: imap imaps pop3 pop3s listen: 10.0.2.134 shutdown_clients: no login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login login_greeting: s03-400-134 Secure server ready. max_mail_processes: 100 verbose_proctitle: yes first_valid_uid: 1000 first_valid_gid: 1000 mail_location: maildir:/var/vmail/%h mail_debug: yes mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugins(default): quota imap_quota trash mail_plugins(imap): quota imap_quota trash mail_plugins(pop3): mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(pop3): /usr/lib/dovecot/pop3 imap_client_workarounds(default): delay-newmail outlook-idle netscape-eoh imap_client_workarounds(imap): delay-newmail outlook-idle netscape-eoh imap_client_workarounds(pop3): pop3_enable_last(default): no pop3_enable_last(imap): no pop3_enable_last(pop3): yes pop3_lock_session(default): no pop3_lock_session(imap): no pop3_lock_session(pop3): yes pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh namespace: type: private separator: . prefix: INBOX. inbox: yes list: yes subscriptions: yes auth default: mechanisms: digest-md5 cram-md5 login plain cache_size: 256 user: nobody master_user_separator: * verbose: yes debug: yes passdb: driver: sql args: /etc/dovecot/sql.conf userdb: driver: sql args: /etc/dovecot/sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: dovecot group: mail master: path: /var/run/dovecot/auth-master mode: 432 user: dovecot group: vmail plugin: quota: maildir trash: /etc/dovecot/trash.conf

1 0

[Dovecot] great disappearing email mystery
by dhottinger＠harrisonburg.k12.va.us 14 Feb '09

14 Feb '09

Have there been any issues with dovecot and using outlook express (imap) as an email client? I have had a couple of users come up with random missing emails. Im trying to figure out if it is user error, or something wacky in my mailserver. I cant find anything telling in maillog files or my messages. Dovecot version is 1.0.3. Its ok to reply to me, I get list messages digest. thanks, ddh -- Dwayne Hottinger Network Administrator Harrisonburg City Public Schools "Everything should be made as simple as possible, but not simpler." -- Albert Einstein "The hottest places in Hell are reserved for those who, in times of moral crisis, preserved their neutrality." -- Dante

10 25

Re: [Dovecot] pop3_lock_session question
by Alan Ferrency 14 Feb '09

14 Feb '09

Rob Mangiafico <rmang(a)lexiconn.com> wrote: >On Wed, 4 Feb 2009, Timo Sirainen wrote: >> On Wed, 2009-02-04 at 11:17 -0700, Mark Costlow wrote: >>> Hello, I'm preparing to convert from qpopper + UW-IMAP to dovecot. >>> So far testing has gone very well. One problem we haven't figured >>> out is that long-running POP sessions keep the mailbox locked, so that >>> the MDA times out while trying to deliver. >> .. >> >> Switch to Maildir and the problem goes away. > > We see this as well with mbox and pop3 accesses where some pop3 clients do > not logout (iphone's are the worst offenders) for 30-90 minutes it seems. > Timeout settings in dovecot.conf do not seem to help. procmail backs up > waiting to get access to the inbox to deliver mail. We are having the same problem. It's not a problem with imap, only with pop3. We're using deliver as the MDA. We're also using flock() locking for the mbox: is that what everyone else is using as well? Migrating our servers to maildir globally is also not an option for us. In some cases, even convincing customers to use IMAP isn't going to work. One problem which might be making this worse than it needs to be, is the fact that mbox_lock_flock in mbox-lock.c is not using a blocking flock(); instead, it's polling for a non-blocking lock. This technique can cause lock starvation, if another process is dropping the lock and picking it back up again frequently: other processes will only see the lock as being available if they happen to poll for the lock at just the right instant. A better technique to use here, if it's adequately cross-platform, would be to set an alarm() for the max_wait_time, and use a blocking flock(). If the alarm times out and you don't have a lock, it's a timeout. In the meantime, you're guaranteed to eventually get the lock, if it is dropped. That said: I'm not sure whether this will solve our problem in practice. Why doesn't this happen with imap? Why can't we make pop3 do what imap does? Even if it's inefficient, it's better than hanging all incoming mail delivery while deliver eats up our local concurrency limits. Thanks! Alan Ferrency pair Networks, Inc. alan(a)pair.com

11 33

[Dovecot] Expire failing with error Remote Disconnected
by dovecot＠corwyn.net 14 Feb '09

14 Feb '09

I'm trying to follow the instructions from: http://wiki.dovecot.org/Plugins/Expire for the BDB backend. Everything is set up as indicated, but when I run: dovecot --exec-mail ext /usr/libexec/dovecot/expire-tool I get back: Error: read(/var/run/dovecot/dict-server) failed: Remote disconnected I'm sure it's something simple.... Thx rick dovecot -n: # 1.1.8: /etc/dovecot.conf # OS: Linux 2.6.18-92.el5 i686 Red Hat Enterprise Linux Server release 5.2 (Tikanga) ext3 ssl_cert_file: /etc/httpd/certs/_com.crt ssl_key_file: /etc/httpd/certs/.com.key.no.password login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login mail_uid: 1015 mail_gid: 105 mail_location: maildir:/var/spool/mail/%d/%n/Maildir mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugins(default): quota imap_quota expire trash mail_plugins(imap): quota imap_quota expire trash mail_plugins(pop3): quota mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(pop3): /usr/lib/dovecot/pop3 imap_client_workarounds(default): delay-newmail outlook-idle imap_client_workarounds(imap): delay-newmail outlook-idle imap_client_workarounds(pop3): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh auth default: mechanisms: plain login passdb: driver: sql args: /etc/dovecot-sql.conf userdb: driver: passwd userdb: driver: sql args: /etc/dovecot-sql.conf userdb: driver: static socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vuser group: vuser plugin: quota: maildir quota_rule: *:storage=102400000 quota_rule2: Trash:storage=100M trash: /etc/dovecot-trash.conf expire: Trash 14 Trash/* 14 Spam 14 expire_dict: proxy::expire dict: expire: db:/var/lib/dovecot/expire.db Rick Steeves http://www.sinister.net "The journey is the destination"

4 9

[Dovecot] Smooth mailbox to maildir migration
by Luciano Mannucci 14 Feb '09

14 Feb '09

Hi all, I have a postfix+procmail+dovecot installation, now running basicaly on pop3 hence mailbox format, with few thousand users and less than twenty gigabytes mail. I need to move all of them to maildir, to take advantage of IMAP subfolders that do not work with mbox format (so I'm told :). Of course I cannot safely stop the service for more than a bunch of seconds... What I'm about to do: create a second dovecot instance to cope with imap client configured for maildir (and possibly pop3 on another port for those using both), which means a dovecot launched with -c /path/to/alternate.conf and a new /etc/init.d/dovecot2-theRevenge having the proper -i option given to startproc in order to make it ignore the other instance. for each user: convert the mailbox with some sort of mb2mb (which is the better one?) create a .procmailrc with a proper DEFAULT= bearing the magic trailing slash indicating maildir to procmail for each converted user. And I'm done... Have I forgotten something? Is there a better way? Any one else tryed such a thing? Cheers to all, luciano. -- /"\ /Via A. Salaino, 7 - 20144 Milano (Italy) \ / ASCII RIBBON CAMPAIGN / PHONE : +39 2 485781 FAX: +39 2 48578250 X AGAINST HTML MAIL / E-MAIL: posthamster(a)sublink.sublink.ORG / \ AND POSTINGS / WWW: http://www.mannucci.ORG/

4 6

[Dovecot] Weird Dovecot 1.1.6 + pop3s certificate issues
by Bernhard Schmidt 14 Feb '09

14 Feb '09

Hello everyone, I've been asked by a colleague to have a look at some extremely weird dovecot SSL issue they are seeing on one of the student mailservers. They are running dovecot 1.1.6 (yes, I know, a bit old ...) on SLES 10.2 x86_64 with imap(+starttls), imaps, pop3(+starttls) and pop3s enabled. Every couple of weeks the pop3s and pop3+starttls part bail out out. Clients can't connect, when you try openssl s_client you get this: 15960:error:0407006A:rsa routines:RSA_padding_check_PKCS1_type_1:block type is not 01:rsa_pk1.c:100: 15960:error:04067072:rsa routines:RSA_EAY_PUBLIC_DECRYPT:padding check failed:rsa_eay.c:699: 15960:error:1408D07B:SSL routines:SSL3_GET_KEY_EXCHANGE:bad signature:s3_clnt.c:1414: server logs don't show much except for various SSL_accept() errors, including TLS handshaking: SSL_accept() failed: error:1408F455:SSL routines:SSL3_GET_RECORD:decryption failed or bad record mac TLS handshaking: SSL_accept() syscall failed: Connection reset by peer TLS handshaking: SSL_accept() failed: error:1409441B:SSL routines:SSL3_READ_BYTES:tlsv1 alert decrypt error imaps and imap+starttls work fine during the whole time (and always have), although they are much more used. Restarting dovecot fixes the issue for a couple of weeks. Does anyone have an idea where to start debugging? As far as I understand the whole SSL_accept() thing is a giant blackbox for Dovecot, but how comes it only affects POP3? dovecot -n (paths and addresses altered, not sure how much I can reveal without getting shot): # 1.1.6: /mnt//usr/local/etc/dovecot.conf # OS: Linux 2.6.16.60-0.25-smp x86_64 SUSE Linux Enterprise Server 10 # (x86_64) protocols: imap imaps pop3 pop3s listen(default): [ipv6]:143, ipv4:143 listen(imap): [ipv6]:143, ipv4:143 listen(pop3): [ipv6]:110, ipv4:110 ssl_listen(default): [ipv6]:993, ipv4:993 ssl_listen(imap): [ipv6]:993, ipv4:993 ssl_listen(pop3): [ipv6]:995, ipv4:995 disable_plaintext_auth: no login_dir: /usr/local/var/run/dovecot/login login_executable(default): /mnt//usr/local/libexec/dovecot/imap-login login_executable(imap): /mnt//usr/local/libexec/dovecot/imap-login login_executable(pop3): /mnt//usr/local/libexec/dovecot/pop3-login login_process_per_connection: no login_max_connections: 128 max_mail_processes: 2500 mail_uid: campus mail_gid: lmu mail_location: maildir:~/Maildir:INDEX=/home/something/indexes/%-1.1n/%n mmap_disable: yes mail_nfs_index: yes mail_executable(default): /mnt//usr/local/libexec/dovecot/imap mail_executable(imap): /mnt//usr/local/libexec/dovecot/imap mail_executable(pop3): /mnt//usr/local/libexec/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /mnt//usr/local/lib/dovecot/imap mail_plugin_dir(imap): /mnt//usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /mnt//usr/local/lib/dovecot/pop3 pop3_uidl_format(default): %08Xu%08Xv pop3_uidl_format(imap): %08Xu%08Xv pop3_uidl_format(pop3): %08Xv%08Xu namespace: type: private separator: . prefix: INBOX. inbox: yes list: yes subscriptions: yes auth default: worker_max_request_count: 500 passdb: driver: ldap args: /mnt//usr/local/etc/dovecot-ldap.conf userdb: driver: prefetch userdb: driver: ldap args: /mnt//usr/local/etc/dovecot-ldap.conf socket: type: listen master: path: /var/run/dovecot/auth-master mode: 384 user: campus group: lmu plugin: quota: maildir quota_rule: Trash:ignore quota_rule2: *:storage=512M Anyone having an idea? bernhard

1 0

Re: [Dovecot] Dovecot-antispam does not work
by Jehan Pagès 14 Feb '09

14 Feb '09

Hi all, just to say that finally the patch of Marcin worked great. Just my configuration is different from yours to get it work: antispam_signature = X-DSPAM-Signature antispam_trash = trash;Trash;Deleted Items antispam_spam = SPAM;Junk antispam_dspam_binary = /usr/bin/dspam antispam_dspam_args = --client;--source=error;--user;%u antispam_mail_tmpdir = /tmp The difference here is in the syntax. You write "--user=somemail" though in the manual, it is written "--user somemail". But also you were proposing 2 lines "antispam_dspam_args" though in all my tests, when I was doing such a thing, the second line seemed to override the first (in the launch logs, only the second line was displayed, so I guess...). Are you sure that it works exactly the way you intend to, Marcin? And that the "--source=error" (your first line) is indeed added as a parameter to the call of dspam? For the patch to the dovecot-antispam, probably it should not be integrated "as is", but I guess that the return code of call_dspam function must probably be checked and fixed if the call can succeed but still give a failure return code. I set Johannes as a main recipient, maybe you will see this email. Bye and thanks all. Jehan On Wed, Feb 4, 2009 at 12:45 PM, Jehan Pagès <jehan.marmottard(a)gmail.com>wrote: > Hi, > > On Wed, Feb 4, 2009 at 12:40 PM, Marcin Rzepecki <m.rzepecki(a)iem.pw.edu.pl > > wrote: > >> Wed, Feb 04, 2009 at 12:12:44PM +0100, Jehan Pag?s wrote: >> > I saw this piece of code and did the same as you in my change of course. >> > Unfortunately, it didn't work (no logged "marked as SPAM", nor change in >> my >> > stats. I have dovecot 1.1.7-r1 though, so it is probably the reason. >> >> Logging '...markes as SPAM..' lines to syslog is my idea, I can send some >> dspam patches, to make it work. It's usefull for me. > > > Ok. Anyway the stats don't change... > > >> >> Are you using mbox or maildir? If you are using mbox, try to upgrade to >> Dovecot >=1.1.8 - it contains several mbox bug fixes (e.g 'Next message >> unexpectedly lost') which are needed to make antispam plugin work. > > > No maildir... > I am doing some stuffs about another way to interact dspam/dovecot/postfix > and will go back to this issue soon. > > Jehan >

2 1

[Dovecot] "failed to map segment from shared object: Cannot allocate memory"
by Vegard Svanberg 13 Feb '09

13 Feb '09

>From /var/log/auth.log (Dovecot 1.1.4 on Ubuntu 8.10): Feb 10 08:29:06 home dovecot-auth: PAM unable to dlopen(/lib/security/pam_ck_connector.so): libdbus-1.so.3: failed to map segment from shared object: Cannot allocate memory Feb 10 08:29:06 home dovecot-auth: PAM adding faulty module: /lib/security/pam_ck_connector.so English, please? :) Strangely, it seemed to disappear after increasing login_processes_count, login_max_processes_count and/or auth_worker_max_count. -- Vegard Svanberg <vegard(a)svanberg.no> [*Takapa@IRC (EFnet)]

2 3