February 2009 - dovecot

Re: [Dovecot] pop3_lock_session question
by Alan Ferrency 14 Feb '09

14 Feb '09

Rob Mangiafico <rmang(a)lexiconn.com> wrote: >On Wed, 4 Feb 2009, Timo Sirainen wrote: >> On Wed, 2009-02-04 at 11:17 -0700, Mark Costlow wrote: >>> Hello, I'm preparing to convert from qpopper + UW-IMAP to dovecot. >>> So far testing has gone very well. One problem we haven't figured >>> out is that long-running POP sessions keep the mailbox locked, so that >>> the MDA times out while trying to deliver. >> .. >> >> Switch to Maildir and the problem goes away. > > We see this as well with mbox and pop3 accesses where some pop3 clients do > not logout (iphone's are the worst offenders) for 30-90 minutes it seems. > Timeout settings in dovecot.conf do not seem to help. procmail backs up > waiting to get access to the inbox to deliver mail. We are having the same problem. It's not a problem with imap, only with pop3. We're using deliver as the MDA. We're also using flock() locking for the mbox: is that what everyone else is using as well? Migrating our servers to maildir globally is also not an option for us. In some cases, even convincing customers to use IMAP isn't going to work. One problem which might be making this worse than it needs to be, is the fact that mbox_lock_flock in mbox-lock.c is not using a blocking flock(); instead, it's polling for a non-blocking lock. This technique can cause lock starvation, if another process is dropping the lock and picking it back up again frequently: other processes will only see the lock as being available if they happen to poll for the lock at just the right instant. A better technique to use here, if it's adequately cross-platform, would be to set an alarm() for the max_wait_time, and use a blocking flock(). If the alarm times out and you don't have a lock, it's a timeout. In the meantime, you're guaranteed to eventually get the lock, if it is dropped. That said: I'm not sure whether this will solve our problem in practice. Why doesn't this happen with imap? Why can't we make pop3 do what imap does? Even if it's inefficient, it's better than hanging all incoming mail delivery while deliver eats up our local concurrency limits. Thanks! Alan Ferrency pair Networks, Inc. alan(a)pair.com

11 33

[Dovecot] Expire failing with error Remote Disconnected
by dovecot＠corwyn.net 14 Feb '09

14 Feb '09

I'm trying to follow the instructions from: http://wiki.dovecot.org/Plugins/Expire for the BDB backend. Everything is set up as indicated, but when I run: dovecot --exec-mail ext /usr/libexec/dovecot/expire-tool I get back: Error: read(/var/run/dovecot/dict-server) failed: Remote disconnected I'm sure it's something simple.... Thx rick dovecot -n: # 1.1.8: /etc/dovecot.conf # OS: Linux 2.6.18-92.el5 i686 Red Hat Enterprise Linux Server release 5.2 (Tikanga) ext3 ssl_cert_file: /etc/httpd/certs/_com.crt ssl_key_file: /etc/httpd/certs/.com.key.no.password login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login mail_uid: 1015 mail_gid: 105 mail_location: maildir:/var/spool/mail/%d/%n/Maildir mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugins(default): quota imap_quota expire trash mail_plugins(imap): quota imap_quota expire trash mail_plugins(pop3): quota mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(pop3): /usr/lib/dovecot/pop3 imap_client_workarounds(default): delay-newmail outlook-idle imap_client_workarounds(imap): delay-newmail outlook-idle imap_client_workarounds(pop3): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh auth default: mechanisms: plain login passdb: driver: sql args: /etc/dovecot-sql.conf userdb: driver: passwd userdb: driver: sql args: /etc/dovecot-sql.conf userdb: driver: static socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vuser group: vuser plugin: quota: maildir quota_rule: *:storage=102400000 quota_rule2: Trash:storage=100M trash: /etc/dovecot-trash.conf expire: Trash 14 Trash/* 14 Spam 14 expire_dict: proxy::expire dict: expire: db:/var/lib/dovecot/expire.db Rick Steeves http://www.sinister.net "The journey is the destination"

4 9

[Dovecot] Smooth mailbox to maildir migration
by Luciano Mannucci 14 Feb '09

14 Feb '09

Hi all, I have a postfix+procmail+dovecot installation, now running basicaly on pop3 hence mailbox format, with few thousand users and less than twenty gigabytes mail. I need to move all of them to maildir, to take advantage of IMAP subfolders that do not work with mbox format (so I'm told :). Of course I cannot safely stop the service for more than a bunch of seconds... What I'm about to do: create a second dovecot instance to cope with imap client configured for maildir (and possibly pop3 on another port for those using both), which means a dovecot launched with -c /path/to/alternate.conf and a new /etc/init.d/dovecot2-theRevenge having the proper -i option given to startproc in order to make it ignore the other instance. for each user: convert the mailbox with some sort of mb2mb (which is the better one?) create a .procmailrc with a proper DEFAULT= bearing the magic trailing slash indicating maildir to procmail for each converted user. And I'm done... Have I forgotten something? Is there a better way? Any one else tryed such a thing? Cheers to all, luciano. -- /"\ /Via A. Salaino, 7 - 20144 Milano (Italy) \ / ASCII RIBBON CAMPAIGN / PHONE : +39 2 485781 FAX: +39 2 48578250 X AGAINST HTML MAIL / E-MAIL: posthamster(a)sublink.sublink.ORG / \ AND POSTINGS / WWW: http://www.mannucci.ORG/

4 6

[Dovecot] Weird Dovecot 1.1.6 + pop3s certificate issues
by Bernhard Schmidt 14 Feb '09

14 Feb '09

Hello everyone, I've been asked by a colleague to have a look at some extremely weird dovecot SSL issue they are seeing on one of the student mailservers. They are running dovecot 1.1.6 (yes, I know, a bit old ...) on SLES 10.2 x86_64 with imap(+starttls), imaps, pop3(+starttls) and pop3s enabled. Every couple of weeks the pop3s and pop3+starttls part bail out out. Clients can't connect, when you try openssl s_client you get this: 15960:error:0407006A:rsa routines:RSA_padding_check_PKCS1_type_1:block type is not 01:rsa_pk1.c:100: 15960:error:04067072:rsa routines:RSA_EAY_PUBLIC_DECRYPT:padding check failed:rsa_eay.c:699: 15960:error:1408D07B:SSL routines:SSL3_GET_KEY_EXCHANGE:bad signature:s3_clnt.c:1414: server logs don't show much except for various SSL_accept() errors, including TLS handshaking: SSL_accept() failed: error:1408F455:SSL routines:SSL3_GET_RECORD:decryption failed or bad record mac TLS handshaking: SSL_accept() syscall failed: Connection reset by peer TLS handshaking: SSL_accept() failed: error:1409441B:SSL routines:SSL3_READ_BYTES:tlsv1 alert decrypt error imaps and imap+starttls work fine during the whole time (and always have), although they are much more used. Restarting dovecot fixes the issue for a couple of weeks. Does anyone have an idea where to start debugging? As far as I understand the whole SSL_accept() thing is a giant blackbox for Dovecot, but how comes it only affects POP3? dovecot -n (paths and addresses altered, not sure how much I can reveal without getting shot): # 1.1.6: /mnt//usr/local/etc/dovecot.conf # OS: Linux 2.6.16.60-0.25-smp x86_64 SUSE Linux Enterprise Server 10 # (x86_64) protocols: imap imaps pop3 pop3s listen(default): [ipv6]:143, ipv4:143 listen(imap): [ipv6]:143, ipv4:143 listen(pop3): [ipv6]:110, ipv4:110 ssl_listen(default): [ipv6]:993, ipv4:993 ssl_listen(imap): [ipv6]:993, ipv4:993 ssl_listen(pop3): [ipv6]:995, ipv4:995 disable_plaintext_auth: no login_dir: /usr/local/var/run/dovecot/login login_executable(default): /mnt//usr/local/libexec/dovecot/imap-login login_executable(imap): /mnt//usr/local/libexec/dovecot/imap-login login_executable(pop3): /mnt//usr/local/libexec/dovecot/pop3-login login_process_per_connection: no login_max_connections: 128 max_mail_processes: 2500 mail_uid: campus mail_gid: lmu mail_location: maildir:~/Maildir:INDEX=/home/something/indexes/%-1.1n/%n mmap_disable: yes mail_nfs_index: yes mail_executable(default): /mnt//usr/local/libexec/dovecot/imap mail_executable(imap): /mnt//usr/local/libexec/dovecot/imap mail_executable(pop3): /mnt//usr/local/libexec/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /mnt//usr/local/lib/dovecot/imap mail_plugin_dir(imap): /mnt//usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /mnt//usr/local/lib/dovecot/pop3 pop3_uidl_format(default): %08Xu%08Xv pop3_uidl_format(imap): %08Xu%08Xv pop3_uidl_format(pop3): %08Xv%08Xu namespace: type: private separator: . prefix: INBOX. inbox: yes list: yes subscriptions: yes auth default: worker_max_request_count: 500 passdb: driver: ldap args: /mnt//usr/local/etc/dovecot-ldap.conf userdb: driver: prefetch userdb: driver: ldap args: /mnt//usr/local/etc/dovecot-ldap.conf socket: type: listen master: path: /var/run/dovecot/auth-master mode: 384 user: campus group: lmu plugin: quota: maildir quota_rule: Trash:ignore quota_rule2: *:storage=512M Anyone having an idea? bernhard

1 0

Re: [Dovecot] Dovecot-antispam does not work
by Jehan Pagès 14 Feb '09

14 Feb '09

Hi all, just to say that finally the patch of Marcin worked great. Just my configuration is different from yours to get it work: antispam_signature = X-DSPAM-Signature antispam_trash = trash;Trash;Deleted Items antispam_spam = SPAM;Junk antispam_dspam_binary = /usr/bin/dspam antispam_dspam_args = --client;--source=error;--user;%u antispam_mail_tmpdir = /tmp The difference here is in the syntax. You write "--user=somemail" though in the manual, it is written "--user somemail". But also you were proposing 2 lines "antispam_dspam_args" though in all my tests, when I was doing such a thing, the second line seemed to override the first (in the launch logs, only the second line was displayed, so I guess...). Are you sure that it works exactly the way you intend to, Marcin? And that the "--source=error" (your first line) is indeed added as a parameter to the call of dspam? For the patch to the dovecot-antispam, probably it should not be integrated "as is", but I guess that the return code of call_dspam function must probably be checked and fixed if the call can succeed but still give a failure return code. I set Johannes as a main recipient, maybe you will see this email. Bye and thanks all. Jehan On Wed, Feb 4, 2009 at 12:45 PM, Jehan Pagès <jehan.marmottard(a)gmail.com>wrote: > Hi, > > On Wed, Feb 4, 2009 at 12:40 PM, Marcin Rzepecki <m.rzepecki(a)iem.pw.edu.pl > > wrote: > >> Wed, Feb 04, 2009 at 12:12:44PM +0100, Jehan Pag?s wrote: >> > I saw this piece of code and did the same as you in my change of course. >> > Unfortunately, it didn't work (no logged "marked as SPAM", nor change in >> my >> > stats. I have dovecot 1.1.7-r1 though, so it is probably the reason. >> >> Logging '...markes as SPAM..' lines to syslog is my idea, I can send some >> dspam patches, to make it work. It's usefull for me. > > > Ok. Anyway the stats don't change... > > >> >> Are you using mbox or maildir? If you are using mbox, try to upgrade to >> Dovecot >=1.1.8 - it contains several mbox bug fixes (e.g 'Next message >> unexpectedly lost') which are needed to make antispam plugin work. > > > No maildir... > I am doing some stuffs about another way to interact dspam/dovecot/postfix > and will go back to this issue soon. > > Jehan >

2 1

[Dovecot] "failed to map segment from shared object: Cannot allocate memory"
by Vegard Svanberg 13 Feb '09

13 Feb '09

>From /var/log/auth.log (Dovecot 1.1.4 on Ubuntu 8.10): Feb 10 08:29:06 home dovecot-auth: PAM unable to dlopen(/lib/security/pam_ck_connector.so): libdbus-1.so.3: failed to map segment from shared object: Cannot allocate memory Feb 10 08:29:06 home dovecot-auth: PAM adding faulty module: /lib/security/pam_ck_connector.so English, please? :) Strangely, it seemed to disappear after increasing login_processes_count, login_max_processes_count and/or auth_worker_max_count. -- Vegard Svanberg <vegard(a)svanberg.no> [*Takapa@IRC (EFnet)]

2 3

[Dovecot] Production server migration from mbox to maildir ( need advice )
by Frank Bonnet 13 Feb '09

13 Feb '09

Hello I'm going to change the raid array of my mailhub in fews days/weeks and I wonder if it would be a good idea to take the opportunity of this migration to also change the storage method from mailbox to maildir. I've never used maildir on my production server only on small satellites. So I really need advice of admins that use maildir format on prod servers. My server has approx 3500 users/mailboxes that is approx 450 Gb of data. The OS is Debian etch amd64 but could change also during the migration Opensuse X86_64, CAOS-NSA, FreeBSD 64 are some options. Any infos/experiences welcome, I'll summarize answers. Thanks a lot.

4 8

[Dovecot] Corrupted index cache file
by Proskurin Kirill 13 Feb '09

13 Feb '09

Hello all. I run on a problems? dovecot: Feb 13 08:36:13 Error: IMAP(logistic(a)fxclub.org): Maildir filename has wrong W value: /var/spool/dovecot/domains/fxclub.org/logistic/.fxclub/1135926892.Re1a4ad89Q1436.newmail.fxclub.org,S=24630,W=2 4849:2, dovecot: Feb 13 08:36:13 Error: IMAP(logistic(a)fxclub.org): Corrupted index cache file /var/spool/dovecot/domains/fxclub.org/logistic/.fxclub/dovecot.index.cache: Broken virtual size for mail UID 332 dovecot: Feb 13 08:36:13 Info: IMAP(logistic(a)fxclub.org): Disconnected: Disconnected bytes=435/135596 dovecot: Feb 13 08:36:14 Info: imap-login: Login: user=<logistic(a)fxclub.org>, method=PLAIN, rip=194.87.53.162, lip=78.129.148.74, TLS dovecot: Feb 13 08:36:15 Info: IMAP(chief(a)omsk.fxclub.org): Disconnected: Logged out bytes=349/5517 dovecot: Feb 13 08:36:15 Info: IMAP(chief(a)omsk.fxclub.org): Disconnected: Logged out bytes=63/280 dovecot: Feb 13 08:36:15 Error: IMAP(logistic(a)fxclub.org): FETCH for mailbox fxclub UID 332 got too little data: 318 vs 319 dovecot: Feb 13 08:36:15 Error: IMAP(logistic(a)fxclub.org): Maildir filename has wrong W value: /var/spool/dovecot/domains/fxclub.org/logistic/.fxclub/1135926892.Re1a4ad89Q1436.newmail.fxclub.org,S=24630,W=2 4849:2, dovecot: Feb 13 08:36:15 Error: IMAP(logistic(a)fxclub.org): Corrupted index cache file /var/spool/dovecot/domains/fxclub.org/logistic/.fxclub/dovecot.index.cache: Broken virtual size for mail UID 332 dovecot: Feb 13 08:36:15 Info: IMAP(logistic(a)fxclub.org): Disconnected: Disconnected bytes=395/135319 It got then reconfigure MS Outlook 2007 SP1 to make a full download of email - not only headers. Then I reconfig it back to headers only - all work fine. How can I fix it? It is safe to delete index files? Self healing seems to not work here. #dovecot -n mail# dovecot -n # 1.1.8: /usr/local/etc/dovecot.conf # OS: FreeBSD 7.0-RELEASE-p9 i386 ufs base_dir: /var/run/dovecot/ log_path: /var/log/dovecot/dovecot.log info_log_path: /var/log/dovecot/dovecot.log protocols: imap imaps pop3 pop3s managesieve listen(default): * listen(imap): * listen(pop3): * listen(managesieve): *:2000 ssl_ca_file: /usr/local/etc/certs/GlobalSignCA.crt ssl_cert_file: /usr/local/etc/certs/fxclub_org.crt ssl_key_file: /usr/local/etc/certs/fxclub_org.key ssl_cipher_list: ALL:!LOW:!SSLv2 login_dir: /var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login login_executable(managesieve): /usr/local/libexec/dovecot/managesieve-login login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(pop3): no login_greeting_capability(managesieve): no login_process_size: 128 login_processes_count: 20 login_max_processes_count: 512 mail_max_userip_connections(default): 256 mail_max_userip_connections(imap): 256 mail_max_userip_connections(pop3): 20 mail_max_userip_connections(managesieve): 10 verbose_proctitle: yes first_valid_gid: 0 mail_privileged_group: mail mail_uid: dmail mail_gid: dmail mail_location(default): maildir:/var/spool/dovecot/domains/%d/%n/ mail_location(imap): maildir:/var/spool/dovecot/domains/%d/%n/ mail_location(pop3): maildir:/var/spool/dovecot/domains/%d/%n/ mail_location(managesieve): mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_executable(managesieve): /usr/local/libexec/dovecot/managesieve mail_process_size: 512 mail_plugins(default): quota imap_quota autocreate mail_plugins(imap): quota imap_quota autocreate mail_plugins(pop3): quota mail_plugins(managesieve): mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 mail_plugin_dir(managesieve): /usr/local/lib/dovecot/managesieve imap_client_workarounds(default): delay-newmail tb-extra-mailbox-sep imap_client_workarounds(imap): delay-newmail tb-extra-mailbox-sep imap_client_workarounds(pop3): imap_client_workarounds(managesieve): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh pop3_client_workarounds(managesieve): sieve_storage(default): sieve_storage(imap): sieve_storage(pop3): sieve_storage(managesieve): % sieve(default): sieve(imap): sieve(pop3): sieve(managesieve): ~/.dovecot.sieve auth default: mechanisms: plain login user: dovecot-auth username_format: %Lu verbose: yes worker_max_count: 100 passdb: driver: ldap args: /usr/local/etc/dovecot-ldap.conf userdb: driver: ldap args: /usr/local/etc/dovecot-ldap.conf socket: type: listen client: path: /var/run/dovecot/auth-client mode: 432 user: exim master: path: /var/run/dovecot/auth-master mode: 384 user: dmail plugin: quota: maildir quota_rule: *:storage=1024M quota_rule2: Trash:storage=1256M quota_rule3: SPAM:ignore quota_warning: storage=80%% /usr/local/bin/quota-warning.sh 80 quota_warning2: storage=95%% /usr/local/bin/quota-warning.sh 95 autocreate: Trash autocreate2: Sent autocreate3: Spam autosubscribe: Trash autosubscribe2: Sent autosubscribe3: Spam -- Best regards, Proskurin Kirill

3 3

[Dovecot] Bynari Insight connector
by Maarten Bezemer 13 Feb '09

13 Feb '09

Hi, Since IMAP support in Outlook is 'somewhat broken', and Bynari has some promising specifications on their website, I might want to try their product. Does anyone here have any experience with their software? Would you recommend it? Or maybe point me at alternatives? (If there are any.. aside from ditching Outlook of course :-P ) Regards, Maarten Bezemer

3 3

[Dovecot] Google Summer of Code
by Charles Marcus 13 Feb '09

13 Feb '09

On another list, someone wrote: > Google Summer of Code is on it's Way. Start collecting students to > throw at projects now! This sounds like a great idea for dovecot too... Timo, maybe there are a few dovecot enhancements that would be good candidates for this? Maybe the test suite? Replication? LMTP server? Anyway, its just a thought... -- Best regards, Charles

1 0