- dovecot - dovecot.org

[Dovecot] Dovecot performance on GFS clustered filesystem
by Allen Belletti 06 Feb '09

06 Feb '09

Hello All, We are using Dovecot 1.1.3 to serve IMAP on a pair of clustered Postfix servers which share a fiber array via the GFS clustered filesystem. This all works very well for the most part, with the exception that certain operations are so inefficient on GFS that they generate significant I/O load and hurt performance. We are using the Maildir format on disk. We're also using Dovecot's deliver from Postfix to handle local delivery. As best I can determine, the worst problems occur when certain users with very large Inboxes (~10k messages) receive new mail and their client looks up information about that message. GFS doesn't seem to efficiently handle the large directories that contain folders like this. As a result, lots of I/O ops are generated and performance suffers for everyone. I am beginning to wonder if it might be more efficient to revert to the old mbox format, with one file per folder (plus whatever indices are creates.) It seems that this ought to work better with GFS which is geared toward smaller numbers of larger files. Is anyone on the list currently doing that? Alternately, any thoughts regarding tuning or other options would be appreciated. Thanks, Allen -- Allen Belletti allen(a)isye.gatech.edu 404-894-6221 Phone Industrial and Systems Engineering 404-385-2988 Fax Georgia Institute of Technology

3 4

[Dovecot] The 'dbox' Format
by Alexander Beisig 06 Feb '09

06 Feb '09

I'd like to give the dbox format a try, but I could not find any information on how to enable it. There's information on how to set mail_location for the Maildir and mbox formats in the wiki and also in the commented configuration file, but how to set it for dbox is strangely missing. I took a guess with 'mail_location = dbox:~/dbox', but that didn't work, so it's probably not the right way. Also, the version of Dovecot that Debian installed was 1.0.15, which is kinda old. Should there any serious reasons why I should manually install a newer version if I'm going to use dbox? __________________________________________________________________ Deutschlands größte Online-Videothek schenkt Ihnen 12.000 Videos!* http://entertainment.web.de/de/entertainment/maxdome/index.html

2 1

[Dovecot] Deferred emails on alias emails
by Jehan Pagès 05 Feb '09

05 Feb '09

Hi, I am currently making a fresh install of email server. I am having some issue with dovecot-antispam (cf. other emails), and noticed that I was having issues on aliases, which apparently where not forwarded to the actual SMTP server after passing through dspam. Still don't figure why but temporarily disable the dspam filter, which anyway is not useful in the current state of this installation. Now the email to an alias are well forwarded to the real finale server. But I still have many emails which were deferred in the last days and I cannot have them decrease. # ls -l /var/spool/postfix/defer/* | wc -l 297 If I run "postqueue -f" to flush this all, I can see the number down, but fastly comes back to 297. Reading the logs, they are passed to dovecot, which was probably the issue before I disabled dspam. It was trying to deliver a message after filtering it, but in the case of aliases, of course, it cannot because the account is not local (so it does not find the account and defers the email forever)! So it should give it back to postfix so that this one contacts the aliased server (gmail.com in the example below). Feb 4 13:33:18 ks34186 postfix/lmtp[31100]: 82769869FA: to=< someemail(a)gmail.com>, orig to=<someemail(a)mydomain.tld>, relay=mail.mydomain.tld[/var/run/dspam/dspam.sock] conn use=28, delay=142296, delays=142277/19/0/0.41, dsn=4.3.0, status=deferred (host mail.mydomain.tld[/var/run/dspam/dspam.sock] said: 421 4.3.0 <someemail(a)gmail.com> Auto-Whitelisted (in reply to end of DATA command)) Feb 4 13:33:18 ks34186 dovecot: auth(default): master in: USER 1 someemail(a)gmail.com service=deliver Feb 4 13:33:18 ks34186 dovecot: auth(default): passwd(someemail(a)gmail.com): lookup Feb 4 13:33:18 ks34186 dovecot: auth(default): passwd(someemail(a)gmail.com): unknown user Feb 4 13:33:18 ks34186 dovecot: auth-worker(default): sql( someemail(a)gmail.com) SELECT maildir as home, 1004 as uid, 1004 as gid FROM mailbox WHERE username = 'someemail(a)gmail.com' AND active = '1' Feb 4 13:33:18 ks34186 dovecot: auth-worker(default): sql( someemail(a)gmail.com) Unknown user Feb 4 13:33:18 ks34186 dovecot: auth(default): master out: NOTFOUND 1 So is there a way please to flush this whole spool so that it now uses the new method (which is stop using dspam until I figure how to use it) and so that I can forward these emails to their recipient? Thanks. Jehan P.S.: this is maybe more a postfix issue than a dovecot issue though... but you probably all know well postfix as well.

3 10

Re: [Dovecot] dovecot Digest, Vol 70, Issue 16
by Anthony Davis 05 Feb '09

05 Feb '09

Please remove me from the mailing list, i have tried a number of times to un-subscribe, but i still keep getting emails... Tony On 4 Feb 2009, at 19:35, dovecot-request(a)dovecot.org wrote: > dovecot(a)dovecot.org

2 1

[Dovecot] dirsize quota in sql pass db?
by Ken Anderson 05 Feb '09

05 Feb '09

Is there a way to return a per user dirsize quota from a passdb sql lookup? If so, what is the syntax? maybe "select quota as userdb_quota"? ..and in mysql table, quota column would be: "quota=dirsize:storage=10240" ? I'm using mbox, with /etc/passwd userdb, and mysql passdb. Thanks, Ken -- Ken Anderson Pacific.Net

3 4

[Dovecot] Error: Maximum number of mail processes exceeded (see max_mail_processes setting)
by Frank Bonnet 05 Feb '09

05 Feb '09

Hello I have this message repeated several times each *seconds* in /var/log/dovecot/dovecot.log the max_mail_processes is set to 8192 and I can see an average of 500 imap processes on the machine , I think there is a problem somewhere ... Debian 64 bits , IBM X3650 biproc , 7 Gb RAM , RAID5 disks , 2 ethernet Gb ports bonded. Dovecot 1.1.11 has been compiled from scratch on the machine Thanks for any info.

4 5

[Dovecot] 1.1.6: PAM passdb/userdb (mis)configuration
by Oved Ben-Aroya 05 Feb '09

05 Feb '09

I'm sure I'm missing something obvious :-( Dovecot version 1.1.6, pam authentication via ldap (openldap). basicly, we use: passdb pam userdb passwd which work fine, except for Outlook/OL Express users that are asked for their password whenever they "send/receive"... We've had also "passdb shadow" that somehow "fixed" this but allowed also users with expired passwords to login :-( re-added for now, untill the correct configuration is achived). Here is the output of dovecot -n: # 1.1.6: /usr/local/etc/dovecot.conf # OS: Linux 2.6.9-55.ELsmp x86_64 Red Hat Enterprise Linux AS release 4 (Nahant Update 7) info_log_path: /var/log/dovecot protocols: imap imaps pop3 pop3s listen(default): * listen(imap): * listen(pop3): *:110 ssl_listen(default): ssl_listen(imap): ssl_listen(pop3): *:995 ssl_ca_file: /usr/local/etc/dovecot/certs/IPS-IPSCABUNDLE.CRT ssl_cert_file: /usr/local/etc/dovecot/certs/dovecot.pem ssl_key_file: /usr/local/etc/dovecot/private/dovecot.pem disable_plaintext_auth: no verbose_ssl: yes login_dir: /usr/local/var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login first_valid_uid: 50 mail_debug: yes mail_full_filesystem_access: yes mmap_disable: yes lock_method: dotlock mbox_read_locks: dotlock mbox_write_locks: dotlock mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 imap_client_workarounds(default): outlook-idle delay-newmail imap_client_workarounds(imap): outlook-idle delay-newmail imap_client_workarounds(pop3): pop3_uidl_format(default): %08Xu%08Xv pop3_uidl_format(imap): %08Xu%08Xv pop3_uidl_format(pop3): %08Xv%08Xu pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh namespace: type: private separator: / prefix: mail/ location: mbox:%h/mail list: yes subscriptions: yes namespace: type: private separator: / location: maildir:%h/Maildir:INDEX=/var/dovecot/index/%u:CONTROL=/var/dovecot/control/%u inbox: yes list: yes subscriptions: yes auth default: verbose: yes debug: yes worker_max_request_count: 10 passdb: driver: pam args: dovecot passdb: driver: shadow userdb: driver: passwd args: blocking=yes Thank you for your help. -- \Oved Dr. Oved Ben-Aroya, Head Unix group, Taub Computer Center, Technion Phone: +972 (4) 829 3688 FAX: +972 (4) 823 6212 oved(a)technion.ac.il PGP key at http://tx.technion.ac.il/~oved/pgp/pubkey PGP Key fingerprint: A9 52 46 04 E8 70 41 99 60 E3 DA 8F BA 39 C2 C8

2 4

[Dovecot] Script: Moving Maildirs between servers
by Peter Lindgren 05 Feb '09

05 Feb '09

Hi, I just migrated from dovecot 1.0.rc29 to 1.1.7, from Sparc platform to Intel, it went very well. In the process I merged folders and excluded folders from being copied. User rights on folders were kept/set. I thought I'd share my migration script with you folks. If there's an interest to put it on the wiki I can do that, but I'm not sure in what page. Use it as a starter if you need to do this yourselves. Prerequisites: * User names are local unix accounts * Old server's /home must be mounted with nfs * Structure and user names must match between the two systems * Source dovecot must be at least version 1.0.x * Destination dovecot must be 1.1.x * Only dovecot data folders and files are copied, neither dovecot indices nor Sieve scripts etc. Here it goes, have fun! /Peter #!/bin/sh # # Migrates dovecot Maildir folders and settings from older dovecot on # old server to new dovecot on new server. # Prerequisites: # * User names are local unix accounts # * Old server's /home must be mounted with nfs # * Structure and user names must match between the two systems # * Source dovecot must be at least version 1.0.x # * Destination dovecot must be 1.1.x # * Only dovecot data folders and files are copied, neither dovecot indices nor Sieve scripts etc. # # Originally created by Peter Lindgren <peter(a)norrskenkonsult.com> # Use this script at your own risk. No warranties whatsoever, of any kind! # Basic settings: The first one you must definitely change, it's where you have mounted the old server's home: SOURCEBASE=/mnt/casiopea-home DESTBASE=/home # Just an abbreviation: MD=Maildir # Change field separator, for find's sake as there might be folders with spaces: # (doesn't play well together with the for loop otherwise) OLDIFS=${IFS} IFS=' ' # User loop: # Depending on your source system's setup, # you probably have non-interactive users with home directories (one of them # is probably named _dovecot) that shall not be copied. Edit the username # exclusion list according to your needs. # for username in `ls $SOURCEBASE`; do { # This is your username exclude list: case $username in (maildir|_dovecot) continue;; esac # Here starts the real work: SOURCEDIR=$SOURCEBASE/$username/$MD DESTDIR=$DESTBASE/$username/$MD echo "================================" echo "Copying user "$username"..." echo "From "$SOURCEDIR" to "$DESTDIR":" # IMAP Folder loop: # Will create the Maildir folder too, the first echo $SOURCEDIR"/." does that. This way all the copying is in one place only. for sourceimap in `{ echo $SOURCEDIR"/." ; find $SOURCEDIR -type d -name \.\* -maxdepth 1 -print ; } | sort -f`; do { # Folder exlusion and special actions: # .Draft is replaced by .Drafts, so content of .Draft must be copied to .Drafts. # (Probably there were a client in use for a while that used .Draft instead of .Drafts.) # Junk mail folders with various names from clients using different languages shall not be copied. # .Trash shall not be copied either. # Folder .INBOX is a leftover from earlier versions of dovecot, and shall not be copied either. # Add your own exceptions to the list. Be sure to quote spaces and dots. basename=`basename $sourceimap` action=copy case $basename in (\.Junk|\.Junk\ E-mail|\.Correo\ electr\&APM-nico\ no\ deseado|\.Skr\&AOQ-ppost|\.Trash|\.INBOX) action=skip;; esac echo -n "$basename" "$action" if [ $action"X" == copy"X" ]; then { echo -n ":" destname=$basename # Merge contents of .Draft and .Drafts to .Drafts: if [ $basename"X" == .Draft"X" ]; then destname=${destname}"s"; fi destimap=${DESTDIR}/${destname} # If this is the Maildir folder (.), a clean desination name is needed for mkdir: if [ $basename"X" == ."X" ]; then destimap=${DESTDIR}; fi mkdir -m 700 "${destimap}" echo -n "." chown ${username} "${destimap}" echo -n "." chmod go-rx "${destimap}" echo -n "." cp -p "${sourceimap}/dovecot-uidlist" "${destimap}" echo -n "." cp -p "${sourceimap}/dovecot-keywords" "${destimap}" echo -n "." cp -pR "${sourceimap}/cur" "${destimap}" echo -n "." cp -pR "${sourceimap}/new" "${destimap}" echo -n "." cp -pR "${sourceimap}/tmp" "${destimap}" echo " done." } else { echo "" } fi }; done # Copy root folder files last in user loop, since the Maildir is created in the imap loop above: cp -p $SOURCEDIR/subscriptions $DESTDIR echo "User "${username}" done." }; done IFS=${OLDIFS} -- Peter Lindgren http://www.norrskenkonsult.com

7 11

[Dovecot] pop3_lock_session question
by Mark Costlow 04 Feb '09

04 Feb '09

Hello, I'm preparing to convert from qpopper + UW-IMAP to dovecot. So far testing has gone very well. One problem we haven't figured out is that long-running POP sessions keep the mailbox locked, so that the MDA times out while trying to deliver. We're using "maildrop" as our MDA if that matters. We don't see this issue for dovecot IMAP sessions. During long-running IMAP sessions, incoming messages are deilvered tot he user's INBOX without delay. But with POP, the MDA times out and the messages go back to sendmail's queue for later delivery. We though this setting would address the issue: # Keep the mailbox locked for the entire POP3 session. pop3_lock_session = no But it doesn't seem to. Any ideas? Here is our "dovecot -n": ---------------------------------------------------------------------- # 1.1.8: /usr/local/etc/dovecot.conf # OS: FreeBSD 7.1-RELEASE-p2 i386 base_dir: /var/run/dovecot/ protocols: imap imaps pop3 pop3s ssl_cert_file: /etc/mail/certs/mail.swcp.com.pem ssl_key_file: /etc/mail/certs/mail.swcp.com.pem ssl_cipher_list: ALL:!LOW:!SSLv2 disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(pop3): no verbose_proctitle: yes first_valid_uid: 1000 first_valid_gid: 100 mail_privileged_group: mail mail_location: mbox:~/:INBOX=/var/mail/%u:INDEX=/var/dovecot/%u mail_full_filesystem_access: yes mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes lock_method: dotlock mbox_lock_timeout: 120 dbox_rotate_days: 0 mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 imap_client_workarounds(default): delay-newmail netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(imap): delay-newmail netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(pop3): pop3_reuse_xuidl(default): no pop3_reuse_xuidl(imap): no pop3_reuse_xuidl(pop3): yes pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh auth default: mechanisms: plain login username_format: %Lu winbind_helper_path: /usr/local/bin/ntlm_auth verbose: yes debug: yes debug_passwords: yes passdb: driver: pam args: session=yes dovecot userdb: driver: passwd args: blocking=yes socket: type: listen client: path: /var/run/dovecot/auth-client mode: 432 master: path: /var/run/dovecot/auth-master mode: 384 ---------------------------------------------------------------------- Thank, Mark -- Mark Costlow | Southwest Cyberport | Fax: +1-505-232-7975 cheeks(a)swcp.com | Web: www.swcp.com | Voice: +1-505-232-7992 abq-strange.com -- Interesting photos taken in Albuquerque, NM Last post: Cruising San Mateo I - 2009-01-04 15:20:30

4 6

[Dovecot] Starting using LaunchDaemons
by Bruce Bodger 04 Feb '09

04 Feb '09

I've figured this one out... Just wanted to get it into the dovecot mailing list archive so that somebody in the future might find it if they need it. Running Dovecot on Mac OS X 10.4.11 and using LaunchDaemon to start. Following a system crash, dovecot failed to restart with this in the system log... Feb 4 10:04:29 G520X2 launchd: org.dovecot: exited with exit code: 89 Feb 4 10:04:29 G520X2 launchd: org.dovecot: respawning too quickly! throttling Feb 4 10:04:29 G520X2 launchd: org.dovecot: 7 more failures without living at least 60 seconds will cause job removal Feb 4 10:04:29 G520X2 launchd: org.dovecot: will restart in 10 seconds Feb 4 10:04:39 G520X2 launchd: org.dovecot: exited with exit code: 89 Feb 4 10:04:39 G520X2 launchd: org.dovecot: respawning too quickly! throttling Feb 4 10:04:39 G520X2 launchd: org.dovecot: 6 more failures without living at least 60 seconds will cause job removal Feb 4 10:04:39 G520X2 launchd: org.dovecot: will restart in 10 seconds Feb 4 10:04:49 G520X2 launchd: org.dovecot: exited with exit code: 89 Feb 4 10:04:49 G520X2 launchd: org.dovecot: respawning too quickly! throttling Feb 4 10:04:49 G520X2 launchd: org.dovecot: 5 more failures without living at least 60 seconds will cause job removal Feb 4 10:04:49 G520X2 launchd: org.dovecot: will restart in 10 seconds The problem turned out to be an old master.pid file (/usr/local/var/ run/dovecot/master.pid) that was left over from the crash. Deleting the file allowed dovecot to restart. Actually, dovecot told me what the problem was when I attempted to start it from the command line as opposed to starting it with the LaunchDaemon... G520X2:root# /usr/local/sbin/dovecot -F Fatal: Dovecot is already running with PID 59 (read from /usr/local/ var/run/dovecot/master.pid) Hope it helps someone else. B. Bodger

2 1