September 2008 - dovecot

[Dovecot] Moving from cyrus to dovecot (a story)
by Steinar Bang 16 Sep '08

16 Sep '08

The transition was quick, and also (so far) seemingly painless. The server is an old Pentium II 233 machine that sits between my home LAN and the internet, and have sitting there, and running night and day since 1999. For the first two years it was running SuSE linux, but since 2001 it has been running debian and been continously upgraded. It was originally installed with potato, but have been apt-get dist-upgrade'd to woody, and then to sarge, and then etch, which is what it's currently running I've been using cyrus imapd since 1999 (compiled by hand on SuSE and using the debian packaged version on debian). But in all these years I've never become really friendly with it. In the years leading up to 1999 I had been running UoW imapd at work, and I wasn't too impressed with it at that point in time. The alternative choices were courier and cyrus, and I picked cyrus more or less arbitrarily. By "never become really friendly", I mean - I've never really gotten a good understanding of the configuration - I've never quite understood what the documentation have said about authentication and SSL support. I've gotten things to work by trial and error - Four times the seen data bases have become corrupt. Two times during upgrades (one of them after I had switched to skip lists), and two times on their own, while I was still running BDB - Since delivery was to a central spool and database, owned by user cyrus, has meant that I had to use sieve (which is another thing I've never quite understood) for mail filtering, instead of dear old procmail Cyrus has lots of stuff related to scaling and having mailboxes for users that have no users on the host, but that's not really relevant to my home server. Around six months ago I stumbled over dovecot, which from my POV have the following crucial features: 1. folders under the users' home directories, either as mbox or as maildir, which allows me to do list filtering with procmail, and no central databases that can become corrupt 2. understandable configuration (the only change I'd had to do in the debian dovecot, is to allow for cram-md5 authentication) I decided om maildir as the folder format. I like the simplicity of mbox folders, but I have some archive folders with around 20k of articles each, and maildir would scale a bit better for them. Also mail delivery without locking is a very nice feature. Here's how I proceeded: - I set up dovecot on my daughter's Ubuntu Hardy machine - I ran imapsync on this machine to extract everything I had on the cyrus server to Maildir folders in the users' home directories on the Ubuntu box - I shut down fetchmail on my gateway box (since ISP port filtering and blacklisting of dynamic addresses stops direct SMTP delivery I have to use ISP gateways and mail drops) - I changed the exim conf on my gateway from using cyrdeliver to delivering to a maildir folder in the users' home directory - I restored my old .procmailrc file from subversion (my home directory is checked into subversion, this was originally a CVS repo that went subversion a couple of years ago), merged in the last version of the .procmailrc that used cyrdeliver (I gave up on that because the .procmailrc file had to be owned by user cyrus, which was very impractical), and changed the filtering rules to deliver to maildir folders - I tested some delivery - I apt-get install'd dovecot from the etch back ports pointed to from the dovecot install page http://wiki.dovecot.org/PrebuiltBinaries#head-477d52d96fcf34ad3033b13e5e5a2… This uninstalled cyrus22-imapd, but not the rest of cyrus 2.2, which blocked dovecot from starting after the install - I stopped cyrus22 and uninstalled all of the cyrus22 packages - I rsync'd the maildir folders from the Ubuntu box to the old mail server - I started dovecot - I use Gnus as my MUA. I had removed all nnimap groups related to the home server in Gnus, and resubscribed them. All folders came up in the state I had left them. Gnus stores article state on the server as custom marks, and imapsync had preserved the custom flags. I unsubscribed the groups to avoid article caching confusion, because imapsync changes the UIDs. However, all folders except INBOX had changed their names, so that didn't matter much - I started fetchmail and sent som test emails, and observed that they ended up in the right Maildir folders, and that dovecot picked them up as expected And today I accessed the server from work. So it looks like I'm back in business. This time with a server I understand a lot better. Thanx for writing it, Timo! - Steinar

2 1

[Dovecot] Quota, maildir, MySQL,
by Bart Dorlandt 16 Sep '08

16 Sep '08

Hi everybody, I am trying to work with quota. I use postfix and dovecot with a sql backend. Below you'll find my configuration. Everything works except for the quota part. It doesn't seem to see the quota and doesn't respond to the sender that the client is over its quota. versions: postfix-2.5.20070531-mysql dovecot-1.0.2-mysql Thanks for the help in advance. My Settings: # 1.0.2: /etc/dovecot.conf base_dir: /var/dovecot/ log_path: /var/log/dovecot.log protocols: imap imaps pop3s listen(default): 127.0.0.1:143 listen(imap): 127.0.0.1:143 listen(pop3): * ssl_listen: * ssl_cert_file: /etc/ssl/dovecotcert.pem login_dir: /var/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login login_user: _dovecot first_valid_uid: 2000 last_valid_uid: 2000 first_valid_gid: 2000 last_valid_gid: 2000 mail_location: maildir:/var/mail/vhosts/%d/%n maildir_copy_with_hardlinks: yes mbox_write_locks: fcntl mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 imap_client_workarounds(default): delay-newmail outlook-idle netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(imap): delay-newmail outlook-idle netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(pop3): outlook-idle pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %08Xu%08Xv pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh auth default: mechanisms: plain login digest-md5 debug: yes passdb: driver: sql args: /etc/dovecot-sql.conf userdb: driver: prefetch userdb: driver: sql args: /etc/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: _postfix group: _postfix master: path: /var/run/dovecot/auth-master mode: 432 user: vmail group: vmail plugin: quota: maildir: my /etc/dovecot-sql.conf: user_query = SELECT uid, gid, home, maildir as mail, concat('maildir:storage=',quota) AS quota FROM users WHERE login = '%u' password_query = SELECT password, uid AS userdb_uid, gid AS userdb_gid, home AS userdb_home, maildir as userdb_mail, concat('maildir:storage=', quota) AS userdb_quota FROM users WHERE login = '%u' My postfix config: # postconf -n alias_database = hash:/etc/postfix/aliases alias_maps = hash:/etc/postfix/aliases bounce_template_file = /etc/postfix/bounce.cf command_directory = /usr/local/sbin config_directory = /etc/postfix content_filter = smtp-amavis:[127.0.0.1]:10024 daemon_directory = /usr/local/libexec/postfix disable_vrfy_command = yes html_directory = /usr/local/share/doc/postfix/html inet_interfaces = all inet_protocols = all invalid_hostname_reject_code = 554 mail_owner = _postfix mailq_path = /usr/local/sbin/mailq manpage_directory = /usr/local/man maximal_queue_lifetime = 40d message_size_limit = 20480000 multi_recipient_bounce_reject_code = 554 mydestination = localhost.$mydomain, localhost mydomain = bamweb.nl myhostname = bama.bamweb.nl mynetworks = 127.0.0.0/8, 82.173.141.217/32 myorigin = $myhostname newaliases_path = /usr/local/sbin/newaliases non_fqdn_reject_code = 554 permit_mx_backup_networks = zoph.getmyip.com queue_directory = /var/spool/postfix readme_directory = /usr/local/share/doc/postfix/readme recipient_delimiter = + relay_domains = $mydestination, hash:/etc/postfix/relay_domains relay_domains_reject_code = 554 relay_recipient_maps = sample_directory = /etc/postfix sendmail_path = /usr/local/sbin/sendmail setgid_group = _postdrop smtpd_banner = $myhostname ESMTP $mail_name smtpd_helo_required = yes smtpd_recipient_restrictions = permit_mynetworks, permit_mx_backup, reject_invalid_hostname, check_relay_domains, reject_non_fqdn_hostname, reject_non_fqdn_sender, reject_non_fqdn_recipient, reject_unknown_sender_domain, reject_unknown_recipient_domain, reject_unauth_destination, reject_rbl_client relays.ordb.org, reject_rbl_client opm.blitzed.org, reject_rbl_client list.dsbl.org, reject_rbl_client sbl.spamhaus.org, reject_rbl_client cbl.abuseat.org, reject_rbl_client dul.dnsbl.sorbs.net, reject_rbl_client dev.null.dk, reject_rbl_client virbl.dnsbl.bit.nl, reject_rbl_client smtp.dnsbl.sorbs.net, reject_rbl_client socks.dnsbl.sorbs.net, reject_rbl_client http.dnsbl.sorbs.net, reject_rbl_client web.dnsbl.sorbs.net, check_policy_service inet:127.0.0.1:60000 permit strict_rfc821_envelopes = yes unknown_address_reject_code = 554 unknown_client_reject_code = 554 unknown_hostname_reject_code = 554 unknown_local_recipient_reject_code = 554 unknown_relay_recipient_reject_code = 554 unknown_virtual_alias_reject_code = 554 unknown_virtual_mailbox_reject_code = 554 unverified_recipient_reject_code = 554 unverified_sender_reject_code = 554 virtual_alias_maps = mysql:/etc/postfix/mysql_virtual_alias_maps.cf mysql:/etc/postfix/mysql_virtual_email2email.cf virtual_gid_maps = static:2000 virtual_mailbox_base = /var/mail/vhosts virtual_mailbox_domains = mysql:/etc/postfix/mysql_virtual_domains.cf virtual_mailbox_maps = mysql:/etc/postfix/mysql_virtual_mailboxes.cf virtual_minimum_uid = 2000 virtual_uid_maps = static:2000 -- www.bamweb.nl

2 1

[Dovecot] How to recover sieve files?
by David Reid 16 Sep '08

16 Sep '08

I'm seeing a lot of these messages in my log files... deliver(...): Aug 10 15:18:34 Error: sieve_execute_bytecode(.../.dovecot.sievec) failed I'm guessing the file needs to be updated/recreated, but what's the "correct" way to do it? should I just remove the existing one and let the server recreate it? Thanks, david

2 1

[Dovecot] Authentication fails
by CJ Keist 15 Sep '08

15 Sep '08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Been running dovecot 1.1.2 for about a month now. I Have strange issue that I have not been able to trouble shoot. What I'm seeing is that dovecot runs along just fine for about 4 days, and then it will just stop authenticating everyone (IMAP and POP). No errors in the logs other than all authentication attempts comes out as password mismatch. The only fix is to stop and restart dovecot. % ./dovecot -n # 1.1.2: /userM/mail-services/dovecot/etc/dovecot.conf Warning: fd limit 256 is lower than what Dovecot can use under full load (more than 3072). Either grow the limit or change login_max_processes_count and max_mail_processes settings base_dir: /userM/mail-services/dovecot/var/run/dovecot/ protocols: imap imaps pop3 pop3s ssl_cert_file: /userM/mail-services/dovecot/etc/ssl/dovecot.pem ssl_key_file: /userM/mail-services/dovecot/etc/ssl/privkey.pem ssl_parameters_regenerate: 0 ssl_cipher_list: MEDIUM:!LOW disable_plaintext_auth: no login_dir: /userM/mail-services/dovecot/var/run/dovecot/login login_executable(default): /userM/mail-services/dovecot/libexec/dovecot/imap-login login_executable(imap): /userM/mail-services/dovecot/libexec/dovecot/imap-login login_executable(pop3): /userM/mail-services/dovecot/libexec/dovecot/pop3-login login_greeting: Cyrus ready. login_max_processes_count: 1024 max_mail_processes: 1024 mail_location: mbox:~/:INBOX=/userM/mail/%u mail_debug: yes mmap_disable: yes mail_nfs_index: yes mbox_lock_timeout: 60 mbox_dotlock_change_timeout: 60 mail_executable(default): /userM/mail-services/dovecot/libexec/dovecot/imap mail_executable(imap): /userM/mail-services/dovecot/libexec/dovecot/imap mail_executable(pop3): /userM/mail-services/dovecot/libexec/dovecot/pop3 mail_plugin_dir(default): /userM/mail-services/dovecot/lib/dovecot/imap mail_plugin_dir(imap): /userM/mail-services/dovecot/lib/dovecot/imap mail_plugin_dir(pop3): /userM/mail-services/dovecot/lib/dovecot/pop3 auth default: mechanisms: plain login verbose: yes debug: yes passdb: driver: pam passdb: driver: passwd args: blocking=yes userdb: driver: passwd args: blocking=yes socket: type: listen client: path: /var/lib/postfix/private/auth mode: 438 user: postfix group: postfix master: path: /userM/mail-services/dovecot/var/run/dovecot//auth-master mode: 384 This is on Solaris 10, using LDAP for authentication. Any ideas on how to trouble shoot what might be going on? - -- C. J. Keist Email: cj.keist(a)colostate.edu UNIX/Network Manager Phone: 970-491-0630 Engineering Network Services Fax: 970-491-5569 College of Engineering, CSU Ft. Collins, CO 80523-1301 All I want is a chance to prove 'Money can't buy happiness' -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIzrciA29OFr7C6jcRAk/NAJoDKskvI4GpQg2yQPrUyH9AdzmFNACgjPTI rsWVRV8WuuH/RFx+1YOnzEY= =toPK -----END PGP SIGNATURE-----

3 3

[Dovecot] IMAP traffic logging
by fp＠onwebworx.net 15 Sep '08

15 Sep '08

Hi! Is there any known way I can log traffic caused by IMAP sessions? First it is adequate to have a rough overall statistic let's say traffic per hour - this is easily done with iptables. But on the long term I also need information on account basis. I thought of rawlogging every IMAP session and then simply count the filesizes. But the created logs have dropouts because dovecot doesn't log whole FETCH commands what is a problem. Thanks for every idea you have! Paul

2 2

[Dovecot] Dovecot cannot unlink non-empty directory
by Włodek Waligórski 15 Sep '08

15 Sep '08

Hi, Currently I use dovecot 1.1.3 on Debian Testing. CPU arch is x86. Mail is stored on ext3 fs. I'm having a following problem with deleting folders (there is no problem with creation of folders): Error: IMAP(test(a)laptop.aonet.pl): unlink_directory(/home/vpopmail/domains/laptop.aonet.pl/test/.//Maildir/..DOVECOT-TRASHED) failed: Permission denied HOW TO REPRODUCE 1. Create a fresh new account 2. Login to a new account and create folder say 'TestFolder' 3. When I try to delete TestFolder - it is deleted and ..DOVECOT-TRASHED appears in user's Maildir AND Error: IMAP(test(a)laptop.aonet.pl): unlink_directory(/home/vpopmail/domains/laptop.aonet.pl/test/.//Maildir/..DOVECOT-TRASHED) failed: Permission denied message appears - it is triggered from line 815 of dovecot-1.1.3/src/lib-storage/index/maildir/maildir-storage.c 4. Create 'TestFolder2' 5. Trying to delete 'TestFolder2' gives the same error: Error: IMAP(test(a)laptop.aonet.pl): unlink_directory(/home/vpopmail/domains/laptop.aonet.pl/test/.//Maildir/..DOVECOT-TRASHED) failed: Permission denied this one however, is triggered from line 806 of dovecot-1.1.3/src/lib-storage/index/maildir/maildir-storage.c 6. The folder 'TestFolder2' is not deleted and from this point no other newly created folders can be deleted too. --------------------------------------------------------- I. Yes, the file permissions are fine - all are writeable to (mail_uid = vpopmail) II. The problem is - dovecot cannot unlink non-empty directory, I artificially deleted contents of ..DOVECOT-TRASHED and dovecot allowed folder deletion from client. After that situation was like in point 4. -My-config------------------------------------------------------ # 1.1.3: /usr/local/etc/dovecot.conf base_dir: /usr/local/var/run/dovecot/ log_path: /dev/stderr info_log_path: /dev/stderr log_timestamp: listen: 127.0.0.1:143 ssl_listen: 0:993 ssl_cert_file: /var/qmail/control/servercert.pem ssl_key_file: /var/qmail/control/servercert.pem ssl_cipher_list: ALL:!LOW verbose_ssl: yes login_dir: /usr/local/var/run/dovecot/login login_executable: /usr/local/libexec/dovecot/imap-login verbose_proctitle: yes first_valid_uid: 89 last_valid_uid: 89 mail_uid: vpopmail mail_gid: vchkpw mail_location: maildir:~/Maildir mail_debug: yes mail_plugins: quota imap_quota imap_client_workarounds: outlook-idle auth default: user: vpopmail verbose: yes debug: yes passdb: driver: vpopmail args: quota_template=quota_rule=*:backend=%q userdb: driver: vpopmail plugin: quota: maildir quota_rule: ?:storage=110485760 quota_rule2: Trash:ignore quota_rule3: SPAM:ignore quota_warning: storage=90%% /usr/local/bin/imap-quota-warning 90 ---------------------------------------------------------------------- is this my mistake in configuration or is dovecot doing something wrong? tnx for help, -Włodek

2 5

[Dovecot] fail2ban 0.8
by Luigi Rosa 15 Sep '08

15 Sep '08

Does anyone have the filter strings for Fail2Ban 0.8 to block Dovecot 1.1 login failures? Thanks! Ciao, luigi -- / +--[Luigi Rosa]-- \ If the odds are a million to one against something occurring, chances are 50-50 it will.

2 2

[Dovecot] Blackberry suggestions?
by Ed W 14 Sep '08

14 Sep '08

Hi, I'm not a Blackberry user, but we have some customers currently using an Exchange server with some kind of blackberry and interested in moving their service over to our dovecot 1.1 server Does anyone have any notes or recommendations on getting a (UK) Blackberry to talk effectively to a normal Dovecot server, and are there any things to consider in order to get whatever features they are normally using on their Exchange link working on the Dovecot server? (It's just a small couple of person office on an Exchange server) Thanks for any thoughts/tips Ed W

2 1

[Dovecot] Corrected permissions for login directory message
by Nicolas Letellier 14 Sep '08

14 Sep '08

Hello. I already have this warn message when I start dovecot (1.1.3). I saw some message in mailing list about this warning, but nothing which solve my problem. See the message: root@machine dovecot $ /usr/local/etc/rc.d/dovecot restart Stopping dovecot. Waiting for PIDS: 47494. Starting dovecot. Warning: Corrected permissions for login directory /var/run/dovecot/login See the permissions: ls -l /var/rundrwxr-xr-x 3 root wheel 512 Sep 11 15:42 dovecot ls -l /var/run/dovecot/ srw-rw---- 1 dovecot mail 0 Sep 11 15:42 auth-master srw------- 1 root wheel 0 Sep 11 15:42 auth-worker.47529 srwxrwxrwx 1 root wheel 0 Sep 11 15:42 dict-server drwxr-x--- 2 root dovecot-auth 512 Sep 11 15:42 login -rw------- 1 root wheel 6 Sep 11 15:42 master.pid ls -l /var/run/dovecot/login/ srw-rw---- 1 root dovecot-auth 0 Sep 11 15:42 default -rw-r--r-- 2 root wheel 230 Sep 11 15:41 ssl-parameters.dat And the dovecot conf: # 1.1.3: /usr/local/etc/dovecot.conf syslog_facility: local6 ssl_cert_file: /etc/ssl/mail/mail.nicoelro.net.cert ssl_key_file: /etc/ssl/mail/mail.nicoelro.net.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/local/libexec/dovecot/imap-login login_user: dovecot-auth login_greeting: Mail Server login_processes_count: 2 verbose_proctitle: yes first_valid_uid: 143 last_valid_uid: 143 first_valid_gid: 143 last_valid_gid: 143 mail_access_groups: mail mail_location: maildir:/var/mail/vmail/%u/ mail_plugins: quota imap_quota imap_client_workarounds: delay-newmail outlook-idle netscape-eoh tb-extra-mailbox-sep auth default: mechanisms: plain login passdb: driver: sql args: /usr/local/etc/dovecot/dovecot-sql.conf userdb: driver: sql args: /usr/local/etc/dovecot/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix master: path: /var/run/dovecot/auth-master mode: 432 user: dovecot group: mail plugin: quota: maildir sieve: /var/mail/vmail/%u/sieve Thanks for your informations. -- - Nicolas.

2 4

[Dovecot] cram-md5 and users maintaining their own passwords?
by Steinar Bang 13 Sep '08

13 Sep '08

Is there any other mechanism than using passwd files with md5-hashed passwords created by dovecotpw that will support cram-md5 authentication? Has anyone created setups where the passwd databases reside in the individual users home directories? Is it possible to persuade dovecotpw to update the passwd databases automatically. Having to use a text editor to paste in the passwords sets a high user treshold on using the cram-md5 authentication mechanism. Thanx! - Steinar

2 7