December 2008 - dovecot

[Dovecot] expire plugin with pgsql not working
by LÉVAI Dániel 09 Jan '09

09 Jan '09

Hi! I've configured dovecot, and created the db/table/trigger according to http://wiki.dovecot.org/Quota/Dict. I'm using dovecot-1.1.7 and pgsql. When moving a message inside the "monitored" directory, the expires table won't get updated because of an error: statement: INSERT INTO expires (expire_stamp, mailbox) VALUES ('1229993423', 'leva/spamassassin.SPAM') INSERT ERROR: null value in column "username" violates not-null constraint Wonder why doesn't dovecot pass the username too. What could be the problem? Daniel -- LEVAI Daniel PGP key ID = 0x4AC0A4B1 Key fingerprint = D037 03B9 C12D D338 4412 2D83 1373 917A 4AC0 A4B1

2 4

[Dovecot] dovecot-auth Temporary But Frequent CPU Spike
by Joe Clark 09 Jan '09

09 Jan '09

Hi.. I am setting up a new server (2GHz CPU, Ubuntu 8.10) and am using dovecot for an IMAP server. I have installed dovecot 1.1.4 and Squirrelmail 1.4.15. sudo dovecot -n shows this: # 1.1.4: /etc/dovecot/dovecot.conf log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login login_process_per_connection: no mail_privileged_group: mail auth default: passdb: driver: pam userdb: driver: passwd The problem I'm noticing is that if I do top -d 1 while clicking links in Squirrelmail (e.g., logging in, the Inbox link, opening a message), I see that a "dovecot-auth" process maximizes the CPU for about a second. While this is not a huge amount, it seems like with multiple users it could quickly bog down a server . Are there any recommendations or thoughts on what might cause the one-second CPU spike? -- or is that just normal operation? Thanks for your thoughts. - Joe Clark

2 1

[Dovecot] Dovecot-auth timeouts
by Javier Fox 08 Jan '09

08 Jan '09

Hello, I've unfortunately been unable to find anything relating to the problem I'm having specifically, in searching the list or google, and so I now plead to you to assistance. I'm running Dovecot as an LDA and SASL auth for Postfix on a Debian 4 box. Dovecot is version 1.0.rc15 (the official debian pkg version). The problem I'm running into is this. After some time of running (lately it's been as little as 5 minutes), I start to see the following errors in dovecot.log: deliver(user(a)domain.com): "Dec 23 14:38:47 "Error: User request from dovecot-auth timed out deliver(anotheruser(a)domain.com): "Dec 23 14:38:48 "Error: User request from dovecot-auth timed out Postfix responds to these by simply deferring the messages. Dovecot itself, however, begins to return 'Authentication failed' messages after significant lag time (sometimes greater than 30s): Connected to localhost. Escape character is '^]'. +OK Dovecot-POP user username +OK pass mypassword -ERR Authentication failed. Now, for authentication, Dovecot is using LDAP on the local server. The only additional information I can find pertaining to these errors is the following from slapd.log: slapd[22593]: connection_input: conn=6 deferring operation: pending operations These messages correspond 1-to-1 to the above 'deliver' errors, where 'conn' is always the same number. Restarting dovecot and ldap resolves the issue for a few minutes, but sure enough the errors start flowing again. I'm really at the end of my rope on this, as nothing I do seems to help. I have a good 500+ customers being effected by this as well, and they're all none too pleased by it. If this is something that will absolutely be resolved by upgrading from source, that is doable, but we'd prefer to stick with the official package version if possible. Dovecot configs follow Thanks, J. Fox ----- configs follow ----- dovecot.conf ------------ auth_verbose = yes auth_debug = yes auth_debug_passwords = yes mail_debug = no base_dir = /var/run/dovecot/ protocols = imap imaps pop3 pop3s protocol lda { postmaster_address = postmaster(a)spiritone.com auth_socket_path = /var/run/dovecot/auth-master log_path = /var/log/dovecot.log info_log_path = /var/log/mail.info } listen = * shutdown_clients = yes mmap_disable = yes lock_method = dotlock maildir_copy_with_hardlinks = no log_path = /var/log/dovecot.log info_log_path = /var/log/mail.log log_timestamp = "%b %d %H:%M:%S " syslog_facility = mail auth_default_realm = involved.com disable_plaintext_auth = no ssl_cert_file = /etc/ssl/certs/dovecot.pem ssl_key_file = /etc/ssl/private/dovecot.pem login_chroot = yes valid_chroot_dirs = /home/vmail/ login_user = postfix login_process_per_connection = yes login_processes_count = 2 login_max_processes_count = 64 login_max_connections = 128 login_greeting = Involved login_log_format_elements = user=<%u> method=%m rip=%r lip=%l %c login_log_format = %$: %s default_mail_env = maildir:/home/vmail/domains/%d/%u first_valid_uid = 103 pop3_uidl_format = %08Xu%08Xv auth_cache_size = 10485760 auth_cache_ttl = 3600 auth_worker_max_count = 10 #auth_worker_max_request_count = 50 auth default { mechanisms = PLAIN LOGIN passdb ldap { args = /etc/dovecot/dovecot-ldap.conf } userdb ldap { args = /etc/dovecot/dovecot-ldap.conf } socket listen { master { path = /var/run/dovecot/auth-master mode = 0666 user = vmail group = vmail } client { path = /var/spool/postfix/private/auth mode = 0660 user = postfix group = postfix } } user = vmail } dovecot-ldap.conf ----------------- hosts = localhost auth_bind = yes auth_bind_userdn = cn=%n,ou=%d,ou=mail,dc=domain,dc=com ldap_version = 3 base = ou=mail,dc=domain,dc=com dn = cn=Manager,dc=domain,dc=com dnpass = secret deref = never scope = subtree pass_attrs = mail=user,userPassword=password user_filter = (&(objectClass=VirtualMailAccount)(accountActive=TRUE)(mail=%u)) pass_filter = (&(objectClass=VirtualMailAccount)(accountActive=TRUE)(mail=%u)) user_global_uid = 1001 user_global_gid = 1001 ---end---

2 1

[Dovecot] delivers mail to bad directory(prefix) using sieve filters
by Radim Roska 08 Jan '09

08 Jan '09

Hi, I've installed dovecot debian stable(from backports) version (1.0.15). Its great..i just have problem with sieve filters. I use only web tool for creating filters that works for me (avelsieve plugin for squirrelmail). it creates rules, where specified folder has prefix INBOX. Thats fine.. Since dovecot has configured prefix INBOX too. BUT mails are delivered to INBOX.INBOX.test_folder and thats ugly and not usable. I've figured out new version 1.2 should work, but i dont like idea to install not stable version to production server. Is there any patch or something for stable version? Thanks a lot! Radim

4 9

[Dovecot] Bug Report in Connection with Expire Plugin
by Jörg Leis 08 Jan '09

08 Jan '09

If the expire plugin is enabled and a new message (not in trash) is deleted via IMAP in Mozilla Thunderbird (only tested in this client), it will indefinately copy the message into the folder. Dovecot, meanwhile, occupies itself with this: root@server:~# cat /var/log/dovecot.log [...] dovecot: 2008-12-26 09:21% Error: child 6132 (imap) killed with signal 11 dovecot: 2008-12-26 09:21% Error: child 6133 (imap) killed with signal 11 dovecot: 2008-12-26 09:21% Error: child 6134 (imap) killed with signal 11 dovecot: 2008-12-26 09:21% Error: child 6135 (imap) killed with signal 11 dovecot: 2008-12-26 09:21% Error: child 6136 (imap) killed with signal 11 dovecot: 2008-12-26 09:21% Error: child 6138 (imap) killed with signal 11 dovecot: 2008-12-26 09:21% Error: child 6139 (imap) killed with signal 11 dovecot: 2008-12-26 09:21% Error: child 6140 (imap) killed with signal 11 dovecot: 2008-12-26 09:21% Error: child 6142 (imap) killed with signal 11 [...] root@server:~# cat /var/log/dovecot.info [...] dovecot: 2008-12-26 09:21% Info: imap-login: Login: user=<admin(a)bildig.ch>, method=CRAM-MD5, rip=84.75.159.19, lip=92.42.190.87, TLS dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): Loading modules from directory: /usr/lib/dovecot/modules/imap dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): Module loaded: /usr/lib/dovecot/modules/imap/lib10_quota_plugin.so dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): Module loaded: /usr/lib/dovecot/modules/imap/lib11_imap_quota_plugin.so dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): Module loaded: /usr/lib/dovecot/modules/imap/lib20_expire_plugin.so dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): Effective uid=5000, gid=5000, home=/home/vmail/Maildirs/bildig.ch/admin dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): Quota root: name=User Quota backend=maildir args= dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): Quota rule: root=User Quota mailbox=* bytes=524288000 (0%) messages=0 (0%) dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): Quota rule: root=User Quota mailbox=Trash bytes=52428800 (0%) messages=0 (0%) dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): Quota warning: bytes=498073600 (95%) messages=0 (0%) command=/usr/local/bin/mail-quota-warning.sh 95 dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): Quota warning: bytes=419430400 (80%) messages=0 (0%) command=/usr/local/bin/mail-quota-warning.sh 80 dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): maildir: data=/home/vmail/Maildirs/bildig.ch/admin dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): maildir++: root=/home/vmail/Maildirs/bildig.ch/admin, index=, control=, inbox=/home/vmail/Maildirs/bildig.ch/admin dovecot: 2008-12-26 09:21% Info: imap-login: Login: user=<admin(a)bildig.ch>, method=CRAM-MD5, rip=84.75.159.19, lip=92.42.190.87, TLS dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): Loading modules from directory: /usr/lib/dovecot/modules/imap dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): Module loaded: /usr/lib/dovecot/modules/imap/lib10_quota_plugin.so dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): Module loaded: /usr/lib/dovecot/modules/imap/lib11_imap_quota_plugin.so dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): Module loaded: /usr/lib/dovecot/modules/imap/lib20_expire_plugin.so dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): Effective uid=5000, gid=5000, home=/home/vmail/Maildirs/bildig.ch/admin dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): Quota root: name=User Quota backend=maildir args= dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): Quota rule: root=User Quota mailbox=* bytes=524288000 (0%) messages=0 (0%) dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): Quota rule: root=User Quota mailbox=Trash bytes=52428800 (0%) messages=0 (0%) dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): Quota warning: bytes=498073600 (95%) messages=0 (0%) command=/usr/local/bin/mail-quota-warning.sh 95 dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): Quota warning: bytes=419430400 (80%) messages=0 (0%) command=/usr/local/bin/mail-quota-warning.sh 80 dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): maildir: data=/home/vmail/Maildirs/bildig.ch/admin dovecot: 2008-12-26 09:21% Info: IMAP(admin(a)bildig.ch): maildir++: root=/home/vmail/Maildirs/bildig.ch/admin, index=, control=, inbox=/home/vmail/Maildirs/bildig.ch/admin [...] When the expire plugin is disabled no error occurs. I use Dovecot 1.1.4 on Ubuntu 8.10 Intrepid with following settings: root@server:~# dovecot -n # 1.1.4: /etc/dovecot/dovecot.conf base_dir: /var/run/dovecot/ log_path: /var/log/dovecot.log info_log_path: /var/log/dovecot.info log_timestamp: %Y-%m-%d %H:%M% protocols: imaps pop3s ssl_cert_file: /etc/ssl/certs/www_bildig_ch__chained.crt ssl_key_file: /etc/ssl/private/www_bildig_ch.key login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_greeting: Dovecot ready login_process_per_connection: no login_processes_count: 1 login_max_processes_count: 8 login_max_connections: 64 valid_chroot_dirs: /var/spool/vmail mail_debug: yes mail_location: maildir:/home/vmail/Maildirs/%d/%n mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_process_size: 64 mail_plugins(default): expire imap_quota quota mail_plugins(imap): expire imap_quota quota mail_plugins(pop3): expire quota mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 pop3_uidl_format(default): %08Xu%08Xv pop3_uidl_format(imap): %08Xu%08Xv pop3_uidl_format(pop3): %08%08Xv auth default: mechanisms: cram-md5 passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: static args: uid=vmail gid=vmail home=/home/vmail/Maildirs/%d/%n allow_all_users=yes socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail plugin: expire: Trash 30 Trash/* 30 Spam 30 expire_dict: proxy::expire quota: maildir:User Quota quota_rule: *:storage=500M quota_rule2: Trash:storage=50M quota_warning: storage=95%% /usr/local/bin/mail-quota-warning.sh 95 quota_warning2: storage=80%% /usr/local/bin/mail-quota-warning.sh 80 dict: expire: mysql:/etc/dovecot/dovecot-dict-expire.conf root@server:~# cat /etc/dovecot/dovecot-sql.conf driver = mysql connect = host=127.0.0.1 dbname=mailserver user=mailserver password=testpwd1 default_pass_scheme = CRAM-MD5 password_query = SELECT email AS user, password, quota_rule FROM view_users WHERE email='%u' root@server:~# cat /etc/dovecot/dovecot-dict-expire.conf connect = host=localhost dbname=mailserver user=mailserver_exp password=testpwd2 table = expire select_field = expire_stamp where_field = mailbox I tried using a "user_field = user" additionally as is written here: http://wiki.dovecot.org/Plugins/Expire. (Incidentally, there must be some mismatch in the documentation because the tables for v1.1 do not contain a field username). Of course I adapted the MySQL database accordingly. Unfortunately I was unable to produce a backtrace. Thanks, Joerg Leis

2 1

[Dovecot] Apple patches 1-5
by Mike Abbott 08 Jan '09

08 Jan '09

Here are the first few simple patches from Apple, based on dovecot-1.1.7. The comments with "APPLE" in them helped us merge in your new releases; feel free to remove them. Please let me know if you want subsequent patches in a different format, or if you have any questions. Patch #1. Some versions of Mac OS X have buggy CMSG_* macros. Patch #2. Don't set "<username>=1" in the environment. Patch #3. Some versions of Mac OS X return near-duplicate kevents. Patch #4. Null renames are actually pretty slow on HFS (Apple's file system), so avoid them. Patch #5. Required by Apple's lawyers.

3 12

[Dovecot] Apple patches 6-8
by Mike Abbott 08 Jan '09

08 Jan '09

Here are a few more patches. Still keeping it easy for now. Again the basis for these patches is dovecot-1.1.7. Patch #6. Solve a cross-compilation endianness issue. Currently, Dovecot assumes that the endianness of the build system is the same as the endianness of the runtime system. This is not necessarily true. We ran into this while compiling for i386 on a ppc machine. The patch switches to using gcc's __BIG_ENDIAN__ macro; see the comment in the patch to configure.in. It also removes the related and unused MAIL_INDEX_COMPAT_FLAGS parameter. This patch may be applicable to other build environments with a little tweaking. Patch #7. Replace all occurrences of "hash_create" and "hash_destroy" with "hash_table_create" and "hash_table_destroy" respectively. The symbols hash_create and hash_destroy conflict with symbols defined in <strhash.h> and libc. This showed up when loading dovecot's quota plugin (one of our future patches will add a hash table to it; stay tuned). The wrong hash_create was called which caused a crash at the first hash_insert. Apparently this is only a problem in loaded dynamic libraries and not linked-in ones. Patch #8. Back off after auth failures to deter abusers. Stalls 5 seconds per failed attempt. PS -- Please forgive me if I make a mistake at some point while splitting these patches out from our source tree. They're all mingled together on my end and it's sometimes tricky to extract just the right changes for each patch. If anything seems fishy just let me know and I'll take another look. Thanks.

4 8

[Dovecot] Checking for strtoumax and strtoimax
by J Springer 08 Jan '09

08 Jan '09

Tru64 5.1b PK6 : dovecot 1.1.7 : cc Configuration checks for strtoumax and strtoimax are positive but both are unresolved during compilation. strtoq and strtouq checks are negative. I can modify compat.c to use "unsigned long ret = 0;" in place of strtoumax (does this break anything?) and/but I would like to know how to handle strtoimax. Thanks in advance.

2 5

[Dovecot] allow_nets overridden by cache
by Guy 08 Jan '09

08 Jan '09

Hi, I've just started trying allow_nets on one of my servers. I have auth_debug and auth_verbose both enabled and the output is as follows: Oct 28 13:05:48 mink dovecot: auth-worker(default): auth(user(a)domain.net,x.x.x.x): allow_nets: Matching for network 127.0.0.1/8 Oct 28 13:05:48 mink dovecot: auth-worker(default): auth(user(a)domain.net,x.x.x.x): allow_nets: Matching for network 10.0.7.176/28 Oct 28 13:05:48 mink dovecot: auth-worker(default): passdb(user(a)domain.net,x.x.x.x): allow_nets check failed: IP not in allowed networks Oct 28 13:05:50 mink dovecot: auth(default): client out: FAIL 1265 user=username(a)aluminati.net Oct 28 13:05:50 mink dovecot: auth(default): cache(user(a)domain.net,x.x.x.x): hit: <hidden> user=user(a)domain.net Oct 28 13:05:50 mink dovecot: auth(default): client out: OK 1266 user=user(a)domain.net auth_cache_ttl is set to 300. If I set it to 1 then the allow_nets successfully rejects. Once I set it back up to 300 the cache overrides the result from the allow_nets check and let's the account log in even though the allow_nets check fails. I've tried waiting for longer than the 300 seconds and then logged in again, but I still get the same result as above. Is there a gotcha that I'm unaware of or have I done something stupid (which seems the most likely :P)? Thanks Guy -- Don't just do something...sit there!

2 4

[Dovecot] mbox "Next message unexpectedly lost" bug fixed
by Timo Sirainen 07 Jan '09

07 Jan '09

Good news, everyone! The most annoying bug in v1.1 series should finally be fixed by these patches: http://hg.dovecot.org/dovecot-1.1/rev/f28348c5201e http://hg.dovecot.org/dovecot-1.1/rev/74c47fcfb420 http://hg.dovecot.org/dovecot-1.1/rev/a28520d26b5a http://hg.dovecot.org/dovecot-1.1/rev/539f8d983285 http://hg.dovecot.org/dovecot-1.1/rev/9b7af230f9f0 They also make sure that mbox is never read while it's not locked. I think that could have happened in some situations with v1.1. Hopefully mbox handling will now finally be as stable in v1.1 as it was in v1.0. I'll make a new v1.1 release after seeing what bug fixes are coming from Apple, so probably next week.

3 3