- dovecot - dovecot.org

[Dovecot] Newbie questions: Load-balanced Dovecot with NFS storage
by Mike Brudenell 07 Mar '07

07 Mar '07

Hi, I experimented with Dovecot a while back and our site is now taking the plunge to switch to it from the UW IMAP server when we rejig the topology of our mail store and IMAP servers. Currently we have a farm of 5 IMAP servers running the UW server. Each has locally attached disk used to store people's mailboxes. Each user is allocated to one of these five servers and can only access their mail via that server. Our aim is to remove this restriction to provide resilience in the event of a server failing. Our thoughts... * We are envisaging having a farm of N (to be determined) servers running Dovecot. These will be closed systems: users do not have shell- access, and can only reach their mail via IMAP. * A single mailstore common to all N servers will be hosted on our NetApp filer and NFS-mounted on each server. Each server would be running Solaris 10. If we use all-new kit it will be x86-based, or we may have to redeploy some of our existing SPARC boxes to the new service. * We will be converting user mailboxes to be in Maildir format. * Our hardware load balancer, which supports persistent sessions, will be used to distribute users between the servers. This could either direct a user to any one of the N servers, or could be set to 'prefer' a given server for a particular user (only failing over to one of the others if their 'usual' server was unavailable). More on this in my questions below. :-) I have been scouring the Dovecot list, archives and Wiki and think I have come up with a set of issues we need to be aware of and things we have to do in order to make this work reliably. Please would you let me know if I have missed or misunderstood anything? Issues To Watch For ------------------- * For timestamp integrity our servers will be synched with NTP to our local time server. * Because sessions could be on different servers memory mapping of index files doesn't work well with NFS, so set mmap_disable=yes in dovecot.conf. * Dovecot relies on the mtime timestamp of mailbox files so the NFS-mounted mailstore needs to be mounted with these options on each server: actimeo=0 (Are there any other mount options we should use too?) * We will be using filesystem quotas on users mailboxes. We understand that Dovecot's index files are best on no-quota filestores so will store these separately. Questions --------- Q1. Would it be better to store the index files on NFS-shared filestore and direct users to any of the IMAP server machines? Or to store the index files on local disk and direct each user to their 'preferred' IMAP server machine? Q2. Does Dovecot (or "something") clean out old index files that haven't been accessed for a while? Eg, when a user has temporarily come through on a different IMAP server to normal. Or do the index files sit there untouched for evermore? Q3. Storing the index files on the NetApp filer would give us the ability to share them between servers and grow their volume as need be, but at the cost of performance. How big do the index files get? Are they typically a few kBytes per message? Per mailbox? Per user? Or more? Q4. We will be using Exim as the MTA, which can deliver direct to Maildir mailboxes. However I understand that Dovecot's "deliver" LDA adds the benefit of updating the index files as each message is delivered. Is this a significant gain? Or is there little difference in actual use? Q5. We have around 20,000 mail accounts and will therefore be seeing lots of concurrent IMAP sessions, usually secure (SSL) ones. I have seen mention that this can give rise to "Too many open files" errors under Solaris. How do we avoid this when we are likely to have several thousand concurrent IMAP sessions per server machine? With many thanks for your time and advice... Cheers, Mike Brudenell -- The Computing Service, University of York, Heslington, York Yo10 5DD, UK Tel:+44-1904-433811 FAX:+44-1904-433740 * Unsolicited commercial e-mail is NOT welcome at this e-mail address. *

5 9

[Dovecot] Time moving backwards error
by Emmett Pate 07 Mar '07

07 Mar '07

I'm running dovecot on an OpenBSD 3.7 server and after upgrading to RC25, the daemon started exiting with: Fatal: Time just moved backwards (1173221579 -> 1173221578)! This might cause a lot of problems, so I'll just kill myself now. After upgrading to RC26, I've gotten the error: Error: Time just moved backwards by 1 seconds. I'll sleep now until we're back in present. Occasionally, however, it's followed by: Fatal: Sleep interrupted, byebye. The server is running NTP, but I can't find any reason why it would have to roll the clock back a second so often. Until I get this figured out, I've gone back to RC19. Any ideas? Thanks. -- Emmett "Buddy" Pate Chief Technology Officer William E. Wood and Associates, Realtors epate(a)williamewood.com

3 2

[Dovecot] dovecot-ldap.conf - Active directory
by Cristian Salan 07 Mar '07

07 Mar '07

Hello, I have a configuration with dovecot-ldap working with openldap (on FreebSD 5.4 Release). Now I'm trying to do the same thing with an Active Directory (win2k3). I have installed an properly configured SFU. Via nss_ldap-pam_ldap is working fine (FreeBSD 6.2 Release). Because nss_ldap is reported broken with dovecot I'm trying to use dovecot-ldap for both passdb and userdb. My guessed filters/attributes follows: user_filter = (&(objectClass=user)(msSFU30Name=%u)) user_attrs = homeDirectory=msSFU30HomeDirectory, uidNumber=msSFU30UidNumber, gidNumber=msSFU30GidNumber pass_attrs = msSFU30Name,msSFU30Password pass_filter = (&(objectClass=user)(msSFU30Name=%u)) But it doesn't work. Any advice? Cristian Salan

2 1

[Dovecot] sporadic problem using ldap authentication
by Nuno Lopes 07 Mar '07

07 Mar '07

Hi, recently some users started complaining about sporadic authentication failures. When examining the logs I saw the following: Mar 7 03:20:24 gollum dovecot: auth(default): file auth-request.c: line 472 (auth_request_lookup_credentials_callback): assertion failed: (request->state == AUTH_REQUEST_STATE_PASSDB) Mar 7 03:20:24 gollum dovecot: auth(default): Raw backtrace: dovecot-auth [0x41d7fc] -> dovecot-auth [0x41da6c] -> dovecot-auth [0x40ddf3] -> dovecot-auth [0x416475] -> dovecot-auth [0x416825] -> dovecot-auth [0x41176d] -> dovecot-auth(io_loop_handler_run+0x13c) [0x420cec] -> dovecot-auth(io_loop_run+0x1c) [0x42005c] -> dovecot-auth(main+0x41e) [0x412ade] -> /lib64/tls/libc.so.6(__libc_start_main+0xea) [0x2a959fbaaa] -> dovecot-auth(ldap_msgfree+0x42) [0x4098da] Mar 7 03:20:24 gollum dovecot: child 10883 (auth) killed with signal 6 Mar 7 03:20:24 gollum dovecot: pop3-login: Can't connect to auth server at default: Resource temporarily unavailable Mar 7 03:20:24 gollum dovecot: auth(default): file auth-request.c: line 472 (auth_request_lookup_credentials_callback): assertion failed: (request->state == AUTH_REQUEST_STATE_PASSDB) Mar 7 03:20:24 gollum dovecot: auth(default): Raw backtrace: dovecot-auth [0x41d7fc] -> dovecot-auth [0x41da6c] -> dovecot-auth [0x40ddf3] -> dovecot-auth [0x416475] -> dovecot-auth [0x416825] -> dovecot-auth [0x41176d] -> dovecot-auth(io_loop_handler_run+0x13c) [0x420cec] -> dovecot-auth(io_loop_run+0x1c) [0x42005c] -> dovecot-auth(main+0x41e) [0x412ade] -> /lib64/tls/libc.so.6(__libc_start_main+0xea) [0x2a959fbaaa] -> dovecot-auth(ldap_msgfree+0x42) [0x4098da] Mar 7 03:20:24 gollum dovecot: child 10945 (auth) killed with signal 6 This output is from dovecot-1.0rc26 but this was first seen in 1.0rc7 and 1.0rc24 has similar results. I have seen no problem with the ldap server. I'm using similar setups in several other servers with no trouble at all. The main difference I see is that this machine is 64-bit and on a 2.6 linux kernel versus x86 on a 2.4. Thanks for any help you can provide. -- Nuno Lopes

2 1

[Dovecot] Question about auth-master, deliver, and SQL maps
by Richard Coleman 07 Mar '07

07 Mar '07

I have been running Postfix/Dovecot with a Postgresql backend for several months, and everything has been running great. Now I want to add server side filtering, so I'm attempting to configure deliver. The SQL maps for Postfix are all keyed upon the recipient email address. The SQL map (user_query) for Dovecot is keyed upon the IMAP login name, which is potentially unrelated to the email address. But when I configure deliver to use auth-master, it doesn't work right, because it is passing the recipient email address as a key to user_query, which is wrong. Somehow, I need auth-master to use a different SQL query from user_query when doing lookups for deliver. In essence, I want deliver to use the same SQL queries where were previously used for the Postfix "virtual" delivery agent. Is there any way to do this? Richard Coleman rcoleman(a)criticalmagic.com

2 1

[Dovecot] 1.0.rc25 released
by Timo Sirainen 07 Mar '07

07 Mar '07

http://dovecot.org/releases/dovecot-1.0.rc25.tar.gz http://dovecot.org/releases/dovecot-1.0.rc25.tar.gz.sig Instead of having "Should v1.0 be released already" discussion, how about having "What's still missing from wiki.dovecot.org and how could it be improved" discussion? And what should the wiki exported to doc/ directory in the tarball look like? * If time moves backwards, Dovecot kills itself instead of giving random problems. + Added --with-headers configure option to install .h files. Binary package builders could use this to create some dovecot-dev package to make compiling plugins easier. - PLAIN authentication: Don't crash dovecot-auth with invalid input. - IMAP APPEND: Don't crash if saving fails - IMAP LIST: If prefix.INBOX has children and we're listing under prefix.%, don't drop the prefix. - mbox: Broken X-UID headers still weren't handled correctly. - mail-log plugin: Fixed deleted/undeleted logging.

16 29

[Dovecot] maildirsize corruption
by Dean Manners 06 Mar '07

06 Mar '07

Using maildir++ with LDA and POP/IMAP we have been seeing intermittant maildirsize corruption. Where an operation will be prepended by a handful of ASCI null characters ^@^@. Eg: # cat maildirsize 5120000S -46895 -1 93359 1 60576 1 2858 1 49547 1 -206340 -4 94603 1 84785 1 54463 1 82486 1 97104 1 ^@^@^@^@^@^@^@^@40488 1 73397 1 83662 1 24668 1 42673 1 12081 1 -713606 -12 The instances of this that we have seen have all been deliver operations. Anyone else seen this ? Regards __________________________________________ Dean Manners

3 5

[Dovecot] auth problem with secure authentication (cram-md5)
by Adrian Stoica 06 Mar '07

06 Mar '07

I use dovecot 1.0rc25 in dovecot.conf: auth default { mechanisms = LOGIN CRAM-MD5 DIGEST-MD5 PLAIN APOP passdb sql { args=/etc/dovecot-crammd5.conf } passdb sql { args=/etc/dovecot-sql.conf } userdb sql { args=/etc/dovecot-sql.conf } } in dovecot-sql.conf: driver = mysql connect = host=/var/run/mysql/mysql.sock port= dbname=mail user=vmailuser password=vmailpassword password_query = SELECT clear AS password , user FROM mailbox WHERE user = '%n' AND IF('%d'='',clear = '%w',domain = '%d') AND status ='1' LIMIT 1 user_query = SELECT CONCAT(home,'/',maildir) AS home, '1001' AS uid, '1001' AS gid,CONCAT('maildir:storage=', quota * 1024 ) AS quota FROM mailbox WHERE (user = '%n' AND status = '1') OR (user = '%n' AND domain = '%d' AND status = '1') default_pass_scheme = CLEARTEXT in dovecot-crammd5.conf: driver = mysql connect = host=/var/run/mysql/mysql.sock port= dbname=mail user=vmailuser password=vmailpassword password_query = SELECT crypt AS password , user FROM mailbox WHERE user = '%n' AND status = '1' LIMIT 1 default_pass_scheme = CRAM-MD5 wen i "use secure authentication" and try login with thunderbird : dovecot: Mar 02 10:49:39 Info: Dovecot v1.0.rc25 starting up dovecot: Mar 02 10:49:40 Info: auth-worker(default): mysql: Connected to /var/run/mysql/mysql.sock (mail) dovecot: Mar 02 10:49:40 Info: auth-worker(default): mysql: Connected to /var/run/mysql/mysql.sock (mail) dovecot: Mar 02 10:49:47 Info: auth(default): client in: AUTH 1 CRAM-MD5 service=IMAP secured lip=192 .168.0.157 rip=192.168.0.64 dovecot: Mar 02 10:49:47 Info: auth(default): client out: CONT 1 PDY3MTAzMjE0OTY4NDA0MTguMTE3MjgzMjU4N0B0ZXN0Pg= = dovecot: Mar 02 10:49:47 Info: auth(default): client in: CONT 1 YWRpIGU4N2JjYTZiYzA5OTZkYjY0MDk5ZTZjZmQ2YjUxNWE 2 dovecot: Mar 02 10:49:47 Info: auth-worker(default): sql(adi,192.168.0.64): query: SELECT crypt AS password , user FROM mailbox WHERE user = 'adi' AND status = '1' LIMIT 1 dovecot: Mar 02 10:49:47 Info: auth(default): password(adi,192.168.0.64): Credentials: abd122fc9335009004012948209313d3 b31b6fdbc502bab773e90f7a195d341b dovecot: Mar 02 10:49:47 Info: auth(default): client out: OK 1 user=adi dovecot: Mar 02 10:49:47 Info: auth(default): master in: REQUEST 1 4319 1 dovecot: Mar 02 10:49:47 Info: auth-worker(default): sql(adi,192.168.0.64): SELECT CONCAT(home,'/',maildir) AS home, '1 001' AS uid, '1001' AS gid,CONCAT('maildir:storage=', quota * 1024 ) AS quota FROM mailbox WHERE (user = 'adi' AND stat us = '1') OR (user = 'adi' AND domain = '' AND status = '1') dovecot: Mar 02 10:49:47 Info: auth(default): master out: USER 1 adi home=/srv/vmail/adrian.eu/adi/ uid=100 1 gid=1001 quota=maildir:storage=512000 dovecot: Mar 02 10:49:47 Info: imap-login: Login: user=<adi>, method=CRAM-MD5, rip=192.168.0.64, lip=192.168.0.157, TLS dovecot: Mar 02 10:49:48 Info: auth(default): client in: AUTH 1 CRAM-MD5 service=IMAP secured lip=192 .168.0.157 rip=192.168.0.64 dovecot: Mar 02 10:49:48 Info: auth(default): client out: CONT 1 PDE0NjQxNTExNDY4MDE5MTQuMTE3MjgzMjU4OEB0ZXN0Pg= = dovecot: Mar 02 10:49:49 Info: auth(default): client in: CONT 1 YWRpIGI5YjliNmM3ZmMxMzUxMTU0Y2JiMGU2M2U4YmQxMzR k dovecot: Mar 02 10:49:49 Info: auth(default): password(adi,192.168.0.64): Credentials: abd122fc9335009004012948209313d3 b31b6fdbc502bab773e90f7a195d341b dovecot: Mar 02 10:49:49 Error: child 4311 (auth) killed with signal 11 restart thunderbird and try again to login: dovecot: Mar 02 10:55:16 Info: auth(default): client in: AUTH 1 CRAM-MD5 service=IMAP secured lip=192.168.0.157 rip=192.168.0.64 dovecot: Mar 02 10:55:16 Info: auth(default): client out: CONT 1 PDMwMTQ0NjI3Njc3MTYzODEuMTE3MjgzMjkxNkB0ZXN0Pg== dovecot: Mar 02 10:55:16 Info: auth(default): client in: CONT 1 YWRpIGMzYTRlODYwMTk1NjVkZDE1NDUzYzI5NGYxNWVhNjVm dovecot: Mar 02 10:55:16 Info: auth(default): password(adi,192.168.0.64): Credentials: abd122fc9335009004012948209313d3b31b6fdbc502bab773e90f7a195d341b dovecot: Mar 02 10:55:16 Error: child 4347 (auth) killed with signal 11 dovecot: Mar 02 10:55:18 Info: auth-worker(default): mysql: Connected to /var/run/mysql/mysql.sock (mail) dovecot: Mar 02 10:55:18 Info: auth-worker(default): mysql: Connected to /var/run/mysql/mysql.sock (mail) (In thunderbird : "Login to server failed.") After that the login works ok : dovecot: Mar 02 10:55:18 Info: auth-worker(default): mysql: Connected to /var/run/mysql/mysql.sock (mail) dovecot: Mar 02 10:55:18 Info: auth-worker(default): mysql: Connected to /var/run/mysql/mysql.sock (mail) dovecot: Mar 02 10:57:46 Info: imap-login: Aborted login: method=CRAM-MD5, rip=192.168.0.64, lip=192.168.0.157, TLS dovecot: Mar 02 10:57:53 Info: auth(default): client in: AUTH 1 CRAM-MD5 service=IMAP secured lip=192.168.0.157 rip=192.168.0.64 dovecot: Mar 02 10:57:53 Info: auth(default): client out: CONT 1 PDc2NzE5MTcwMzE4MzI1NzUuMTE3MjgzMzA3M0B0ZXN0Pg== dovecot: Mar 02 10:57:53 Info: auth(default): client in: CONT 1 YWRpIGIxOGQ3NmFmMGIwYmU3OTkyMjk4MmI1OWRlYTQzNTQz dovecot: Mar 02 10:57:53 Info: auth-worker(default): sql(adi,192.168.0.64): query: SELECT crypt AS password , user FROM mailbox WHERE user = 'adi' AND status = '1' LIMIT 1 dovecot: Mar 02 10:57:53 Info: auth(default): password(adi,192.168.0.64): Credentials: abd122fc9335009004012948209313d3b31b6fdbc502bab773e90f7a195d341b dovecot: Mar 02 10:57:53 Info: auth(default): client out: OK 1 user=adi dovecot: Mar 02 10:57:53 Info: auth(default): master in: REQUEST 6 4337 1 dovecot: Mar 02 10:57:53 Info: auth-worker(default): sql(adi,192.168.0.64): SELECT CONCAT(home,'/',maildir) AS home, '1001' AS uid, '1001' AS gid,CONCAT('maildir:storage=', quota * 1024 ) AS quota FROM mailbox WHERE (user = 'adi' AND status = '1') OR (user = 'adi' AND domain = '' AND status = '1') dovecot: Mar 02 10:57:53 Info: auth(default): master out: USER 6 adi home=/srv/vmail/adrian.eu/adi/ uid=1001 gid=1001quota=maildir:storage=512000 dovecot: Mar 02 10:57:53 Info: imap-login: Login: user=<adi>, method=CRAM-MD5, rip=192.168.0.64, lip=192.168.0.157, TLS Everytime i have to input password for 2-3 times to login with secure authentication.

2 2

[Dovecot] 1.0.rc24 released
by Timo Sirainen 06 Mar '07

06 Mar '07

And more fixes. * Dovecot now fails to load plugins that were compiled for different Dovecot version, unless version_ignore=yes is set. This needs to be explicitly set in plugins, so out-of-tree plugins won't have this check by default. - pop3_lock_session=yes could cause deadlocks, and with maildir the uidlist lock could have been overridden after 2 minutes causing problems - PAM wasted CPU by calling a timeout function 1000x too often - Trash plugin was more or less broken with multiple namespaces and with multiple trash mailboxes

19 36

[Dovecot] quota - dict permission denied
by Abu Gid 06 Mar '07

06 Mar '07

when I enabled quota plugin the following error occured : net_connect_unix(/usr/local/var/run/dovecot/dict-server) failed: Permission denied Any ideas? My config are as follows : .. dict { #quota = mysql:/etc/dovecot-dict-quota.conf quotadict = mysql:/usr/local/etc/dovecot-dict-quota.conf } .. quota = dict:storage=81920:messages=5000 proxy::quotadict /usr/local/etc/dovecot-dict-quota.conf: connect = host=xxxxx dbname=xxxxx user=xxxxx password=xxxxx table = quota select_field = current where_field = path username_field = username ----- I can connect to the database thru mysql prompt. dovecot version 1.0 rc5 -- azhar --

2 10