March 2007 - dovecot - dovecot.org

[Dovecot] Migration
by John s 06 Mar '07

06 Mar '07

Hi, I have a Mailserver setup with Qmail+Vpopmail+courier-imap with 2000 users.The authentication in vpopmail was in plaint text file based.Right now the issue is while opening a user email box with 50-100 mails it is opening fastly.Butfor email-boxes with attachments and increased number of mails it take long time to open the Imap connection. So I would like to change the Imaps to Dovecot.Please suggest if I can migrate from courier-imap to dovecot with vpopmail and qmail without changing the present username and passwords. Thanks in advance for any help. Regards John

2 1

[Dovecot] new crashes: is the index/mail cache endian neutral?
by Dan Price 06 Mar '07

06 Mar '07

Part of our migration plan takes our users from one endianness to another (big to little). Will the index and mail cache files survive? I'm seeing some new core dumps as the first test user is migrated, which makes me think... not. What about if we want to build dovecot 64 bit in the future? Will that cause problems too? Stack trace is below the log messages. I've edited the user's name out and replaced it with foobar, and cleansed out a few sensitive things. -dp dovecot: Mar 01 08:46:49 Info: imap-login: Login: user=<foobar>, method=PLAIN, r ip=WWW.XXX.YYY.ZZ, lip=AAA.BBB.CC.DD, TLS dovecot: Mar 01 08:46:49 Info: IMAP(foobar): Effective uid=88442, gid=10, home=/ home/foobar dovecot: Mar 01 08:46:49 Info: IMAP(foobar): mbox: data=/home/foobar:INBOX=/var/ mail/foobar:INDEX=/home/foobar/Mail/imapd-indices dovecot: Mar 01 08:46:49 Info: IMAP(foobar): mbox: root=/home/foobar, index=/hom e/foobar/Mail/imapd-indices, inbox=/var/mail/foobar dovecot: Mar 01 08:46:50 Error: IMAP(foobar): Corrupted index file /home/foobar/ Mail/imapd-indices/.imap/INBOX/dovecot.index: messages_count too large (0 > 0) dovecot: Mar 01 08:46:50 Error: IMAP(foobar): Corrupted index file /home/foobar/ Mail/imapd-indices/.imap/INBOX/dovecot.index: messages_count too large (0 > 0) dovecot: Mar 01 08:46:50 Error: IMAP(foobar): Corrupted index cache file /home/f oobar/Mail/imapd-indices/.imap/INBOX/dovecot.index.cache: field header points ou tside file dovecot: Mar 01 08:46:51 Error: IMAP(foobar): Corrupted transaction log file /ho me/foobar/Mail/imapd-indices/.imap/INBOX/dovecot.index.log: unknown record type 0x10 dovecot: Mar 01 08:46:51 Warning: IMAP(foobar): fscking index file /home/foobar/ Mail/imapd-indices/.imap/INBOX/dovecot.index dovecot: Mar 01 08:46:51 Error: IMAP(foobar): Unexpected transaction log desync with index /home/foobar/Mail/imapd-indices/.imap/INBOX/dovecot.index dovecot: Mar 01 08:46:51 Info: IMAP(foobar): Disconnected: Mailbox is in inconsi stent state, please relogin. dovecot: Mar 01 08:48:06 Info: imap-login: Login: user=<foobar>, method=PLAIN, r ip=WWW.XXX.YYY.ZZ, lip=AAA.BBB.CC.DD, TLS dovecot: Mar 01 08:48:06 Info: IMAP(foobar): Effective uid=88442, gid=10, home=/ home/foobar dovecot: Mar 01 08:48:06 Info: IMAP(foobar): mbox: data=/home/foobar:INBOX=/var/ mail/foobar:INDEX=/home/foobar/Mail/imapd-indices dovecot: Mar 01 08:48:06 Info: IMAP(foobar): mbox: root=/home/foobar, index=/hom e/foobar/Mail/imapd-indices, inbox=/var/mail/foobar dovecot: Mar 01 09:07:09 Error: IMAP(foobar): Corrupted index file /home/foobar/ Mail/imapd-indices/Mail/.imap/zfs/dovecot.index: messages_count too large (27853 45536 > 1) dovecot: Mar 01 09:07:10 Error: IMAP(foobar): Corrupted index cache file /home/f oobar/Mail/imapd-indices/Mail/.imap/zfs/dovecot.index.cache: field header points outside file dovecot: Mar 01 09:20:44 Error: IMAP(foobar): Corrupted index file /home/foobar/ Mail/imapd-indices/Mail/.imap/NDMP/dovecot.index: messages_count too large (5200 93696 > 3) dovecot: Mar 01 09:20:44 Error: IMAP(foobar): Corrupted transaction log file /ho me/foobar/Mail/imapd-indices/Mail/.imap/NDMP/dovecot.index.log: start_offset (61 44) > file size (1768) dovecot: Mar 01 09:20:44 Warning: IMAP(foobar): fscking index file /home/foobar/ Mail/imapd-indices/Mail/.imap/NDMP/dovecot.index dovecot: Mar 01 09:20:44 Error: child 111112 (imap) killed with signal 11 -dp #0 0x0809eaa1 in mail_index_lock_exclusive (index=0x8117c70, lock_id_r=0x8047490) at mail-index-lock.c:326 ret = 0 __PRETTY_FUNCTION__ = "mail_index_lock_exclusive" #1 0x0809e15e in mail_index_fsck (index=0x8117c70) at mail-index-fsck.c:169 error = 0x80e9498 "" lock_id = 135173240 file_seq = 16777216 file_offset = 6144 ret = 134509928 lock_log = true #2 0x080a5c49 in mail_transaction_log_file_set_corrupted (file=0x8101c30, fmt=0x80d51a8 "start_offset (%llu) > file size (%llu)") at mail-transaction-log.c:64 No locals. #3 0x080a785e in mail_transaction_log_file_map (file=0x8101c30, start_offset=6144, end_offset=18446744073709551615) at mail-transaction-log.c:1313 index = (struct mail_index *) 0x8117c70 st = {st_dev = 47777187, st_pad1 = {0, 0, 0}, st_ino = 15761, st_mode = 33152, st_nlink = 1, st_uid = 88442, st_gid = 10, st_rdev = 4294967295, st_pad2 = {0, 0}, st_size = 1768, st_atim = { tv_sec = 1172769644, tv_nsec = 151892829}, st_mtim = {tv_sec = 1172522367, tv_nsec = 0}, st_ctim = {tv_sec = 1172732240, tv_nsec = 566025559}, st_blksize = 2048, st_blocks = 4, st_fstype = "lofs", '\0' <repeats 11 times>, st_pad4 = {0, 0, 0, 0, 0, 0, 0, 0}} ret = 1768 use_mmap = 1 __PRETTY_FUNCTION__ = "mail_transaction_log_file_map" #4 0x080a7bca in mail_transaction_log_sync_lock (log=0x8101bd8, file_seq_r=0x0, file_offset_r=0x0) at mail-transaction-log.c:1373 __PRETTY_FUNCTION__ = "mail_transaction_log_sync_lock" #5 0x0809dad3 in mail_index_open (index=0x8117c70, flags=17, lock_method=MAIL_INDEX_LOCK_FCNTL) at mail-index.c:1423 i = 0 ret = 135294216 #6 0x08096241 in index_storage_mailbox_init (ibox=0x81164b8, name=0x0, flags=0, move_to_memory=false) at index-storage.c:360 storage = (struct mail_storage *) 0x80f62a0 index_flags = 17 lock_method = MAIL_INDEX_LOCK_FCNTL ret = 0 __PRETTY_FUNCTION__ = "index_storage_mailbox_init" #7 0x0807cbfc in mbox_alloc (storage=0x80f62a0, index=0x0, name=0x80f68b0 "Mail/NDMP", path=0x80e92f0 "/home/foobar/Mail/NDMP", flags=14) at mbox-storage.c:546 mbox = (struct mbox_mailbox *) 0x81164b8 pool = 0x1 #8 0x0807cd7c in mbox_open (storage=0x80f62a0, name=0x80f68b0 "Mail/NDMP", flags=14) at mbox-storage.c:662 mbox = (struct mbox_mailbox *) 0x80e9310 index = (struct mail_index *) 0x0 path = 0x80e92f0 "/home/foobar/Mail/NDMP" index_dir = 0x80e9310 "/home/foobar/Mail/imapd-indices/Mail/.imap/NDMP" #9 0x0807cf67 in mbox_mailbox_open (_storage=0x80f62a0, name=0x80f68b0 "Mail/NDMP", input=0x0, flags=14) at mbox-storage.c:773 path = 0x80e92d8 "/home/foobar/Mail/NDMP" st = {st_dev = 47777187, st_pad1 = {0, 0, 0}, st_ino = 15330, st_mode = 33152, st_nlink = 1, st_uid = 88442, st_gid = 10, st_rdev = 4294967295, st_pad2 = {0, 0}, st_size = 1189083, st_atim = { tv_sec = 1172732807, tv_nsec = 577163661}, st_mtim = {tv_sec = 1172679608, tv_nsec = 0}, st_ctim = {tv_sec = 1172731930, tv_nsec = 691932658}, st_blksize = 131072, st_blocks = 2224, st_fstype = "lofs", '\0' <repeats 11 times>, st_pad4 = {0, 0, 0, 0, 0, 0, 0, 0}} #10 0x080acbf6 in mailbox_open (storage=0x0, name=0x80f68b0 "Mail/NDMP", input=0x0, flags=14) at mail-storage.c:365 No locals. #11 0x08067f63 in cmd_copy (cmd=0x80f65cc) at cmd-copy.c:83 client = (struct client *) 0x80f6588 storage = (struct mail_storage *) 0x80f62a0 destbox = (struct mailbox *) 0x8 t = (struct mailbox_transaction_context *) 0x806bec0 search_arg = (struct mail_search_arg *) 0x80f8800 messageset = 0x80f68a8 "542506" mailbox = 0x80f68b0 "Mail/NDMP" sync_flags = 0 ret = 134511032 #12 0x0806affd in cmd_uid (cmd=0x80f65cc) at cmd-uid.c:19 cmd_name = 0x80f68a0 "copy" #13 0x0806b9ce in client_handle_input (cmd=0x80f65cc) at client.c:331 client = (struct client *) 0x80f6588 __PRETTY_FUNCTION__ = "client_handle_input" #14 0x0806b962 in client_handle_input (cmd=0x80f65cc) at client.c:388 client = (struct client *) 0x80f6588 __PRETTY_FUNCTION__ = "client_handle_input" #15 0x0806bb10 in _client_input (context=0x80f6588) at client.c:428 client = (struct client *) 0x80f6588 cmd = (struct client_command_context *) 0x80f65cc ret = 2 #16 0x080bdc35 in io_loop_handler_run (ioloop=0x80f3d80) at ioloop-poll.c:199 ctx = (struct ioloop_handler_context *) 0x80f3db8 pollfd = (struct pollfd *) 0x2 tv = {tv_sec = 0, tv_usec = 617423} io = (struct io *) 0x8105460 t_id = 2 msecs = 0 ret = 0 call = true -- Daniel Price - Solaris Kernel Engineering - dp(a)eng.sun.com - blogs.sun.com/dp

2 1

[Dovecot] Plugins in Python?
by Maykel Moya 06 Mar '07

06 Mar '07

Timo Do you have plans of adding support for writing plugins in high level languages like Python? Regards maykel

2 1

[Dovecot] messages lingering in .Trash/new instead of .Trash/cur
by Glenn Leavell 06 Mar '07

06 Mar '07

We're running Dovecot rc15 on Solaris 9, providing IMAP access to e-mail stored in Maildir++. I've noticed that a significant number of message files are currently sitting in the directories ~/Maildir/.Trash/new/. I would have expected that all messages in .Trash would be in the /cur subdirectory (as most are), and I'm not sure how they're landing in /new. If I connect to Dovecot and run something like the following, the messages all *do* immediately end up in .Trash/cur/ : x select blah * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags * permitted. * 4 EXISTS * 0 RECENT * OK [UIDVALIDITY 1150071896] UIDs valid * OK [UIDNEXT 5] Predicted next UID x OK [READ-WRITE] Select completed. x copy 4 "Trash" x OK Copy completed. Most of our users are using Thunderbird, and after looking at various outputs of rawlog, TB seems to be simply issuing uid copy commands to "Trash" like the ones above when a user deletes a message. Do any of you know what IMAP interaction with Dovecot can lead to message files appearing in .Trash/new/ ? Thanks very much, Glenn -- Glenn Leavell <glenn(a)usg.edu> Office of Information and Instructional Technology Board of Regents of the University System of Georgia

2 1

[Dovecot] Driving quota out of a database.
by David Craigon 06 Mar '07

06 Mar '07

Hi there, I'm using postgres as a userdb for dovecot. I'm am trying to get the quota plugin to work. I have a column in my table that returns the quota in KB. I am using the postgres expression "('maildir:storage=' || mailstore_control.mailbox.quota) AS quota" as recommended in the manuals. This expression is NULL to start with, which I take to mean unlimited quota. Trouble is, if I change it to a value, and then change it back to null, I don't seem to go back to unlimited quota. I'm using maildir++ quotas. What gives? :-) David Griffin Information Systems Ltd 20 St Christopher's Way, Pride Park, Derby DE24 8JY www.griffin.com <http://www.griffin.com/> Registered in England and Wales. Company Registration Number: 3676297.

3 2

[Dovecot] Replacing qmail + vpopmail with: sendmail + dovecot
by Oliver Schulze L. 06 Mar '07

06 Mar '07

Hi, I wonder if it posible to replace qmail + vpopmail with sendmail + dovecot? The idea is like this: - use sendmail + milter API to filter spam/virus and deliver the email to the dovevot MDA (LDA) - dovecot should save the email to the directory as specified in the vpopmail DB - maintain the vpopmail BD (users + passwd) - use dovecot pop3/imap for the clients - use sendmail again to receive the email from the pop3/imap users and deliver the email to the internet as usual. Anyone has done this? It is posible in the paper, in the real world? many thanks Oliver -- Oliver Schulze L. | Get my e-mail after a captcha in: Asuncion - Paraguay | http://tinymailto.com/oliver

5 6

[Dovecot] My "fighting spam" article is live!
by Esther Schindler 06 Mar '07

06 Mar '07

Getting Clueful: Five Things You Should Know About Fighting Spam The battle for your users’ e-mail inboxes probably will never end, but it’s not a failure of technology. Experienced e-mail and system administrators share the key points they really, really wish you understood. http://www.cio.com/technology/infrastructure/security/spam/ five_things_about_fighting_spam.html?CID=28830 Thanks to everybody who contributed. :-) Esther

10 14

[Dovecot] Bug: mbox-sync-parse.c: line 228
by Dean Quanne 06 Mar '07

06 Mar '07

[root@mail postfix]# dovecot --version 1.0.rc25 [root@mail postfix]# dovecot -n # /usr/local/etc/dovecot.conf ssl_cert_file: /usr/share/ssl/certs/dovecot.pem ssl_key_file: /usr/share/ssl/private/dovecot.pem login_dir: /usr/local/var/run/dovecot/login login_executable: /usr/local/libexec/dovecot/imap-login mbox_write_locks: fcntl auth default: passdb: driver: pam userdb: driver: passwd Attempting to delete an email yields ... (from Thunderbird, Becky and Squirrel WebMail) Mar 5 23:58:30 localhost dovecot: imap-login: Login: user=<dean>, method=PLAIN, rip=192.168.0.81, lip=81.235.169.182, TLS Mar 5 23:58:30 localhost dovecot: IMAP(dean): file mbox-sync-parse.c: line 228 (parse_x_imap_base): assertion failed: (ctx->sync_ctx->next_uid > ctx->sync_ctx->prev_msg_uid) Mar 5 23:58:30 localhost dovecot: IMAP(dean): Raw backtrace: imap [0x80a8265] -> imap [0x80a7de7] -> imap [0x8070a04] -> imap [0x8070a17] -> imap(mbox_sync_parse_next_mail+0x200) [0x807101c] -> imap(mbox_sync_rewrite+0x94f) [0x8077813] -> imap [0x80724bc] -> imap(mbox_sync+0x3e2) [0x807447a] -> imap [0x806cd9a] -> imap(cmd_copy+0x23d) [0x8057599] -> imap(cmd_uid+0x76) [0x805a60a] -> imap [0x805afb6] -> imap [0x805af4a] -> imap(_client_input+0x78) [0x805b0fc] -> imap(io_loop_handler_run+0x131) [0x80ad9e1] -> imap(io_loop_run+0x21) [0x80acf79] -> imap(main+0x458) [0x8061fc4] -> /lib/tls/libc.so.6(__libc_start_main+0xd3) [0x467e23] -> imap [0x8056799] Mar 5 23:58:30 localhost dovecot: child 4221 (imap) killed with signal 6 -- Dean Quanne

1 0

[Dovecot] MBOX to MAILDIR conversion
by Marc Perkel 05 Mar '07

05 Mar '07

Thinking about going from mbox to maildir. Started with mbox way back when but thinking about making the conversion. The main reason is that I use rsync for backups and when these huge mailboxes are touched I have to copy the whole thing. With maildir I only copy the files that change. So - my questions is - what's the easiest way to make this happan. I assume there are conversion utilities. I'm using Exim as an MTA. Also - using Ext3 for the file system on a Raid 10 array. Backup server will run Ext3 Raid 1. Is the number of files in one directory going to be a problem? And - any tip/tricks would be appreciated. Things to look out for? Still in planning stage. Thanks in Advance.

4 4

[Dovecot] Dovecot Deliver
by Marc Perkel 04 Mar '07

04 Mar '07

Not quite sure where the docs are on Dovecot deliver. Went to the wiki and it left me confused. The docs there definitely need some work. Anyone have some examples to share?

2 1