- dovecot - dovecot.org

[Dovecot] uw-imap to dovecot conversion tool
by Joe Pruett 14 Apr '09

14 Apr '09

i plan to switch from uw-imap to dovecot on a couple of medium size systems and so i have written a conversion tool that understands mbox, mbx, and mix formats and converts to maildir format while trying to preserve uid info, keywords, flags, subscriptions, etc. it seems to work for me now, but i think some feedback might be useful. so if you have some files to convert and a desire to help, please download: http://www.spiretech.com/~joey/uw2dovecot.pl and give it a whirl. it is very sparsely documented right now but isn't too complex (yet). by default it copies your INBOX, mail dir, and .mailboxlist info into Maildir. those filenames can be overridden. i'd suggest running with -v to watch it. it requires that the output directory not exist before starting, so it should be relatively safe. it doesn't lock any files being read, so an active uw tree could cause it indigestion. it's designed to run while inbound mail and pop/imap sessions are disabled. it uses a few perl modules, but they should be standard with perl itself. it does do everthing in memory, so the largest message you have will be in memory and will be duplicated at least once as crlf pairs are replaced. also, this assumes you'll change the uid format in dovecot.conf to %08Xu%08Xv so that it doesn't change uids for the clients. send feedback to me and cc the list if you feel it useful. once this is stable, i'll get it added to the dovecot wiki.

2 2

[Dovecot] why not install utils (idxview logview ...) with dovecot in bin/sbin dir?
by Xueron Nee 14 Apr '09

14 Apr '09

Hi, I just found these useful tools :) they were installed in libexec dir :) -- Xueron Nee http://www.xueron.com

3 4

[Dovecot] Handling ENFILE
by Mike Abbott 14 Apr '09

14 Apr '09

My mail server consumed all of its configured file table slots and started returning ENFILE ("Too many open files in system") for many operations. This wreaked havoc with dovecot-1.1.13. Here are some areas where dovecot should detect and more gracefully handle ENFILE error returns: 1. Deliver should report an error more helpful than "Unknown internal error" when fdatasync_path fails: deliver(user): Apr 10 14:53:20 Error: fdatasync_path(/Volumes/MailData/ user/new) failed: Too many open files in system deliver(user): Apr 10 14:53:20 Info: msgid=<...>: save failed to INBOX: BUG: Unknown internal error 2. Index files should not be fscked when they could not be opened due to ENFILE: Error: IMAP(user): open() failed with index file /Volumes/MailData/ user/.Deleted Messages/dovecot.index: Too many open files in system Warning: IMAP(user): fscking index file /Volumes/MailData/ user/.Deleted Messages/dovecot.index 3. Dovecot should not crash when mail transaction logs become corrupt, or appear to have become corrupt due to ENFILE. Here is a sequence of error messages from the log but I can't tell whether the IMAP ones up through the panic in the middle are all from the same process, or from different processes. deliver(user): Apr 10 14:55:32 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 14:55:54 Error: open() failed with index file / Volumes/MailData/user/dovecot.index: Too many open files in system deliver(user): Apr 10 14:55:54 Error: open() failed with file /Volumes/ MailData/user/dovecot.index.log.2: Too many open files in system deliver(user): Apr 10 14:55:57 Error: open() failed with index cache file /Volumes/MailData/user/dovecot.index.cache: Too many open files in system dovecot: Apr 10 14:56:08 Warning: IMAP(user): fscking index file / Volumes/MailData/user/dovecot.index dovecot: Apr 10 14:56:16 Error: IMAP(user): Fixed index file /Volumes/ MailData/user/dovecot.index: log_file_seq 15 -> 2 deliver(user): Apr 10 14:56:19 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 14:56:20 Error: Corrupted transaction log file / Volumes/MailData/user/dovecot.index.log seq 2: record size too small (type=0x0, offset=23088, size=0) (sync_offset=36848) deliver(user): Apr 10 14:56:20 Warning: fscking index file (in-memory index) deliver(user): Apr 10 14:56:21 Info: msgid=<...>: save failed to INBOX: Internal error occurred. Refer to server log for more information. [2009-04-10 14:56:20] dovecot: Apr 10 14:56:40 Panic: IMAP(user): file mail-transaction-log- view.c: line 103 (mail_transaction_log_view_set): assertion failed: (min_file_seq <= max_file_seq) dovecot: Apr 10 14:56:40 Error: IMAP(user): Raw backtrace: [see below for better backtrace] deliver(user): Apr 10 14:56:45 Error: Transaction log file /Volumes/ MailData/user/dovecot.index.log: marked corrupted deliver(user): Apr 10 14:57:04 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 14:57:23 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 14:57:49 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 14:58:21 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 14:58:41 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 14:59:09 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 14:59:28 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 14:59:43 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 15:00:57 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 15:03:01 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 15:03:27 Info: msgid=<...>: saved mail to INBOX dovecot: Apr 10 15:21:41 Error: IMAP(user): Corrupted transaction log file /Volumes/MailData/user/dovecot.index.log seq 2: Invalid transaction log size (36848 vs 23088): /Volumes/MailData/user/ dovecot.index.log (sync_offset=36848) dovecot: Apr 10 15:21:42 Info: imap-login: Login: user=<user>, method=CRAM-MD5, rip=..., lip=..., TLS The backtrace for the assertion failure was: 0 libSystem.B.dylib 0x00007fff8007c102 __kill + 10 1 libSystem.B.dylib 0x00007fff800f7ffc abort + 83 2 imap 0x000000010006a22e default_fatal_finish + 70 3 imap 0x000000010006a27c i_syslog_fatal_handler + 0 4 imap 0x00000001000699db i_info + 0 5 imap 0x0000000100058542 mail_transaction_log_view_set + 104 6 imap 0x0000000100053b4c view_sync_set_log_view_range + 125 7 imap 0x0000000100053fc5 mail_index_view_sync_begin + 378 8 imap 0x000000010003c26e index_mailbox_sync_init + 129 9 imap 0x0000000100018457 maildir_storage_sync_init + 229 10 imap 0x000000010000c6bb imap_sync_init + 132 11 imap 0x000000010000c90d cmd_sync_delayed + 477 12 imap 0x000000010000716a client_handle_input + 409 13 imap 0x0000000100007244 client_input + 148 14 imap 0x0000000100071228 io_loop_handler_run + 272 15 imap 0x00000001000705ed io_loop_run + 55 16 imap 0x000000010000e219 main + 1635 17 imap 0x00000001000016c0 start + 52 I have since configured more open files, and enabled core dumps and added the pid to the mail_log_prefix for better tracking, but this is all I have to work with for now. Too bad the values of min_file_seq and max_file_seq are not logged in the panic.

2 1

[Dovecot] Dovecot+Sieve
by Andrey Garkin 13 Apr '09

13 Apr '09

Hi All!!! I'm configure mail server with Postix+Mysql+Dovecot. How can I'm using to work sieve plugins? My dovecot.conf /protocol lda { mail_plugin_dir = /usr/lib/dovecot/lda #sendmail_path = /usr/sbin/postfix #auth_socket_path = /var/run/dovecot/auth-master mail_plugins = cmusieve #global_script_path = /etc/sieve.conf } plugin { sieve = /etc/sieve.conf sieve = /home/vmail/domain.ru/user(a)domain.ru/.dovecot.sieve/ My sieve.conf cat sieve.conf /require "fileinto"; if header :contains "Subject" "***SPAM***" { fileinto ".Spam"; stop; My .dovecot.sieve require "fileinto"; if header :contains "Subject" "***SPAM***" { fileinto ".Spam"; stop; }/ The mail server is already installed and work, but the sieve plugins is not work((( /ls /usr/lib/dovecot/lda lib01_acl_plugin.so lib11_trash_plugin.so lib20_expire_plugin.so lib20_mail_log_plugin.so lib90_cmusieve_plugin.la lib10_quota_plugin.so lib20_convert_plugin.so lib20_fts_plugin.so lib21_fts_squat_plugin.so lib90_cmusieve_plugin.so/ All messages marked as ***SPAM*** , get to a folder the .INBOX... _Sieve plugins is not working._ In what there can be a problem? Thanks for All!!! Best Regards Andrey...

4 5

[Dovecot] How can I skip EOH in headers?
by Konstantin Lepa 13 Apr '09

13 Apr '09

Hello. My callback for get_stream is called from mail_get_stream in mail_storage_copy. Further, mbox_save_init is called and it called mbox_save_get_input_stream. backtrace (A) for my callback: #0 i_stream_header_filter_read (stream=0x8147e00) at istream-header- filter.c:314 #1 0x080cdbaa in parse_header (mstream=0x8147e00) at istream-header- filter.c:356 #2 0x080cdc0c in i_stream_header_filter_seek (stream=0x8147e00, v_offset=0, mark=false) at istream-header-filter.c:370 #3 0x080dea5b in i_stream_seek (stream=0x8147e28, v_offset=0) at istream.c:155 #4 0x080dfeae in i_stream_create_concat (input=0x8115528) at istream- concat.c:260 #5 0x2831e03d in rarules_get_stream (mail=0x8141860, hdr_size=0x0, body_size=0x0, stream_r=0xbfbfe920) at /home/pitman/work/rarules/src/rarules-plugin.c:181 #6 0x0809d830 in mail_get_stream (mail=0x8141860, hdr_size=0x0, body_size=0x0, stream_r=0xbfbfe920) at mail.c:149 #7 0x0809d936 in mail_storage_copy (t=0x811b980, mail=0x8141860, flags=0, keywords=0x0, dest_mail=0x0) at mail-copy.c:17 #8 0x080a0632 in mailbox_copy (t=0x811b980, mail=0x8141860, flags=0, keywords=0x0, dest_mail=0x0) at mail-storage.c:763 #9 0x0805960d in deliver_save (namespaces=0x8121c68, storage_r=0xbfbfeb94, mailbox=0x811a360 "Draft", mail=0x8141860, flags=0, keywords=0x0) at deliver.c:234 ... backtrace (B) for mbox_save_init: #0 mbox_save_get_input_stream (ctx=0x811bb00, input=0x814c728) at mbox-save.c:384 #1 0x08078165 in mbox_save_init (_t=0x811b980, flags=0, keywords=0x0, received_date=1239634649, timezone_offset=0, from_envelope=0x80f292b "MAILER-DAEMON", input=0x814c728, dest_mail=0x8150060, ctx_r=0xbfbfe924) at mbox-save.c:512 #2 0x080a0560 in mailbox_save_init (t=0x811b980, flags=0, keywords=0x0, received_date=-1, timezone_offset=0, from_envelope=0x80f292b "MAILER-DAEMON", input=0x814c728, dest_mail=0x0, ctx_r=0xbfbfe924) at mail-storage.c:728 #3 0x0809d9c4 in mail_storage_copy (t=0x811b980, mail=0x8141860, flags=0, keywords=0x0, dest_mail=0x0) at mail-copy.c:28 #4 0x080a0632 in mailbox_copy (t=0x811b980, mail=0x8141860, flags=0, keywords=0x0, dest_mail=0x0) at mail-storage.c:763 #5 0x0805960d in deliver_save (namespaces=0x8121c68, storage_r=0xbfbfeb94, mailbox=0x80f2af9 "INBOX", mail=0x8141860, flags=0, keywords=0x0) at deliver.c:234 ... The function mbox_save_get_input_stream calls i_stream_create_header_filter and i_stream_create_concat. The function read_header (istream-header-filter.c) finds EOH for call of i_stream_seek (backtrace B), but my calls of i_stream_create_header_filter and i_stream_create_concat doesn't find EOH (backtrace A). Why? I don't understand. I want to call rarules_header_callback for disable eoh so as to add my headers. Thanks P.S. Sorry for my bad english :-( It's mbox_save_get_input_stream: -------------------------------------------- mbox_save_get_input_stream(struct mbox_save_context *ctx, struct istream *input) { struct istream *filter, *ret, *cache_input, *streams[3]; /* filter out unwanted headers and keep track of headers' MD5 sum */ filter = i_stream_create_header_filter(input, HEADER_FILTER_EXCLUDE | HEADER_FILTER_NO_CR | HEADER_FILTER_ADD_MISSING_EOH, mbox_save_drop_headers, mbox_save_drop_headers_count, save_header_callback, ctx); if ((ctx->mbox->storage->storage.flags & MAIL_STORAGE_FLAG_KEEP_HEADER_MD5) != 0) { /* we're using MD5 sums to generate POP3 UIDLs. clients don't like it much if there are duplicates, so make sure that there can't be any by appending our own X-Delivery-ID header. */ const char *hdr; T_BEGIN { mbox_save_x_delivery_id(ctx); } T_END; hdr = ctx->x_delivery_id_header; streams[0] = i_stream_create_from_data(hdr, strlen(hdr)); streams[1] = filter; streams[2] = NULL; ret = i_stream_create_concat(streams); i_stream_unref(&filter); filter = ret; } It's my callback for get_stream: --------------------------------------- static int rarules_get_stream(struct mail *mail, struct message_size *hdr_size, struct message_size *body_size, struct istream **stream_r) { struct rarules_mail *m = RA_MAIL_CONTEXT(mail); struct istream *stream = NULL; unsigned int deleted = 0; unsigned int added = 0; int ret = 0; DPRINT("func is called"); /* standard get_stream for mail storage format */ ret = m->module_ctx.super.get_stream(mail, hdr_size, body_size, &stream); added = array_count(&m->add_hdr_arr); deleted = array_count(&m->del_hdr_arr); if (m->enabled && (deleted || added)) { struct istream * tmp = NULL; struct istream **chunks = NULL; const unsigned int max_chunks = 4; unsigned int i = 0; unsigned int cnt = 0; DPRINT("modifing message headers"); /* create message from chunks with new hdrs and skip excluded hdrs */ chunks = t_new(struct istream *, max_chunks); chunks[cnt++] = make_header_filter(stream, &m->del_hdr_arr, deleted); if (added) { chunks[cnt++] = create_header_stream(&m->add_hdr_arr, added); } chunks[cnt++] = create_stream_for_msgbody(stream); chunks[cnt++] = NULL; tmp = i_stream_create_concat(chunks); for (i = 0; chunks[i] != NULL; ++i) { i_stream_unref(&chunks[i]); } i_stream_unref(&stream); stream = tmp; } *stream_r = stream; DPRINT("func is executed"); return ret; }

1 0

[Dovecot] Issue with converting users from cyrus user.domain.com
by Preston Lord 13 Apr '09

13 Apr '09

I have been using cyrus for the past 6 years, but it is a pain to config. So ... I just installed dovecot 1.0.15 on a new debian 4r7 box. In cyrus we were using usernames as "user.domain.tld". In dovecot I realize I can: auth_username_translation = .@ But this converts user.domain.tld to user@domain@tld which of course fails. Is there a regex or something I can apply to pull this off? I also tried auth_username_format = %n.%d But I think this does the opposite of what I need. Any help is GREATLY appreciated. Thanks in advance P.

4 7

[Dovecot] deliver vs lda
by Timo Sirainen 12 Apr '09

12 Apr '09

deliver is the binary name. but it's configured inside protocol lda {} section. This is getting annoying, any thoughts on what would be a good unifying name? a) deliver binary, protocol deliver {} b) lda binary, protocol lda {} c) dovecot-lda binary, protocol lda {} d) mda binary, protocol mda {} e) dovecot-mda binary, protocol mda {} f) something else? In any case protocol lda {} would work for a while longer for backwards compatibility. c) and e) choices also makes me think if e.g. imap and imap-login should be called dovecot-imap and dovecot-imap-login instead. People have had trouble finding them since ps|grep dovecot doesn't find them..

14 39

[Dovecot] Dovecot crash
by Wolfgang.Friebel＠desy.de 12 Apr '09

12 Apr '09

tonight dovecot 1.2beta4 crashed for the first time with an assertion failed. See attached log file. Around that time nobody was logged in into dovecot and there were no other mail deliveries going on. The mail in question should have been delivered to three recipients, one got the mail, the two others not. There is unfortunately no core dump file. I have attached the dovecot -n output as well. -- Wolfgang Friebel Deutsches Elektronen-Synchrotron DESY Phone/Fax: +49 33762 77372/216 Platanenallee 6 Mail: Wolfgang.Friebel AT desy.de D-15738 Zeuthen Germany

3 2

[Dovecot] allow_nets and deliver + userdb lookup
by Pavel Volkovitskiy 12 Apr '09

12 Apr '09

Hello! I'm trying to restrict imap logins to our internal network for several users, but this breaks dovecot delivery too Even if i set allow_nets to NULL or 0.0.0.0/0 deliver exits with "Error: Auth lookup returned failure" i'm running it as 'command = /usr/lib/dovecot/deliver -e -d "$local_part@$domain" -s' in exim.conf i guess delivery lookups should be independent of allow_nets i'm using dovecot 1.1.13 + managesieve patch -- Pavel

2 1

[Dovecot] SSL
by Mohsen Pahlevanzadeh 11 Apr '09

11 Apr '09

Dear all, I have installed qmail+dovecot+vpopmail. It work, Now i wanna enable ssl on dovecot.i have pop3 & imap on my dovocet.When i was reading SSL on wiki, i saw config of that i define 4 CA file(2cert+2key), But i see 2 ca file in mkcert.(1cert+1key) It seem wiki or mkcert.sh is incomplete. Please help me..... Yours, Mohsen

2 2