- dovecot - dovecot.org

[Dovecot] Handling ENFILE
by Mike Abbott 14 Apr '09

14 Apr '09

My mail server consumed all of its configured file table slots and started returning ENFILE ("Too many open files in system") for many operations. This wreaked havoc with dovecot-1.1.13. Here are some areas where dovecot should detect and more gracefully handle ENFILE error returns: 1. Deliver should report an error more helpful than "Unknown internal error" when fdatasync_path fails: deliver(user): Apr 10 14:53:20 Error: fdatasync_path(/Volumes/MailData/ user/new) failed: Too many open files in system deliver(user): Apr 10 14:53:20 Info: msgid=<...>: save failed to INBOX: BUG: Unknown internal error 2. Index files should not be fscked when they could not be opened due to ENFILE: Error: IMAP(user): open() failed with index file /Volumes/MailData/ user/.Deleted Messages/dovecot.index: Too many open files in system Warning: IMAP(user): fscking index file /Volumes/MailData/ user/.Deleted Messages/dovecot.index 3. Dovecot should not crash when mail transaction logs become corrupt, or appear to have become corrupt due to ENFILE. Here is a sequence of error messages from the log but I can't tell whether the IMAP ones up through the panic in the middle are all from the same process, or from different processes. deliver(user): Apr 10 14:55:32 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 14:55:54 Error: open() failed with index file / Volumes/MailData/user/dovecot.index: Too many open files in system deliver(user): Apr 10 14:55:54 Error: open() failed with file /Volumes/ MailData/user/dovecot.index.log.2: Too many open files in system deliver(user): Apr 10 14:55:57 Error: open() failed with index cache file /Volumes/MailData/user/dovecot.index.cache: Too many open files in system dovecot: Apr 10 14:56:08 Warning: IMAP(user): fscking index file / Volumes/MailData/user/dovecot.index dovecot: Apr 10 14:56:16 Error: IMAP(user): Fixed index file /Volumes/ MailData/user/dovecot.index: log_file_seq 15 -> 2 deliver(user): Apr 10 14:56:19 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 14:56:20 Error: Corrupted transaction log file / Volumes/MailData/user/dovecot.index.log seq 2: record size too small (type=0x0, offset=23088, size=0) (sync_offset=36848) deliver(user): Apr 10 14:56:20 Warning: fscking index file (in-memory index) deliver(user): Apr 10 14:56:21 Info: msgid=<...>: save failed to INBOX: Internal error occurred. Refer to server log for more information. [2009-04-10 14:56:20] dovecot: Apr 10 14:56:40 Panic: IMAP(user): file mail-transaction-log- view.c: line 103 (mail_transaction_log_view_set): assertion failed: (min_file_seq <= max_file_seq) dovecot: Apr 10 14:56:40 Error: IMAP(user): Raw backtrace: [see below for better backtrace] deliver(user): Apr 10 14:56:45 Error: Transaction log file /Volumes/ MailData/user/dovecot.index.log: marked corrupted deliver(user): Apr 10 14:57:04 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 14:57:23 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 14:57:49 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 14:58:21 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 14:58:41 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 14:59:09 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 14:59:28 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 14:59:43 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 15:00:57 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 15:03:01 Info: msgid=<...>: saved mail to INBOX deliver(user): Apr 10 15:03:27 Info: msgid=<...>: saved mail to INBOX dovecot: Apr 10 15:21:41 Error: IMAP(user): Corrupted transaction log file /Volumes/MailData/user/dovecot.index.log seq 2: Invalid transaction log size (36848 vs 23088): /Volumes/MailData/user/ dovecot.index.log (sync_offset=36848) dovecot: Apr 10 15:21:42 Info: imap-login: Login: user=<user>, method=CRAM-MD5, rip=..., lip=..., TLS The backtrace for the assertion failure was: 0 libSystem.B.dylib 0x00007fff8007c102 __kill + 10 1 libSystem.B.dylib 0x00007fff800f7ffc abort + 83 2 imap 0x000000010006a22e default_fatal_finish + 70 3 imap 0x000000010006a27c i_syslog_fatal_handler + 0 4 imap 0x00000001000699db i_info + 0 5 imap 0x0000000100058542 mail_transaction_log_view_set + 104 6 imap 0x0000000100053b4c view_sync_set_log_view_range + 125 7 imap 0x0000000100053fc5 mail_index_view_sync_begin + 378 8 imap 0x000000010003c26e index_mailbox_sync_init + 129 9 imap 0x0000000100018457 maildir_storage_sync_init + 229 10 imap 0x000000010000c6bb imap_sync_init + 132 11 imap 0x000000010000c90d cmd_sync_delayed + 477 12 imap 0x000000010000716a client_handle_input + 409 13 imap 0x0000000100007244 client_input + 148 14 imap 0x0000000100071228 io_loop_handler_run + 272 15 imap 0x00000001000705ed io_loop_run + 55 16 imap 0x000000010000e219 main + 1635 17 imap 0x00000001000016c0 start + 52 I have since configured more open files, and enabled core dumps and added the pid to the mail_log_prefix for better tracking, but this is all I have to work with for now. Too bad the values of min_file_seq and max_file_seq are not logged in the panic.

2 1

[Dovecot] Dovecot+Sieve
by Andrey Garkin 13 Apr '09

13 Apr '09

Hi All!!! I'm configure mail server with Postix+Mysql+Dovecot. How can I'm using to work sieve plugins? My dovecot.conf /protocol lda { mail_plugin_dir = /usr/lib/dovecot/lda #sendmail_path = /usr/sbin/postfix #auth_socket_path = /var/run/dovecot/auth-master mail_plugins = cmusieve #global_script_path = /etc/sieve.conf } plugin { sieve = /etc/sieve.conf sieve = /home/vmail/domain.ru/user(a)domain.ru/.dovecot.sieve/ My sieve.conf cat sieve.conf /require "fileinto"; if header :contains "Subject" "***SPAM***" { fileinto ".Spam"; stop; My .dovecot.sieve require "fileinto"; if header :contains "Subject" "***SPAM***" { fileinto ".Spam"; stop; }/ The mail server is already installed and work, but the sieve plugins is not work((( /ls /usr/lib/dovecot/lda lib01_acl_plugin.so lib11_trash_plugin.so lib20_expire_plugin.so lib20_mail_log_plugin.so lib90_cmusieve_plugin.la lib10_quota_plugin.so lib20_convert_plugin.so lib20_fts_plugin.so lib21_fts_squat_plugin.so lib90_cmusieve_plugin.so/ All messages marked as ***SPAM*** , get to a folder the .INBOX... _Sieve plugins is not working._ In what there can be a problem? Thanks for All!!! Best Regards Andrey...

4 5

[Dovecot] How can I skip EOH in headers?
by Konstantin Lepa 13 Apr '09

13 Apr '09

Hello. My callback for get_stream is called from mail_get_stream in mail_storage_copy. Further, mbox_save_init is called and it called mbox_save_get_input_stream. backtrace (A) for my callback: #0 i_stream_header_filter_read (stream=0x8147e00) at istream-header- filter.c:314 #1 0x080cdbaa in parse_header (mstream=0x8147e00) at istream-header- filter.c:356 #2 0x080cdc0c in i_stream_header_filter_seek (stream=0x8147e00, v_offset=0, mark=false) at istream-header-filter.c:370 #3 0x080dea5b in i_stream_seek (stream=0x8147e28, v_offset=0) at istream.c:155 #4 0x080dfeae in i_stream_create_concat (input=0x8115528) at istream- concat.c:260 #5 0x2831e03d in rarules_get_stream (mail=0x8141860, hdr_size=0x0, body_size=0x0, stream_r=0xbfbfe920) at /home/pitman/work/rarules/src/rarules-plugin.c:181 #6 0x0809d830 in mail_get_stream (mail=0x8141860, hdr_size=0x0, body_size=0x0, stream_r=0xbfbfe920) at mail.c:149 #7 0x0809d936 in mail_storage_copy (t=0x811b980, mail=0x8141860, flags=0, keywords=0x0, dest_mail=0x0) at mail-copy.c:17 #8 0x080a0632 in mailbox_copy (t=0x811b980, mail=0x8141860, flags=0, keywords=0x0, dest_mail=0x0) at mail-storage.c:763 #9 0x0805960d in deliver_save (namespaces=0x8121c68, storage_r=0xbfbfeb94, mailbox=0x811a360 "Draft", mail=0x8141860, flags=0, keywords=0x0) at deliver.c:234 ... backtrace (B) for mbox_save_init: #0 mbox_save_get_input_stream (ctx=0x811bb00, input=0x814c728) at mbox-save.c:384 #1 0x08078165 in mbox_save_init (_t=0x811b980, flags=0, keywords=0x0, received_date=1239634649, timezone_offset=0, from_envelope=0x80f292b "MAILER-DAEMON", input=0x814c728, dest_mail=0x8150060, ctx_r=0xbfbfe924) at mbox-save.c:512 #2 0x080a0560 in mailbox_save_init (t=0x811b980, flags=0, keywords=0x0, received_date=-1, timezone_offset=0, from_envelope=0x80f292b "MAILER-DAEMON", input=0x814c728, dest_mail=0x0, ctx_r=0xbfbfe924) at mail-storage.c:728 #3 0x0809d9c4 in mail_storage_copy (t=0x811b980, mail=0x8141860, flags=0, keywords=0x0, dest_mail=0x0) at mail-copy.c:28 #4 0x080a0632 in mailbox_copy (t=0x811b980, mail=0x8141860, flags=0, keywords=0x0, dest_mail=0x0) at mail-storage.c:763 #5 0x0805960d in deliver_save (namespaces=0x8121c68, storage_r=0xbfbfeb94, mailbox=0x80f2af9 "INBOX", mail=0x8141860, flags=0, keywords=0x0) at deliver.c:234 ... The function mbox_save_get_input_stream calls i_stream_create_header_filter and i_stream_create_concat. The function read_header (istream-header-filter.c) finds EOH for call of i_stream_seek (backtrace B), but my calls of i_stream_create_header_filter and i_stream_create_concat doesn't find EOH (backtrace A). Why? I don't understand. I want to call rarules_header_callback for disable eoh so as to add my headers. Thanks P.S. Sorry for my bad english :-( It's mbox_save_get_input_stream: -------------------------------------------- mbox_save_get_input_stream(struct mbox_save_context *ctx, struct istream *input) { struct istream *filter, *ret, *cache_input, *streams[3]; /* filter out unwanted headers and keep track of headers' MD5 sum */ filter = i_stream_create_header_filter(input, HEADER_FILTER_EXCLUDE | HEADER_FILTER_NO_CR | HEADER_FILTER_ADD_MISSING_EOH, mbox_save_drop_headers, mbox_save_drop_headers_count, save_header_callback, ctx); if ((ctx->mbox->storage->storage.flags & MAIL_STORAGE_FLAG_KEEP_HEADER_MD5) != 0) { /* we're using MD5 sums to generate POP3 UIDLs. clients don't like it much if there are duplicates, so make sure that there can't be any by appending our own X-Delivery-ID header. */ const char *hdr; T_BEGIN { mbox_save_x_delivery_id(ctx); } T_END; hdr = ctx->x_delivery_id_header; streams[0] = i_stream_create_from_data(hdr, strlen(hdr)); streams[1] = filter; streams[2] = NULL; ret = i_stream_create_concat(streams); i_stream_unref(&filter); filter = ret; } It's my callback for get_stream: --------------------------------------- static int rarules_get_stream(struct mail *mail, struct message_size *hdr_size, struct message_size *body_size, struct istream **stream_r) { struct rarules_mail *m = RA_MAIL_CONTEXT(mail); struct istream *stream = NULL; unsigned int deleted = 0; unsigned int added = 0; int ret = 0; DPRINT("func is called"); /* standard get_stream for mail storage format */ ret = m->module_ctx.super.get_stream(mail, hdr_size, body_size, &stream); added = array_count(&m->add_hdr_arr); deleted = array_count(&m->del_hdr_arr); if (m->enabled && (deleted || added)) { struct istream * tmp = NULL; struct istream **chunks = NULL; const unsigned int max_chunks = 4; unsigned int i = 0; unsigned int cnt = 0; DPRINT("modifing message headers"); /* create message from chunks with new hdrs and skip excluded hdrs */ chunks = t_new(struct istream *, max_chunks); chunks[cnt++] = make_header_filter(stream, &m->del_hdr_arr, deleted); if (added) { chunks[cnt++] = create_header_stream(&m->add_hdr_arr, added); } chunks[cnt++] = create_stream_for_msgbody(stream); chunks[cnt++] = NULL; tmp = i_stream_create_concat(chunks); for (i = 0; chunks[i] != NULL; ++i) { i_stream_unref(&chunks[i]); } i_stream_unref(&stream); stream = tmp; } *stream_r = stream; DPRINT("func is executed"); return ret; }

1 0

[Dovecot] Issue with converting users from cyrus user.domain.com
by Preston Lord 13 Apr '09

13 Apr '09

I have been using cyrus for the past 6 years, but it is a pain to config. So ... I just installed dovecot 1.0.15 on a new debian 4r7 box. In cyrus we were using usernames as "user.domain.tld". In dovecot I realize I can: auth_username_translation = .@ But this converts user.domain.tld to user@domain@tld which of course fails. Is there a regex or something I can apply to pull this off? I also tried auth_username_format = %n.%d But I think this does the opposite of what I need. Any help is GREATLY appreciated. Thanks in advance P.

4 7

[Dovecot] deliver vs lda
by Timo Sirainen 12 Apr '09

12 Apr '09

deliver is the binary name. but it's configured inside protocol lda {} section. This is getting annoying, any thoughts on what would be a good unifying name? a) deliver binary, protocol deliver {} b) lda binary, protocol lda {} c) dovecot-lda binary, protocol lda {} d) mda binary, protocol mda {} e) dovecot-mda binary, protocol mda {} f) something else? In any case protocol lda {} would work for a while longer for backwards compatibility. c) and e) choices also makes me think if e.g. imap and imap-login should be called dovecot-imap and dovecot-imap-login instead. People have had trouble finding them since ps|grep dovecot doesn't find them..

14 39

[Dovecot] Dovecot crash
by Wolfgang.Friebel＠desy.de 12 Apr '09

12 Apr '09

tonight dovecot 1.2beta4 crashed for the first time with an assertion failed. See attached log file. Around that time nobody was logged in into dovecot and there were no other mail deliveries going on. The mail in question should have been delivered to three recipients, one got the mail, the two others not. There is unfortunately no core dump file. I have attached the dovecot -n output as well. -- Wolfgang Friebel Deutsches Elektronen-Synchrotron DESY Phone/Fax: +49 33762 77372/216 Platanenallee 6 Mail: Wolfgang.Friebel AT desy.de D-15738 Zeuthen Germany

3 2

[Dovecot] allow_nets and deliver + userdb lookup
by Pavel Volkovitskiy 12 Apr '09

12 Apr '09

Hello! I'm trying to restrict imap logins to our internal network for several users, but this breaks dovecot delivery too Even if i set allow_nets to NULL or 0.0.0.0/0 deliver exits with "Error: Auth lookup returned failure" i'm running it as 'command = /usr/lib/dovecot/deliver -e -d "$local_part@$domain" -s' in exim.conf i guess delivery lookups should be independent of allow_nets i'm using dovecot 1.1.13 + managesieve patch -- Pavel

2 1

[Dovecot] SSL
by Mohsen Pahlevanzadeh 11 Apr '09

11 Apr '09

Dear all, I have installed qmail+dovecot+vpopmail. It work, Now i wanna enable ssl on dovecot.i have pop3 & imap on my dovocet.When i was reading SSL on wiki, i saw config of that i define 4 CA file(2cert+2key), But i see 2 ca file in mkcert.(1cert+1key) It seem wiki or mkcert.sh is incomplete. Please help me..... Yours, Mohsen

2 2

[Dovecot] outlook problem if TLS mailsettings are off
by Georg Käfer 10 Apr '09

10 Apr '09

Dear List, I dont know if its a dovecot config problem or whatever else issue. Behaviour: If I set /etc/dovecot/dovecot.conf: Following is valid with IMAP mailboxes only: disable_plaintext_auth = yes # outlook 2007 with TLS in/out enabled can show index of mails and klick on mail does preview / doubleklick opens mail PERFECT AS INTENDED disable_plaintext_auth = no # outlook 2007 with TLS/SSL OFF can show index of mails, but (double)-/klick on mail gives me "sandwatch"-mouspointer and I've to restart outlook again (and enable TLS again to get it work again) with POP3 mailbox both variants are working smoothly. Other ssl/tls settings: ssl_disable = no login_process_per_connection = yes using or not using following makes no difference... imap_client_workarounds(default): outlook-idle delay-newmail imap_client_workarounds(imap): outlook-idle delay-newmail pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh I'm sad - but I can't force SSL/TLS for my users - so I can't "solve" this by using "disable_plaintext_auth = yes" Is there any way out of the dark? my data: gentoo linux dovecot 1.1.7 postfix 2.5.5 postfixadmin-2.2.1.1 mysql 5.0.70-r1 php 5.2.8-pl12 dovecot -n # 1.1.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.26-gentoo-r3 i686 Gentoo Base System release 1.12.11.1 ext3 protocols: pop3 pop3s imap imaps ssl_cert_file: /etc/ssl/dovecot/server.pem ssl_key_file: /etc/ssl/dovecot/server.key login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login mail_max_userip_connections(default): 10 mail_max_userip_connections(imap): 10 mail_max_userip_connections(pop3): 30 first_valid_uid: 8 last_valid_uid: 8 first_valid_gid: 12 last_valid_gid: 12 mail_privileged_group: mail mail_uid: mail mail_gid: mail mail_location: maildir:/var/mail/%d/%n/Maildir/:INDEX=/var/mail/%d/%n/indexes mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(pop3): /usr/lib/dovecot/pop3 imap_client_workarounds(default): outlook-idle delay-newmail imap_client_workarounds(imap): outlook-idle delay-newmail imap_client_workarounds(pop3): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh auth default: mechanisms: plain login passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: prefetch userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: mail group: mail regards, Georg Kaefer

2 2

[Dovecot] Postfix + Dovecot + Sieve + SpamAssassin
by Sascha Scandella 10 Apr '09

10 Apr '09

Hello Until now I used Postfix in combination with Procmail and Dovecot. I am convinced that the Dovecot Sieve plugin is a much more elegant solution. So I changed to Postfix + Dovcecot Deliver + Dovecot + Sieve. If I understood correctly I cannot call other applications within a Sieve script. Until now I used Procmail to filter Spam Messages with SpamAssassin. Is there a possibility to filter spam messages before/after Dovecot deliver. I want to use the users .spammassassin user_prefs located in their home folder. I suppose if I use a content filter in Postfix this is to early, since the user is not known at that moment. Or am I wrong? Any help is highly appreciated. Best regards, Sascha

3 2