August 2014 - dovecot

Panic at virtual folder fts-search-serialize.c
by Janette Kelm 27 Aug '14

27 Aug '14

Hi all, I'm using dovecot 2.2.13 with virtual folders and fts-solr (see attached dovecot-n.conf). At using virtual folder with fts, the dovecot crashes with following panic: Aug 20 10:49:43 imap(...): Panic: file fts-search-serialize.c: line 63 (fts_search_deserialize_add_idx): assertion failed: (*idx < buf->used) Aug 20 10:49:43 imap(test4(a)india.com): Error: Raw backtrace: /opt/dovecot2/lib/dovecot/libdovecot.so.0(+0x6c6ca) [0x7f93de51c6ca] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(+0x6c746) [0x7f93de51c746] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(+0x2542a) [0x7f93de4d542a] -> /opt/dovecot2/modules/lib20_fts_plugin.so(+0xc757) [0x7f93dd4e1757] -> /opt/dovecot2/modules/lib20_fts_plugin.so(fts_search_deserialize_add_nonmatches+0x1c) [0x7f93dd4e177c] -> /opt/dovecot2/modules/lib20_fts_plugin.so(+0xdafe) [0x7f93dd4e2afe] -> /opt/dovecot2/modules/lib20_fts_plugin.so(+0xdcd2) [0x7f93dd4e2cd2] -> /opt/dovecot2/lib/dovecot/libdovecot-storage.so.0(+0xa96da) [0x7f93de82b6da] -> /opt/dovecot2/lib/dovecot/libdovecot-storage.so.0(index_storage_search_next_nonblock+0x48) [0x7f93de82bd28] -> /opt/dovecot2/modules/lib20_virtual_plugin.so(virtual_search_next_nonblock+0x13e) [0x7f93dd2cd73e] -> /opt/dovecot2/modules/lib20_virtual_plugin.so(virtual_search_next_nonblock+0x7e) [0x7f93dd2cd67e] -> /opt/dovecot2/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonblock+0x27) [0x7f93de803f17] -> dovecot/imap() [0x41dddf] -> dovecot/imap() [0x41e462] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0xbb) [0x7f93de52d23b] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x61) [0x7f93de52df81] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7f93de52ce89] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f93de52cf18] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f93de4d9f33] -> dovecot/imap(main+0x2a8) [0x421278] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7f93de145cad] -> dovecot/imap() [0x40bda9] Aug 20 10:49:43 imap(...): Fatal: master: service(imap): child 7254 killed with signal 6 (core dumps disabled) imap-commands: telnet localhost imap ... 3 select virtual/all 4 search text searchtext - > panic crash (see above) So I do nothing special, I just do fts in the virtual folder. At google I found the patch http://hg.dovecot.org/dovecot-2.2/rev/d63b209737be, but this is already included in dovecot2.2.13, so it is not the reason. Regards, Janette

1 1

LMTP and passdb deny=yes not working
by Jogi Hofmüller 27 Aug '14

27 Aug '14

Hi all, I start this on in a new thread. I setup the deny-user setting to temporarily prevent users from logging in and (that would be great) also stop mail delivery for said user. All this is because of our upcoming migration. Now it seems that lmtp is ignoring this setting. This is from the logfile: Aug 21 13:01:00 klee dovecot: lmtp(pid): Connect from IP Aug 21 13:01:00 klee dovecot: auth: passwd-file(fischer,IP): User found from deny passdb Aug 21 13:01:00 klee dovecot: lmtp(pid, fischer): 2UErAGzR9VMBPQAAGyzfLQ: sieve: msgid=unspecified: stored mail into mailbox 'INBOX' Aug 21 13:01:00 klee dovecot: lmtp(pid): Disconnect from 172.16.16.205: Successful quit So lmtp seems to find the user and correctly states the fact but then stores the email anyhow. Is this intended or did I hit a bug? Or do I have an error in my config? Cheers, -- j.hofmüller mur.sat -- a space art project http://sat.mur.at/

3 6

multiple IMAP sessions when connecting from one client
by Martin Vegter 27 Aug '14

27 Aug '14

Hello, I am using Postfix/Dovecot/IMAP. Everything is working fine, but I have noticed that every time I connect with Thunderbird to my server via IMAP, not one but 4 connections are being logged into /var/log/mail/mail.log: 2014-08-27 09:17:46 dovecot: imap-login: Login: user=<martin>, method=PLAIN, rip=11.22.33.44, lip=55.66.77.88, mpid=12519, TLS, session=<uyb/N5cB+gBUDFRB> 2014-08-27 09:17:53 dovecot: imap-login: Login: user=<martin>, method=PLAIN, rip=11.22.33.44, lip=55.66.77.88, mpid=12521, TLS, session=<jioqN5cB/ABUDFRB> 2014-08-27 09:17:53 dovecot: imap-login: Login: user=<martin>, method=PLAIN, rip=11.22.33.44, lip=55.66.77.88, mpid=12523, TLS, session=<pqwwN5cB/QBUDFRB> 2014-08-27 09:17:54 dovecot: imap-login: Login: user=<martin>, method=PLAIN, rip=11.22.33.44, lip=55.66.77.88, mpid=12525, TLS, session=<cffremmc/gBUDFRB> This looks like 4 separate sessions are created when I log in from one client. Is this normal behavior? Can somebody please explain why this is so? My Dovecot version is 2.1.7 regards, Martin

5 5

Help with transaction log and index file corruption.
by CLMDRIVE 27 Aug '14

27 Aug '14

I am using dovecot 2.2.13 on a VPS. One (only one!) of the users is experiencing IMAP disconnections. The maillog file is full of errors about corrupted transaction log and index files. The configurations follows: # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-028stab107.1 x86_64 CentOS release 6.4 (Final) auth_cache_size = 1 k auth_mechanisms = plain login auth_username_chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!#$-=?^_{}~./@+%" disable_plaintext_auth = no namespace { inbox = yes location = prefix = INBOX. type = private } passdb { args = /usr/local/cpanel/bin/dovecot-wrap driver = checkpassword } plugin { acl = vfile quota = maildir quota_rule = INBOX.Trash:ignore } protocols = imap service auth { unix_listener auth-client { mode = 0666 } } service imap-login { client_limit = 500 inet_listener imap { address = * } inet_listener imaps { address = * } process_limit = 50 process_min_avail = 5 service_count = 0 vsz_limit = 64 M } service imap { process_limit = 512 vsz_limit = 256 M } service managesieve-login { client_limit = 500 process_limit = 50 process_min_avail = 5 service_count = 0 vsz_limit = 64 M } service managesieve { process_limit = 512 vsz_limit = 256 M } service pop3-login { client_limit = 500 inet_listener pop3 { address = * } inet_listener pop3s { address = * } process_limit = 50 process_min_avail = 5 service_count = 0 vsz_limit = 64 M } service pop3 { process_limit = 512 vsz_limit = 256 M } ssl_cert = </etc/dovecot/ssl/dovecot.crt ssl_cipher_list = ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:-LOW:-SSLv2:-EXP ssl_key = </etc/dovecot/ssl/dovecot.key userdb { driver = prefetch } userdb { args = /usr/local/cpanel/bin/dovecot-wrap driver = checkpassword } protocol imap { imap_capability = +NAMESPACE imap_idle_notify_interval = 24 mins imap_logout_format = in=%i, out=%o, bytes=%i/%o mail_max_userip_connections = 30 mail_plugins = acl quota imap_quota } protocol pop3 { mail_max_userip_connections = 3 mail_plugins = quota pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s, bytes=%i/%o pop3_uidl_format = UID%u-%v } protocol lda { postmaster_address = postmaster(a)example.com } Can anyone help me on this??? I am about to cry! thank you! Claudio

3 2

Problem with BINARY extension (Invalid data in MIME part)
by Pavel M. 27 Aug '14

27 Aug '14

1 0

crash with "The specified message set is invalid."
by Ralf Hildebrandt 27 Aug '14

27 Aug '14

We're using dovecot as a proxy between the user and an exchange server. All over a sudden we're getting: Aug 27 13:16:07 mproxy dovecot: imap(username): Fatal: master: service(imap): child 18376 killed with signal 6 (core dumps disabled) Aug 27 13:16:09 mproxy dovecot: imap(username): Error: imapc(email.charite.de:993): Command '12 UID FETCH 0 (BODY.PEEK[])' failed with BAD: 12 The specified message set is invalid. Aug 27 13:16:09 mproxy dovecot: imap(username): Error: imapc: Mail FETCH failed: The specified message set is invalid. Aug 27 13:16:09 mproxy dovecot: imap(username): Panic: file mail-index-map.c: line 547 (mail_index_map_lookup_seq_range): assertion failed: (first_uid > 0) Aug 27 13:16:09 mproxy dovecot: imap(username): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x5f770) [0x7f76ad2d9770] -> /usr/lib/dovecot/libdovecot.so.0(+0x5f84e) [0x7f76ad2d984e] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f76ad293f2e] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xac44e) [0x7f76ad5f044e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_lookup_seq+0x12) [0x7f76ad601372] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x5ad94) [0x7f76ad59ed94] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x5af02) [0x7f76ad59ef02] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_get_hdr_stream+0x34) [0x7f76ad5adb44] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_get_header_stream+0x107) [0x7f76ad5d7247] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x5ab09) [0x7f76ad59eb09] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x5ab95) [0x7f76ad59eb95] -> /usr/lib/dovecot/modules/lib20_mail_log_plugin.so(+0x1a0c) [0x7f76ac8a9a0c] -> /usr/lib/dovecot/modules/lib20_mail_log_plugin.so(+0x2159) [0x7f76ac8aa159] -> /usr/lib/dovecot/modules/lib15_notify_plugin.so(notify_contexts_mail_copy+0x5c) [0x7f76acaad18c] -> /usr/lib/dovecot/modules/lib15_notify_plugin.so(+0x22c6) [0x7f76acaae2c6] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_copy+0x6d) [0x7f76ad5b5fdd] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_move+0x15) [0x7f76ad5b60b5] -> dovecot/imap [username 91.22.142.109 UID move]() [0x40ca95] -> dovecot/imap [username 91.22.142.109 UID move](command_exec+0x3c) [0x416cfc] -> dovecot/imap [username 91.22.142.109 UID move]() [0x415d5f] -> dovecot/imap [username 91.22.142.109 UID move]() [0x415e15] -> dovecot/imap [username 91.22.142.109 UID move](client_handle_input+0x115) [0x416115] -> dovecot/imap [username 91.22.142.109 UID move](client_input+0x72) [0x4164a2] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x3f) [0x7f76ad2e9e6f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd7) [0x7f76ad2ead77] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7f76ad2e9ed9] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f76ad2e9f58] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f76ad299133] -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein

1 0

LMTP during dsync migration
by Jogi Hofmüller 27 Aug '14

27 Aug '14

Hi all, Another question regarding migration. While migrating a mailbox with dsync is it safe to deliver mail via LMTP to the new (target) mailbox or is it wiser to deactivate LMTP delivery to this mailbox until it's fully migrated? And what methods could I use to stop delivery to a mailbox during migration? Our user data is stored on an LDAP server. Cheers, -- j.hofmüller http://thesix.mur.at/

8 18

Special-Use Mailboxes ?
by Per Jessen 27 Aug '14

27 Aug '14

Lots of people must have had this issue before me - users using different devices&software (pc, smartphone, tablet, thunderbird, outlook, webmail) and languages (e.g. English on the PC, German on the iPhone), some folders end of with different names - e.g. "Sent" and "Gesendete Elemente" - meaning the same thing, namely "sent mails". How do you map these to just one folder such that it shows up correctly independently of what the user happens to be using? It sounds like something for the mailbox "Sent" { special_use = \Drafts } config directive? Is that what it might be used for? mailbox "Sent" { special_use = \Sent } mailbox "Gesendete Elemente" { special_use = \Sent } -- Per Jessen, Zürich (12.9°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland.

5 15

Object storage
by List 26 Aug '14

26 Aug '14

Timo, We are contemplating building an S3 compatible cluster of storage servers using Pithos <https://github.com/exoscale/pithos> and Cassandra to get multi data center redundancy and availability. Would it be possible to have multiple Dovecot servers in disparate geographical locations all talking to the same object storage, assuming it's consistent between sites? From what I remember about obox the dovecot servers all keep a copy of the index locally and merge it with a file in the object storage at intervals, is this an issue with our concept? Also does your team have any performance analysis data on the obox plugin, or data on it's use in production?

1 0

Running dsync results in Panic: file mbox-lock.c: line 799 (mbox_lock)
by Piotr Paczynski 26 Aug '14

26 Aug '14

Hello, I'm converting our system to use dovecot. As part of the migration we need to convert existing mbox'es to Maildir format. I'm using latest Dovecot2 available as FreeBSD pkg - 2.2.13. I'm trying to use dsync, which results in: ./dsync -vfu ppaczyn -o mail_location=mbox:~/mail:INBOX=/var/mail/ppaczyn mirror maildir:~/Maildir dsync(ppaczyn): Panic: file mbox-lock.c: line 799 (mbox_lock): assertion failed: (lock_type == F_RDLCK || mbox->mbox_lock_type != F_RDLCK) Abort trap (core dumped) gdb ./dsync doveadm.core (gdb) bt #0 0x00000008014aae1a in kill () from /lib/libc.so.7 #1 0x00000008014a9ac9 in abort () from /lib/libc.so.7 #2 0x00000008010f0b52 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0) at failures.c:193 #3 0x00000008010f0962 in default_fatal_handler (ctx=0x7fffffffd158, format=0x800e14112 "file %s: line %d (%s): assertion failed: (%s)", args=0x7fffffffd140) at failures.c:207 #4 0x00000008010f1190 in i_panic (format=0x800e14112 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:267 #5 0x0000000800d421ea in mbox_lock (mbox=0x801ce5c40, lock_type=3, lock_id_r=0x801c24af4) at mbox-lock.c:799 #6 0x0000000800d46251 in mbox_save_init_file (ctx=0x801c1a780, t=0x801c24a00, want_mail=true) at mbox-save.c:294 #7 0x0000000800d45cc9 in mbox_save_begin (_ctx=0x801c1a780, input=0x801ca4c30) at mbox-save.c:466 #8 0x0000000800d83670 in mailbox_save_begin (ctx=0x7fffffffd330, input=0x801ca4c30) at mail-storage.c:2034 #9 0x0000000800d74f8c in mail_storage_try_copy (_ctx=0x7fffffffd330, mail=0x801c55840) at mail-copy.c:76 #10 0x0000000800d74e6c in mail_storage_copy (ctx=0x801c1a780, mail=0x801c55840) at mail-copy.c:101 #11 0x0000000800d83b4d in mailbox_copy (_ctx=0x7fffffffd3e8, mail=0x801c55840) at mail-storage.c:2149 #12 0x0000000000447eda in dsync_mailbox_save_body (importer=0x801d1f040, mail=0x801c47410, newmail=0x801d1f2b8, all_newmails_forcopy=0x7fffffffd428) at dsync-mailbox-import.c:1950 #13 0x0000000000446bbb in dsync_mailbox_save_newmails (importer=0x801d1f040, mail=0x801c47410, all_newmails=0x801d1f2b8) at dsync-mailbox-import.c:2027 #14 0x0000000000446b3d in dsync_mailbox_import_mail (importer=0x801d1f040, mail=0x801c47410) at dsync-mailbox-import.c:2069 #15 0x00000000004437c5 in dsync_brain_recv_mail (brain=0x801ca9040) at dsync-brain-mails.c:249 #16 0x0000000000442eb9 in dsync_brain_sync_mails (brain=0x801ca9040) at dsync-brain-mails.c:339 #17 0x000000000043d109 in dsync_brain_run_real (brain=0x801ca9040, changed_r=0x7fffffffd5c5) at dsync-brain.c:575 #18 0x000000000043cf0b in dsync_brain_run (brain=0x801ca9040, changed_r=0x7fffffffd5c5) at dsync-brain.c:610 #19 0x000000000043a75a in cmd_dsync_run_local (ctx=0x801c2d840, user=0x801c40040, brain=0x801ca9040, ibc2=0x801c47480, changes_during_sync_r=0x7fffffffd67a) at doveadm-dsync.c:383 #20 0x0000000000439d07 in cmd_dsync_run (_ctx=0x801c2d840, user=0x801c40040) at doveadm-dsync.c:596 #21 0x0000000000413735 in doveadm_mail_next_user (ctx=0x801c2d840, input=0x7fffffffd7d8, error_r=0x7fffffffd848) at doveadm-mail.c:314 #22 0x0000000000413541 in doveadm_mail_single_user (ctx=0x801c2d840, input=0x7fffffffd7d8, error_r=0x7fffffffd848) at doveadm-mail.c:337 #23 0x0000000000414100 in doveadm_mail_cmd (cmd=0x801c72528, argc=5, argv=0x7fffffffd9b8) at doveadm-mail.c:525 #24 0x0000000000413c75 in doveadm_mail_try_run (cmd_name=0x7fffffffdc56 "sync", argc=5, argv=0x7fffffffd998) at doveadm-mail.c:617 #25 0x00000000004219f9 in main (argc=5, argv=0x7fffffffd998) at doveadm.c:398 Here's my config: doveconf -n # 2.2.13: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 10.0-RELEASE-p7 amd64 mail_location = maildir:~/Maildir mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { driver = pam } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { driver = passwd } Any suggestions how to proceed? Piotr

1 0