- dovecot - dovecot.org

[BUG] 2.2.21 Panic: file imap-client.c: line 841 (client_check_command_hangs): assertion failed: (!have_wait_unfinished || unfinished_count > 0)
by Florian Pritz 04 Jan '16

04 Jan '16

Hi, I'm seeing the following in my logs. Twice so far, no idea what caused it. I do however have the core dump if that is of any use. > Jan 4 11:14:11 karif dovecot[20876]: imap(username): Panic: file imap-client.c: line 841 (client_check_command_hangs): assertion failed: (!have_wait_unfinished || unfinished_count > 0) > Jan 4 11:14:11 karif dovecot[20876]: imap(username): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xa64ea) [0x7f6f99fa64ea] -> /usr/lib/dovecot/libdovecot.so.0(+0xa7a18) [0x7f6f99fa7a18] -> /usr/lib/dovecot/libdovecot.so.0(i_ > fatal+0) [0x7f6f99fa686d] -> dovecot/imap() [0x41dde6] -> dovecot/imap(client_continue_pending_input+0xd6) [0x41df50] -> dovecot/imap() [0x4122a9] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0xcd) [0x7f6f99fc3b01] -> /usr/lib/dovec > ot/libdovecot.so.0(io_loop_handler_run_internal+0x209) [0x7f6f99fc606e] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x24) [0x7f6f99fc3caa] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0xaf) [0x7f6f99fc3bf6] -> /usr/lib/dovec > ot/libdovecot.so.0(master_service_run+0x2e) [0x7f6f99f317af] -> dovecot/imap(main+0x2da) [0x430da2] -> /usr/lib/libc.so.6(__libc_start_main+0xf0) [0x7f6f99b7c610] -> dovecot/imap(_start+0x29) [0x40c639] > Jan 4 11:14:11 karif dovecot[20876]: imap(username): Fatal: master: service(imap): child 19463 killed with signal 6 (core dumped) In case it is easier to read, here's a gdb backtrace: > #0 0x00007f6f99b8f5f8 in raise () from /usr/lib/libc.so.6 > #1 0x00007f6f99b90a7a in abort () from /usr/lib/libc.so.6 > #2 0x00007f6f99fa6539 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0) at failures.c:201 > #3 0x00007f6f99fa7a18 in i_internal_fatal_handler (ctx=0x7ffe660d4700, format=0x438a60 "file %s: line %d (%s): assertion failed: (%s)", args=0x7ffe660d4720) at failures.c:670 > #4 0x00007f6f99fa686d in i_panic (format=0x438a60 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:275 > #5 0x000000000041dde6 in client_check_command_hangs (client=0x2363450) at imap-client.c:841 > #6 0x000000000041df50 in client_continue_pending_input (client=0x2363450) at imap-client.c:884 > #7 0x00000000004122a9 in idle_client_input (ctx=0x23642a8) at cmd-idle.c:111 > #8 0x00007f6f99fc3b01 in io_loop_call_io (io=0x2374600) at ioloop.c:559 > #9 0x00007f6f99fc606e in io_loop_handler_run_internal (ioloop=0x232c740) at ioloop-epoll.c:220 > #10 0x00007f6f99fc3caa in io_loop_handler_run (ioloop=0x232c740) at ioloop.c:607 > #11 0x00007f6f99fc3bf6 in io_loop_run (ioloop=0x232c740) at ioloop.c:583 > #12 0x00007f6f99f317af in master_service_run (service=0x232c5e0, callback=0x430a35 <client_connected>) at master-service.c:640 > #13 0x0000000000430da2 in main (argc=1, argv=0x232c390) at main.c:442 Florian

3 3

postmaster domain
by Timothy Murphy 04 Jan '16

04 Jan '16

I recently upgraded a remote server to CentOS-7.2, after which email stopped coming through. I suspect that this was probably due to the fact that I had started a version of dovecot installed some time ago. I found messages in /var/log/maillog like Jan 4 12:56:56 alfred postfix/lmtp[7612]: 20A662202B4: to=<root(a)gayleard.eu>, relay=alfred.gayleard.eu[private/dovecot-lmtp], delay=34272, delays=34272/0.16/0.01/0.05, dsn=4.3.0, status=deferred (host alfred.gayleard.eu[private/dovecot-lmtp] said: 451 4.3.0 <root(a)gayleard.eu> Invalid user settings. Refer to server log for more information. (in reply to RCPT TO command)) Googling for this, I found the suggestion that one should insert the following postmaster_address line in /etc/dovecot/conf.d/20-lmtp protocol lmtp { postmaster_address = postmaster(a)alfred.eu # Space separated list of plugins to load (default is global ail_plugins). mail_plugins = $mail_plugins sieve } (alfred.eu being the server's domain). This seems to have solved the problem, after re-starting dovecot. I'm wondering if this was an issue that arose some time ago, and if so whether my solution was the recommended one? -- Timothy Murphy gayleard /at/ eircom.net School of Mathematics, Trinity College, Dublin

1 0

Re: Howto get information about quota used
by Макаров Денис 04 Jan '16

04 Jan '16

I tried this for correct user: sudo doveadm quota get -u user Quota name Type Value Limit % User STORAGE 0 614400 0 User MESSAGE 0 - 0 Ok, limit is correct. But why value and % = 0? Via Thunderbird I see 52%. >> Hello guys! >> >> I have Dovecot server (version: 2.2.9) with quota and imap_quota plugins. >> >> I wanna know how to get information about quota used of current email. >> >> I can do this with IMAP client like Thunderbird, but how I can do it >> on server? > Try this: > > # doveadm quota get -A > ... > > p@rick > -- Best regards, *Makarov Denis* *email*: makarovdenis11(a)gmail.com <mailto:makarovdenis11@gmail.com>

1 1

How to debug lmtpd?
by kjonca＠o2.pl 04 Jan '16

04 Jan '16

1. I have dovecot installation with lmtp 2. there are some sieve filter which call mailpost to put mails in my news server. 3. SOME mails are not mailposted :( I do not know why. Moreover these mails filtered by dovecot-lda ARE mailposted whit the same sieve filter. 4. How can I debug lmtp and it's descendant processes? strace -ff only? KJ -- http://wolnelektury.pl/wesprzyj/teraz/ Boys, you have ALL been selected to LEAVE th' PLANET in 15 minutes!!

2 1

Double folders?
by Frank W 04 Jan '16

04 Jan '16

Hi, I upgraded my dovecot install from 2.0.19 to 2.2.19. I haved update the configuration, accordingly, but I when I open roundcube, I see multiple folders like sent, trash, facebook, etc... I not sure if am missing something or not. Here is my configure: [root@mgmt ~]# cat dovecot_config.txt # 2.2.19: /etc/dovecot/dovecot.conf # OS: Linux 3.4.4 armv5tel Gentoo Base System release 2.1 ext3 auth_mechanisms = plain login disable_plaintext_auth = no first_valid_gid = 207 first_valid_uid = 207 last_valid_gid = 207 last_valid_uid = 207 listen = *, [::] mail_gid = 207 mail_location = maildir:/var/vmail/%u/Maildir/:INDEX=/var/vmail/%u/indexes mail_plugins = quota mail_uid = 207 namespace { inbox = yes location = prefix = INBOX. separator = . name = } namespace { location = mailbox { special_use = \Drafts name = Drafts } mailbox { special_use = \Junk name = Junk } mailbox { special_use = \Sent name = Sent } mailbox { special_use = \Trash name = Trash } prefix = name = inbox } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocols = imap pop3 service replication-notify-fifo { name = aggregator } service anvil-auth-penalty { name = anvil } service auth-worker { user = $default_internal_user name = auth-worker } service { unix_listener { group = postfix mode = 0666 user = postfix path = /var/spool/postfix/private/auth } unix_listener { group = mail mode = 0666 user = mail path = auth-userdb } name = auth } service config { name = config } service dict-async { name = dict-async } service dict { name = dict } service login/proxy-notify { name = director } service dns-client { name = dns_client } service doveadm-server { name = doveadm } service imap-hibernate { name = imap-hibernate } service imap { name = imap-login } service imap-urlauth { name = imap-urlauth-login } service imap-urlauth-worker { name = imap-urlauth-worker } service token-login/imap-urlauth { name = imap-urlauth } service imap-master { name = imap } service indexer-worker { name = indexer-worker } service indexer { name = indexer } service ipc { name = ipc } service lmtp { name = lmtp } service log-errors { name = log } service pop3 { name = pop3-login } service login/pop3 { name = pop3 } service replicator-doveadm { name = replicator } service login/ssl-params { name = ssl-params } service stats-mail { name = stats } ssl_cert = </etc/postfix/ssl/mail.xxxxxx.net.crt ssl_cipher_list = ALL:!LOW:!SSLv2 ssl_key = </etc/postfix/ssl/mail.xxxxxxx.net.key userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { service replication-notify-fifo { name = aggregator } service anvil-auth-penalty { name = anvil } service auth-worker { name = auth-worker } service auth-client { name = auth } service config { name = config } service dict-async { name = dict-async } service dict { name = dict } service login/proxy-notify { name = director } service dns-client { name = dns_client } service doveadm-server { name = doveadm } service imap-hibernate { name = imap-hibernate } service imap { name = imap-login } service imap-urlauth { name = imap-urlauth-login } service imap-urlauth-worker { name = imap-urlauth-worker } service token-login/imap-urlauth { name = imap-urlauth } service imap-master { name = imap } service indexer-worker { name = indexer-worker } service indexer { name = indexer } service ipc { name = ipc } service lmtp { name = lmtp } service log-errors { name = log } service pop3 { name = pop3-login } service login/pop3 { name = pop3 } service replicator-doveadm { name = replicator } service login/ssl-params { name = ssl-params } service stats-mail { name = stats } } protocol lda { service replication-notify-fifo { name = aggregator } service anvil-auth-penalty { name = anvil } service auth-worker { name = auth-worker } service auth-client { name = auth } service config { name = config } service dict-async { name = dict-async } service dict { name = dict } service login/proxy-notify { name = director } service dns-client { name = dns_client } service doveadm-server { name = doveadm } service imap-hibernate { name = imap-hibernate } service imap { name = imap-login } service imap-urlauth { name = imap-urlauth-login } service imap-urlauth-worker { name = imap-urlauth-worker } service token-login/imap-urlauth { name = imap-urlauth } service imap-master { name = imap } service indexer-worker { name = indexer-worker } service indexer { name = indexer } service ipc { name = ipc } service lmtp { name = lmtp } service log-errors { name = log } service pop3 { name = pop3-login } service login/pop3 { name = pop3 } service replicator-doveadm { name = replicator } service login/ssl-params { name = ssl-params } service stats-mail { name = stats } } protocol imap { mail_plugins = quota imap_quota service replication-notify-fifo { name = aggregator } service anvil-auth-penalty { name = anvil } service auth-worker { name = auth-worker } service auth-client { name = auth } service config { name = config } service dict-async { name = dict-async } service dict { name = dict } service login/proxy-notify { name = director } service dns-client { name = dns_client } service doveadm-server { name = doveadm } service imap-hibernate { name = imap-hibernate } service imap { name = imap-login } service imap-urlauth { name = imap-urlauth-login } service imap-urlauth-worker { name = imap-urlauth-worker } service token-login/imap-urlauth { name = imap-urlauth } service imap-master { name = imap } service indexer-worker { name = indexer-worker } service indexer { name = indexer } service ipc { name = ipc } service lmtp { name = lmtp } service log-errors { name = log } service pop3 { name = pop3-login } service login/pop3 { name = pop3 } service replicator-doveadm { name = replicator } service login/ssl-params { name = ssl-params } service stats-mail { name = stats } } protocol pop3 { service replication-notify-fifo { name = aggregator } service anvil-auth-penalty { name = anvil } service auth-worker { name = auth-worker } service auth-client { name = auth } service config { name = config } service dict-async { name = dict-async } service dict { name = dict } service login/proxy-notify { name = director } service dns-client { name = dns_client } service doveadm-server { name = doveadm } service imap-hibernate { name = imap-hibernate } service imap { name = imap-login } service imap-urlauth { name = imap-urlauth-login } service imap-urlauth-worker { name = imap-urlauth-worker } service token-login/imap-urlauth { name = imap-urlauth } service imap-master { name = imap } service indexer-worker { name = indexer-worker } service indexer { name = indexer } service ipc { name = ipc } service lmtp { name = lmtp } service log-errors { name = log } service pop3 { name = pop3-login } service login/pop3 { name = pop3 } service replicator-doveadm { name = replicator } service login/ssl-params { name = ssl-params } service stats-mail { name = stats } }

2 1

Released Pigeonhole v0.4.11.rc1 for Dovecot v2.2.21
by Stephan Bosch 04 Jan '16

04 Jan '16

Hello Dovecot users, First, a happy new year to you all! :) In the last few weeks I got quite a few bug reports. I also found a few things myself. Some of the bugs are quite annoying, so a new release is necessary. First, I release RC1. If all is well, I'll release the definitive Pigeonhole v0.4.11 within the next few days. Changelog v0.4.11: - Sieve mime extension: Fixed the header :mime :anychild test to work properly outside a foreverypart loop. - Several fixes in message body part handling: - Fixed assert failure occurring when text extraction is attempted on an empty or broken text part. - Fixed assert failure in handling of body parts that are converted to text. - Fixed header unfolding for (mime) headers parsed from any mime part. - Fixed trimming for (mime) headers parsed from any mime part. - Fixed erroneous changes to the message part tree structure performed when re-parsing the message. - LDA Sieve plugin: Fixed logging of actions; sometimes the configured log format was not followed. - LDA Sieve plugin: Fixed bug in error handling of script storage initialization. - Sieve Extprograms plugin: Ignored ENOTCONN error in shutdown(fd, SHUT_WR) call. - Fixed duplication of discard actions in the script result. Each discard was counted as a separate action, which means that action limit would be crossed too early. - Made sure that quota errors never get logged as errors in syslog. - Fixed handling of implicit keep for a partially executed transaction that yielded a temporary failure. - Fixed handling of global errors. If master and user error handler were identical, in some cases the log message could be lost. - Fixed AIX compile issue in message body parser. The release is available as follows: http://pigeonhole.dovecot.org/releases/2.2/rc/dovecot-2.2-pigeonhole-0.4.11… http://pigeonhole.dovecot.org/releases/2.2/rc/dovecot-2.2-pigeonhole-0.4.11… Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this release candidate and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan(a)rename-it.nl

1 0

Howto get information about quota used
by Макаров Денис 03 Jan '16

03 Jan '16

Hello guys! I have Dovecot server (version: 2.2.9) with quota and imap_quota plugins. I wanna know how to get information about quota used of current email. I can do this with IMAP client like Thunderbird, but how I can do it on server? doveconf -n | grep quota mail_plugins = quota, antispam quota = maildir:User quota_exceeded_message = Message quota_rule = *:storage=100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u quota_warning3 = -storage=100%% quota-warning below %u service quota-warning { executable = /usr/local/bin/quota-warning.sh mail_plugins = quota imap_quota mail_plugins = quota sieve For each mailboxes I use quota limits from MySQL. -- С уважением, *Макаров Денис* *почта*: makarovdenis11(a)gmail.com <mailto:makarovdenis11@gmail.com> Best regards, *Makarov Denis* *email*: makarovdenis11(a)gmail.com <mailto:makarovdenis11@gmail.com>

2 1

SASL binds
by Brendan Kearney 01 Jan '16

01 Jan '16

i am looking to get SASL binds working in Dovecot for userdb lookups, and i am not sure what i might be doing wrong. Dovecot version - 2.2.19 running on Fedora 22. MIT Kerberos and OpenLDAP are being used. my LDAP configs: uris = ldap://server1.bpk2.com ldap://server2.bpk2.com sasl_bind = yes sasl_mech = gssapi sasl_realm = BPK2.COM sasl_authz_id = imap/imap.bpk2.com(a)BPK2.COM base = dc=bpk2,dc=com the above results in the below error logs: Jan 01 13:56:58 mail auth[16747]: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (No Kerberos credentials available) Jan 01 13:56:58 mail dovecot[16722]: auth-worker(16747): Error: LDAP: binding failed (dn (none)): Local error, SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (No Kerberos credentials available) i am assuming the keytab, /etc/dovecot/dovecot.keytab would be used to bind to the directory, but i am not sure. the auth_krb5_keytab directive is set with the absolute path and keytab name. is there something i am missing, such as a /etc/sasl2/dovecot.conf file? in the directory, i am mapping the Kerberos ID to LDAP user object as such: uid=imap\/(.*).bpk2.com,cn=bpk2.com,cn=gssapi,cn=auth uid=mda,ou=processUsers,ou=Users,dc=bpk2,dc=com if i change the sasl_authz_id to uid=mda,ou=processUsers,ou=Users,dc=bpk2,dc=com, and restart dovecot, i still get the same error. can anyone shed light on where i am going wrong? thanks in advance, brendan

1 0

doveadm expunge no longer works
by Fabian Groffen 31 Dec '15

31 Dec '15

Heya, Just upgraded from 2.2.19 to dovecot-2.2.21 and noticed expunging mail no longer works. # doveadm expunge -u theuser MAILBOX spam all Fatal: expunge: To avoid accidents, each branch in search query must contain something else besides MAILBOX (e.g. just add "all" if you want everything) Clearly, I've already got all in there, so I tried capitalising, etc. Looking at the code, I see expunge_search_args_is_msgset_or_ok apparently is failing. I'm wondering if this function [1] is actually doing what it is supposed to do, since it uses return, instead a var to figure out whether it is ok or not. In my case, just because it finds MAILBOX it returns without looking whether there is any further arguments. Or am I missing the point here (and how should I expunge a folder of mail?) Thanks, Fabian [1] https://github.com/dovecot/core/blob/6971937a6f3e93844dbd43bdbe903628e21a94… -- Fabian Groffen Gentoo on a different level

1 1

sieve-filter to script fails with SHUT_WR
by Mark Stover 31 Dec '15

31 Dec '15

Trying to make vnd.dovecot.filter to exchange an email with my script. (I have a vnd.dovecot.pipe script working, so I'm pretty sure it's not the configuration...) I'm passing the email using stdin/stdout and _not_ using sockets. My script receives the email and processes it, but the filter process fails: lda(<username>): Error: shutdown(/usr/local/lib/dovecot-sieve/sieve-filter/CleanAttachments.py, SHUT_WR) failed: Socket is not connected lda(<username>): Error: sieve: 20-filter-incoming: line 15: filter action: failed to execute to program `CleanAttachments.py' Not sure, what I'm doing wrong... Thanks, Mark $ doveconf -n # 2.2.21 (5345f22): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.9 (357ac0a0e68b+) # OS: Darwin 15.2.0 x86_64 lda_mailbox_autocreate = yes mail_location = maildir:~/Maildir maildir_broken_filename_sizes = yes maildir_very_dirty_syncs = yes plugin { sieve = file:~/.dovecot/sieve;active=~/.dovecot/sieve/user.sieve sieve_before = /usr/local/lib/dovecot-sieve/sieve.d/ sieve_execute_bin_dir = /usr/local/lib/dovecot-sieve/sieve-execute sieve_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute +vnd.dovecot.filter +vnd.dovecot.debug sieve_filter_bin_dir = /usr/local/lib/dovecot-sieve/sieve-filter sieve_pipe_bin_dir = /usr/local/lib/dovecot-sieve/sieve-pipe sieve_plugins = sieve_extprograms sieve_user_log = ~/.dovecot/sieve/sieve.log } postmaster_address = mstover(a)cisco.com protocol lda { info_log_path = log_path = mail_plugins = " sieve" } Sieve Script: require [ "variables", "envelope", "vnd.dovecot.debug", "vnd.dovecot.filter" ]; if envelope :matches "to" "*" { set "to" "${1}"; } if envelope :matches "from" "*" { set "from" "${1}"; } debug_log "Received message TO=${to} FROM=${from}"; filter "CleanAttachments.py"; debug_log "Returned from CleanAttachments"; if envelope :matches "to" "*" { set "to" "${1}"; } if envelope :matches "from" "*" { set "from" "${1}"; } debug_log "Returned message TO=${to} FROM=${from}";

2 2

LMTP crashes on Debian with dovecot backports from rename-it.nl
by Christian Schmidt 30 Dec '15

30 Dec '15

Hello dovecot-users, we're running Debian 7.9 with dovecot from Stephan Bosch's repository: deb http://xi.rename-it.nl/debian/ oldstable-auto/dovecot-2.2 main Starting with dovecot 2.2.20-1 build 28, we suddenly discovered that some mails (to be locally delivered) "got stuck". dovecot's log revealed messages like the following one: Dec 29 09:56:50 mailhost dovecot: lmtp(MAILRECIPIENT(a)chemie.uni-hamburg.de): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x8360f) [0x7f58fce4f60f] -> /usr/lib/dovecot/libdovecot.so.0(+0x8366e) [0x7f58fce4f66e] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f58fcdf8a3e] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x3cc75) [0x7f58fd107c75] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xabeda) [0x7f58fd176eda] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_get_first_header+0xba) [0x7f58fd17723a] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_get_first_header+0x38) [0x7f58fd108cd8] -> /usr/lib/dovecot/modules/lib20_mail_log_plugin.so(+0x1b5c) [0x7f58fb16cb5c] -> /usr/lib/dovecot/modules/lib20_mail_log_plugin.so(+0x22e9) [0x7f58fb16d2e9] -> /usr/lib/dovecot/modules/lib15_notify_plugin.so(notify_contexts_mail_copy+0x46) [0x7f58fb371576] -> /usr/lib/dovecot/modules/lib15_notify_plugin.so(+0x2750) [0x7f58fb372750] -> /usr/lib/dovecot/modules/lib10_quota_plugin.so(+0xbc20) [0x7f58fb580c20] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_copy+0x7d) [0x7f58fd11305d] -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver_save+0x171) [0x7f58fd4019c1]-> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x11b) [0x7f58fd401dfb] -> dovecot/lmtp [DATA MAILRECIPIENT(a)chemie.uni-hamburg.de](+0x7263) [0x7f58fd831263] -> dovecot/lmtp [DATA MAILRECIPIENT(a)chemie.uni-hamburg.de](+0x79d9) [0x7f58fd8319d9] -> /usr/lib/dovecot /libdovecot.so.0(io_loop_call_io+0x5b) [0x7f58fce6282b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd7) [0x7f58fce63c07] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7f58fce628d5] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f58fce62a78] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f58fcdfe443] -> dovecot/lmtp [DATA MAILRECIPIENT(a)chemie.uni-hamburg.de](main+0x19c) [0x7f58fd82f8dc] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f58fca5fead] -> dovecot/lmtp [DATA MAILRECIPIENT(a)chemie.uni-hamburg.de](+0x59ad) [0x7f58fd82f9ad] Dec 29 09:56:50 mailhost dovecot: lmtp(MAILRECIPIENT(a)chemie.uni-hamburg.de): Fatal: master: service(lmtp): child 15897 killed with signal 6 (core dumps disabled) Dec 29 09:56:51 mailhost dovecot: lmtp(MAILRECIPIENT(a)chemie.uni-hamburg.de): Panic: file ../../../src/lib/array.h: line 219 (array_idx_i): assertion failed: (idx * array->element_size < array->buffer->used) All dovecot builds newer than 2.2.20-1~auto+26 show the same behaviour. Is there anything we can do? # doveconf -n # 2.2.20 (): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.10 # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.9 ext4 auth_default_realm = chemie.uni-hamburg.de auth_master_user_separator = * auth_mechanisms = plain login auth_verbose = yes login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %k mail_gid = vmail mail_location = maildir:/mail/%d/%n/Maildir mail_plugins = acl mail_log notify quota zlib mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart imapflags notify namespace { list = children location = maildir:/mail/%%d/%%n/Maildir:INDEX=/mail/%d/%n/shared/%%u:CONTROL=/mail/%d/%n/shared/%%u prefix = SHARED/%%u/ separator = / type = shared } namespace inbox { hidden = no inbox = yes list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = INBOX/ separator = / subscriptions = yes type = private } passdb { args = /etc/dovecot/deny-users deny = yes driver = passwd-file } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile acl_shared_dict = file:/var/lib/dovecot/shared-mailboxes mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir:Mailbox-Quota quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = file:~/sieve;active=~/.dovecot.sieve sieve_extensions = +notify +imapflags zlib_save = gz zlib_save_level = 6 } pop3_client_workarounds = outlook-no-nuls oe-ns-eoh postmaster_address = postmaster(a)chemie.uni-hamburg.de protocols = " imap lmtp sieve pop3" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 8 service_count = 0 vsz_limit = 256 M } service imap { executable = imap postlogin process_limit = 1024 vsz_limit = 512 M } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } unix_listener /var/spool/postfix/private/lmtp-dovecot { group = postfix mode = 0660 user = postfix } } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { port = 995 ssl = yes } } service pop3 { executable = pop3 postlogin } service postlogin { executable = script-login -d rawlog } ssl = required ssl_ca = </ssl/CA.pem ssl_cert = </ssl/cert.pem ssl_key = </ssl/cert.key ssl_protocols = !SSLv2 !SSLv3 syslog_facility = daemon userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } verbose_proctitle = yes protocol lmtp { mail_plugins = acl mail_log notify quota zlib sieve } protocol lda { mail_plugins = acl mail_log notify quota zlib sieve } protocol imap { mail_max_userip_connections = 50 mail_plugins = acl mail_log notify quota zlib imap_acl imap_quota imap_zlib } protocol sieve { mail_max_userip_connections = 100 } protocol pop3 { mail_max_userip_connections = 50 } Thanks & kind regards, Christian Schmidt -- No signature available.

3 6

Re: sieve-filter crashes
by kjonca＠o2.pl 30 Dec '15

30 Dec '15

I tried to post earlier, but something goes wrong. KJ kjonca(a)o2.pl (Kamil Jońca) writes: > I want to refilter my mails in inbox so I tried to use: > > sieve-filter -e -W -D -C -v ~/.dovecot.sieve INBOX > > on three messages I got: > > info: filtering: [Fri, 28 Dec 2012 15:40:33 -0500; 14399 bytes] `[SPAM] Keep Defective by Design fight...'. > sieve-filter(kjonca): Panic: file str.c: line 22 (str_new_const): assertion failed: (str[len] == '\0') > sieve-filter(kjonca): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x82462) [0x7f2012002462] -> /usr/lib/dovecot/libdovecot.so.0(+0x824d9) [0x7f20120024d9] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f2011fa9a01] -> /usr/lib/dovecot/libdovecot.so.0(+0xa9098) [0x7f2012029098] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f1dc) [0x7f20128211dc] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) [0x7f2012807721] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5fa85) [0x7f2012821a85] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) [0x7f20127fe657] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) [0x7f20127fe7db] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4faca) [0x7f2012811aca] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) [0x7f2012812637] -> /usr/bin/sieve-filter(main+0x754) [0x7f2012c96d24] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f20119f4b45] -> /usr/bin/sieve-filter(+0x43ca) [0x7f2012c973ca] > > > # 2.2.19 (a9c75e48cf8c): /etc/dovecot/dovecot.conf > # Pigeonhole version 0.4.9 > # OS: Linux 3.16.0-4-amd64 x86_64 Debian stretch/sid > > KJ -- http://stopstopnop.pl/stop_stopnop.pl_o_nas.html A stitch in time saves nine.

3 3

How to apply the patch for disable SSL3 on Dovecot 2.0.9
by Marco 29 Dec '15

29 Dec '15

Hi,I see on Dovecot 2.0.9 is no possibile disable SSL3 Until I wait the panel of my server will look into this issue and maybe put a more updated version, how I can fix this?I found on the Internet http://askubuntu.com/questions/537196/how-do-i-patch-workaround-sslv3-poodl… also link to the patch http://www.mail-archive.com/dovecot@dovecot.org/msg59945.html I don't know what to do for fix the SSL 3 Issue I AM On CentoOs 6.3Thanks.

2 1

Howto get information about quota used
by Макаров Денис 29 Dec '15

29 Dec '15

Hello guys! I have Dovecot server (version: 2.2.9) with quota and imap_quota plugins. I wanna know how to get information about quota used of current email. I can do this with IMAP client like Thunderbird, but how I can do it on server? doveconf -n | grep quota mail_plugins = quota, antispam quota = maildir:User quota_exceeded_message = Message quota_rule = *:storage=100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u quota_warning3 = -storage=100%% quota-warning below %u service quota-warning { executable = /usr/local/bin/quota-warning.sh mail_plugins = quota imap_quota mail_plugins = quota sieve For each mailboxes I use quota limits from MySQL. -- С уважением, *Макаров Денис* *почта*: makarovdenis11(a)gmail.com <mailto:makarovdenis11@gmail.com> Best regards, *Makarov Denis* *email*: makarovdenis11(a)gmail.com <mailto:makarovdenis11@gmail.com>

2 1

Query regarding migration from Maillenium to Dovecot mail server
by Sirisha R 29 Dec '15

29 Dec '15

Hi, We have few queries regarding dovecot mail server. Our application used to supported maillenium mail server, but soon this server is going to retire. So we are planning to migrate our application mail server to dovecot for which we seek your help. Previously, the mailboxes that our application used to support were under att domain, then the mailboxes were migrated to maillenium mail server, and aliases of the existing att mailboxes were created by maillenium support group, so that any mail sent to the att mailbox automatically gets redirected to maillenium alias mailbox. For instance, for the mailbox abc.att.com, an alias was created as abc(a)maillenium.att.com<mailto:abc@maillenium.att.com>. Maillenium support group provided us mailServerHost, mailServerUsername and mailServerPassword. We seek your help in migrating from Maillenium mail system to Dovecot. Thanks & Regards, R Sirisha This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement, you may review at http://www.amdocs.com/email_disclaimer.asp

1 0

Sieve: Invalid/unknown path to storage
by Kay-Michael Voit 25 Dec '15

25 Dec '15

Hi, I have just updated from Ubuntu 15.04 to 15.10. Unfortunately, sieve scripts are not recognized anymore. Here is my version: > dovecot --version > 2.2.18 > apt-cache policy dovecot-sieve > dovecot-sieve: > Installed: 1:2.2.18-2ubuntu2 > Candidate: 1:2.2.18-2ubuntu2 > Version table: > *** 1:2.2.18-2ubuntu2 0 > 500 http://us.archive.ubuntu.com/ubuntu/ wily/main amd64 Packages > 100 /var/lib/dpkg/status And this is the log entry: > Dec 25 09:24:12 home-le dovecot: lda(user1): Warning: sieve: file > storage: Active sieve script symlink /home/user1/.dovecot.sieve is > broken: Invalid/unknown path to storage (points to /home/user1/sieve). > Dec 25 09:24:12 home-le dovecot: lda(user1): > msgid=<567CFD2C.7070301(a)example.org>: saved mail to INBOX > Dec 25 09:24:12 home-le postfix/local[28536]: 70876BD035A: > to=<user1(a)example.org>, relay=local, delay=0.07, > delays=0.02/0.01/0/0.04, dsn=2.0.0, status=sent (delivered to command: > /usr/lib/dovecot/deliver -c /etc/dovecot/dovecot.conf -m "${EXTENSION}") It is not a permission issue (though there should even be one with more restrictive permissions) > otheruser@home-le:~$ cat /home/user1/.dovecot.sieve > require ["variables","date", "include"]; > > include :personal "00-delete"; > include :personal "01-spam"; > include :personal "50-bulk_filter"; It is apparently also not a path normalization issue (also according to the correct "points to" path in the log). I used an absolute path in the symlink: > user1@home-le:~$ ls -la .dovecot.sieve > lrwxrwxrwx 1 user1 user1 33 Dec 25 09:23 .dovecot.sieve -> > /home/kay/sieve/managesieve.sieve Any idea what could be wrong here? Here is my dovecot -n output: > namespace inbox { > inbox = yes > location = > mailbox Drafts { > special_use = \Drafts > } > mailbox Junk { > special_use = \Junk > } > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > special_use = \Trash > } > prefix = > separator = . > type = private > } > passdb { > driver = pam > } > plugin { > sieve = file:~/.dovecot.sieve > sieve_dir = ~/sieve > } > postmaster_address = postmaster(a)example.net > protocols = " imap sieve" > service managesieve-login { > inet_listener sieve { > port = 4190 > } > } > ssl_ca = </etc/ssl/certs/startssl.ca.pem > ssl_cert = </etc/ssl/certs/net.example.home-le.pem > ssl_key = </etc/ssl/private/net.example.home-le.key > submission_host = 192.168.153.1 > userdb { > driver = passwd > } > protocol lda { > mail_plugins = " sieve" > } > protocol imap { > mail_max_userip_connections = 50 > }

2 2

Debug Dovecot Plugin (shared library)
by David Becher 24 Dec '15

24 Dec '15

Hey! I was just wondering what would be the best way to debug a plugin I am currently writing for dovecot. The plugin basically registers a new IMAP capability and I want to set a breakpoint to the "cmd_handle_new_capability" function (thats just a function name for better explaining purposes). Right now I have opened a connection via openssl ('openssl s_client -connect example:143 -starttls imap') to be able to mess around with the my server. Then I am able to login into the server via 'a login "username" "password"' and feed the server with my self created IMAP capability command. I know that I can start gdb and attach to an already running process via its PID. But how do I know which process I have to attach to? Looking at the output of 'top' it seems to be an instance of the 'imap' command PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 13413 vmail 20 0 21004 1844 1524 S 0.0 0.0 0:00.00 Iman But its only started the exact moment I issue my IMAP command towards the server, so of course I am not fast enough to track down the PID before the function has returned, so the process handling the command seems to be up only for the short period of time to handle the command, there is no process before which I can attach to. My current workaround is that I put a tail onto /var/log/mail.log and from within my shared library function (cmd_handle_new_capability) I log the PID of the current process, then sleep(10) for 10 seconds, which gives me enough time to have a look at mail.log to get to know the PID which I then can attach to. But isn't there a better way to do this? Maybe attach to the process which originally creates the new process handling the IMAP command and then stepping into it? I hope someone can help me out :) All the best, David

2 2

Dovecot 2.2.20 autoexpunge
by Robert Blayzor 23 Dec '15

23 Dec '15

I have configured autoexpunge on some folders: namespace inbox { inbox = yes location = mailbox Junk { auto = subscribe autoexpunge = 30 days special_use = \Junk } mailbox Trash { auto = subscribe autoexpunge = 2 weeks special_use = \Trash } prefix = separator = / } After adding these and restarting, logging in and out several times, I still see messages days/weeks past the autoexpunge setting. I checked the logs but never see any autoexpunge messages logged. Is there something else that needs to be set to trigger this? -- Robert inoc.net!rblayzor XMPP: rblayzor.AT.inoc.net PGP Key: 78BEDCE1 @ pgp.mit.edu

4 6

dsync and namespaces
by Charlie Hothersall-Thomas 22 Dec '15

22 Dec '15

`doveconf -n' output: > # 2.2.10: /etc/dovecot/dovecot.conf > # OS: Linux 3.10.0-229.20.1.el7.x86_64 x86_64 CentOS Linux release 7.2.1511 (Core) > listen = * > mail_location = maildir:~/Maildir/ > namespace { > inbox = yes > location = > prefix = > separator = / > type = private > } > namespace { > location = maildir:/srv/mail/ > prefix = Public/ > separator = / > subscriptions = no > type = public > } > passdb { > driver = pam > } > ssl_cert = </etc/pki/dovecot/certs/dovecot.pem > ssl_key = </etc/pki/dovecot/private/dovecot.pem > userdb { > driver = passwd > } I have some public mailboxes in /srv/mail -- Public/Alpha and Public/Mike -- using configuration taken more or less verbatim from http://wiki2.dovecot.org/SharedMailboxes/Public. These are stored in Maildir format; here's a sample of how things look on disk at the moment: > /srv/mail/ > ├── .Alpha > │ ├── cur > │ │ └── 1450775948.M24820P23708.cht,S=8186,W=8360:2,S > │ ├── dovecot.index.cache > │ ├── dovecot.index.log > │ ├── dovecot-uidlist > │ ├── maildirfolder > │ ├── new > │ └── tmp > ├── cur > ├── dovecot.index.log > ├── dovecot-uidlist > ├── dovecot-uidvalidity > ├── dovecot-uidvalidity.5679158d > ├── maildirfolder > ├── .Mike > │ ├── cur > │ │ ├── 1450775948.M202327P23708.cht,S=8405,W=8576:2,S > │ │ ├── 1450775948.M257575P23708.cht,S=5590,W=5732:2,S > │ ├── dovecot.index.cache > │ ├── dovecot.index.log > │ ├── dovecot-uidlist > │ ├── maildirfolder > │ ├── new > │ └── tmp > ├── new > └── tmp I'm trying to convert these public mailboxes from Maildir to mdbox format. My end goal is a directory structure like the following. I obtained this example by emptying /srv/mail, changing `location = maildir:/srv/mail/' to use mdbox, and then delivering some messages into Public/Alpha and Public/Mike: > /srv/mail/ > ├── dovecot-uidvalidity > ├── dovecot-uidvalidity.567915fa > ├── mailboxes > │ ├── Alpha > │ │ └── dbox-Mails > │ │ ├── dovecot.index.cache > │ │ └── dovecot.index.log > │ ├── dbox-Mails > │ │ └── dovecot.index.log > │ └── Mike > │ └── dbox-Mails > │ ├── dovecot.index.cache > │ └── dovecot.index.log > └── storage > ├── dovecot.map.index.log > └── m.1 I know that dsync can be used to convert between mailbox formats, but is this possible in conjunction with namespaces? I can convert one of my mailboxes to an arbitrary mdbox location on disk using... > dsync -o mail_location=mdbox:/tmp/dbox/ mirror maildir:/srv/mail/.Mike/ ... although this puts the messages inside an INBOX mailbox: > /tmp/dbox/ > ├── mailboxes > │ └── INBOX > │ └── dbox-Mails > │ ├── dovecot.index.cache > │ └── dovecot.index.log > └── storage > ├── dovecot.map.index.log > └── m.1 I could run this once per public mailbox, writing each to a different mdbox location, but am unaware of a method to merge them such that they would work with my Public/ namespace. Is it possible to use dsync to convert my public Maildir mailboxes to mdbox, resulting in my desired directory structure above? I think I want to be able to tell dsync the location as a mailbox name, rather than a disk location, although this doesn't appear to be possible. If not, is there another solution? My best idea at the moment is to make the configuration change and then re-deliver all messages by mailbox name. Thanks in advance, and happy holidays, -- Charlie Hothersall-Thomas | w: https://charlie.ht | e: me(a)charlie.ht | pgp: 4096R/C3676E3E

1 0

[bug][Pigeonhole 0.4.10] Missing msgid
by Alexander Moisseev 22 Dec '15

22 Dec '15

Pigeonhole 0.4.10 doesn't log msgid for some messages. Dec 16 09:57:57 mx dovecot: lmtp(user1(a)example.com): EbLlL2gLcVYCBgEAPZa6/Q: sieve: msgid=<20151216065756.BE1DF1FFD8(a)cp.bitrix24.com>: forwarded to <user3(a)example.com> Dec 16 09:57:57 mx dovecot: lmtp(user1(a)example.com): EbLlL2gLcVYCBgEAPZa6/Q: sieve: stored mail into mailbox 'INBOX' Dec 16 09:57:57 mx dovecot: lmtp(user2(a)example.com): FrLlL2gLcVYCBgEAPZa6/Q: sieve: msgid=<20151216065756.C77BB1FFD8(a)cp.bitrix24.com>: forwarded to <user3(a)example.com> Dec 16 09:57:57 mx dovecot: lmtp(user2(a)example.com): FrLlL2gLcVYCBgEAPZa6/Q: sieve: msgid=<20151216065756.C77BB1FFD8(a)cp.bitrix24.com>: stored mail into mailbox 'bitrix'

2 6

Where exactly are these Junk]Trash etc folders stored?
by Robert Chalmers 22 Dec '15

22 Dec '15

These folders are stored somewhere .. but where? mailbox Trash { auto = no special_use = \Trash } mailbox Drafts { auto = no special_use = \Drafts } mailbox Sent { auto = subscribe # autocreate and autosubscribe the Sent mailbox special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Spam { auto = create # autocreate Spam, but don't autosubscribe special_use = \Junk } mailbox virtual/All { # if you have a virtual "All messages" mailbox auto = no special_use = \All }

2 1

Thanks again for Dovecot
by Steve Litt 21 Dec '15

21 Dec '15

Hi all, All email clients suck. IMHO Claws-Mail sucks less, but even on the Claws-Mail mailing list there are all sorts of problems discussed. Most of these problems are a moot point for me, because I use Claws-Mail only as a window into my locally hosted Dovecot IMAP server. Claws-Mail is like a little window into Dovecot. Meanwhile, all those "don't quite work right" functionalities normally done by an email client I do outside of Claws. For filtering, I use Procmail, which sorts messages into the correct (Dovecot) folders and sends trolls to /dev/null. If Claws-Mail ever becomes as lame as Kmail, I can just switch to another email client, because most of my setup is in Dovecot and the input to Dovecot. Email clients are interchangeable for me, and this REALLY makes my life less worrysome. And Dovecot does one thing and does it right. It never breaks. Thank you for this great tool! SteveT Steve Litt November 2015 featured book: Troubleshooting Techniques of the Successful Technologist http://www.troubleshooters.com/techniques

1 0

questions for a unusual master/slave like mail-server setup
by Christoph Anton Mitterer 21 Dec '15

21 Dec '15

Hey folks. I was planning since a long time for some more advanced mail server setup (described below), which basically has some weird ;) master/slave ideas behind. It's probably a bit more complicated (and maybe also a bit more insane) than the usual mail server setup, so I'd be happy about any advice and some questions answered: There are two mailservers (dovecot, as you may have guessed ;) ): - IMAP shall be used; using POP3 is not planned. - A format in which each mail is stored in a separate file shall be used. That leaves for either maildir or sdbox, where I'd prefer maildir, as I like to be able to loose the indexes. - Each (per user) mailbox shall have many subfolders and will receive loads of mail (including from several high traffic mailing lists). - Mail filtering shall take place *at least* on the "online" server, probably either with sieve or maildrop. - One of the mailservers is always "online", directly connected to the internet (and globally available). - The other mailserver is "offline", not directly connected to the internet (and only locally available). The "online" mailserver: - Is in a not fully trusted location (i.e. at some ISP). - Has somewhat limited storage. - Makes the mails on it available to MUAs connecting from anywhere. - Receives mail from various sources, e.g. via postfix/LDA or something like fetchmail. - Has only a parts of *all* mails, that means for example: - *All* mail from some personal contacts. - Only the last 1 year for low volume mailing lists. - Only the last 4 months for high volume mailing lists. The "offline" mailserver: - Is in a trusted location (beneath my pillow ;) ). - Has enough storage for the purposes of mail. - Makes all mails available to MUAs connecting from the LAN. - Receives mails from the "online" mailserver, when manually "moved" to it via some MUA over IMAP. - Has *all* mails. Client's MUAs, may connect to both, but the "offline" server will only be available at home. As you can imagine, the idea is basically the following: I get large amounts of mails, from all kind of mailing lists as well as personal mail. All that is filtered to a tree hierarchy of folders. There's a not fully secure mailserver, where mail arrives and which I can use when I'm on the way. It contains either just recent mails (the amount depending on the traffic of the list) or for selected senders/lists everything, and I control basically on a per folder basis how much should be retained (all, last 10 years, last week, etc.). There's another server, where *all* mail should eventually end up, for archival and security reasons. This implies obviously, that some of the mail is on both servers, some only on the "offline" server, and that this may vary from folder to folder. Further, on both servers, I may want to add filters, which basically split up a folder, e.g. of high traffic lists, so that mail of previous years or months is moved to subfolder structures, e.g.: linux-kernel +--2014 \--2013 and so on. So far, all that wouldn't be too difficult,... but as you can imagine, I'd like to have a bit more, which actually makes things quite tricky. How the two mailservers shall synchronise: I don't want to "simply" rsync - the whole system needs to be a bit smarter: 1) It's okay to assume, that synchronisation happens only at discrete times and not continuously (the "offline" server doesn't run always). 2) It would be ok, if the synchronisation requires a downtime of the "online" mailserver (i.e. the LDA). Since btrfs is used, I could also easily do snapshots. 3) One can also assume that, when synchronisation happens, the "offline" server contains at least all those folders, that the "online" server has as well (and possibly more). 4) Obviously, only maildir's cur/ directories should be synced (stuff in tmp&new may be still arriving). 5) As mentioned previously, not all mail may stay on the "online" server, depending on the folder, it will either stay permanently, or only for some period. 6) However, on the "offline" server, all mail stays, unless I manually delete it. 7) One must assume, that mails on *both* servers: - change their IMAP status (as in (S)een, (A)nswered, (F)lagged, etc.) - are moved to other mail folders - may be deleted via MUAs (not clear yet where this can happen) - "Fresh" mail, however, only appears on the "online" server. 8) Mails, once on the "offline" server, shouldn't be rewritten, in the sense that the mail is deleted and recreated (hopefully with the same content), which could happen either wenn the status of the mail changes or when it's moved to another folder (on the other server). The reason is simply, that I wouldn't fully trust the online server, so once the mail is local, it should be immutable (except for the status and folder location). The last, two points make things pretty difficult, or at least I wouldn't see a straightforward solution. (4) would be quite easy, e.g. when rsync would be used, then I'd just rsync all the cur/ directory. The rsync would happen not using --delete, so that mail which is no longer on the "online" server, doesn't get deleted from the "offline" server. (5) one could do, by "simply", on a per folder basis, remove mails older than X on the "online" server, after it has been copied to the "offline" server. An already existing tool for doing that would be nice, though. (7) and (8) is IMHO the most difficult: The only idea I have so far assumes that each mail file has an ID, which is unique for the *whole* mailbox (not just for each folder, as the mails may be moved between folders). The idea would be roughly, the following: * When I synchronise, I either shut down the LDA or make a snapshot of the "online" server's mailbox and work with that. * Then, I make a list (ID, status and pathname) of *all* mails on the online server and another one of *all* mails of the offline server. * Then, I probably scan for duplicate IDs in each list... just to be safe. * Then I process the mail lists as follows: a) If an ID is on *both* servers with the same status and pathname, than that mail had been previously synced already and neither moved nor changed its status since then. Nothing to do. b) If an ID is on *both* servers but with different pathnames or status, then the mail has either been moved or it's status has changed. Consequently, I would move/rename the mail to match the new location and/or IMAP-status. I would move/rename that with the older ctime to match the one with the newer (ctime gets set at move or rename and thus at status change). Even if an attacker would have fiddled with the "online" server's time or somehow else my system times were bogus, the worst that would happen was, that status changes are lost or mail ends up in the wrong folder. c) If an ID is on the "offline" server but not the "online" server, it could be either a mail that has already been garbage collected on the "online" server (so nothing would needed to be done),... or it could be mail hat was removed (via a MUA) on the "online" server and that should as well be removed on the "offline" server, either automatically, or manually after checking. How to find out which of that applies!? d) If an ID is on the "online" server but not the "offline" server, it could be either fresh mail that would needed to be synced,... or it could be mail that was removed on the "offline" server and should as well deleted on the "online" server, either automatically, or manually after checking. (a) and (b) are again rather simple, (c) and (d) not so much... Possible solutions that I could think of: - Either simply don't sync the removal of mails from the "online" server to the "offline" server and vice-versa. It will anyway happen rather rarely. - Remember a list of all IDs, that have been garbage collected on the "online" server. - Remebmer a list of all IDs, that have been removed via MUAs. => not sure if that's easily doable, probably not, is there any dierct way one could do this with dovecot. - Something to find out which mail has been gone on both, the each, the "offline" and the "online" server, since the last synchronisation, via tricks using the snapshot features of btrfs. - Somehow using the date of now and the last synchronisation and the mtimes (which are here basically like creation times, as the mail file contents never change). But I guess that would only work for (d) and not for (c),... and I'd rather not depend on times too much. * In any case, I could give the user a list of mails that would be deleted, for double checking. I actually think this would scale, as I don't expect many mails to be manually deleted through the MUA *once they had been synced*. After that, I think the two servers should be in sync - in the sense described in the beginning. Some further parts of the idea: As said, snapshots and the refcopy feature of btrfs could be quite helpful. So I could basically keep: - the "online" mailserver's mailbox - as of the last sync - as of now - the "offline" mailserver's mailbox - as of the last sync - as of now at the disk of the "offline" server without much additional cost. They'd all share reflinks and wouldn't use (much) additional space (except for fs meta-data) and I could use the snapshots to find out which files were deleted on either of the two servers since last time. Actually copying new mail from the "online" server to the offline server's disk (in the "as of now" area), could then happen via rsync'ing every cur/ from the "online" server to the "offline" server, with --delete and --ignore-existing (as I don't trust the "online" server and mail file contents should be immutable). So... questions... :D 1) Obviously, does anyone know a better solution, or has some improvements, other ideas? Or are there any problems with the above, which I haven't realised, or would you think it would work? 2) Does dovecot *ever* change the contents (i.e. the file contents) of a maildir mail file? 3) As ID I'd have thought that the maildir mail filename might work, e.g. 1234567890.M20046P2137.mailserver,S=4542,W=4642:2,Sb I mean especially the first part: 1234567890.M20046P2137.mailserver Is that unique across *all* folders of a mailbox? AFAIU, that name would be determined by whatever delivers the mail, right? So can I somehow configure that to make it unique? 4) Does dovecot ever change anything from the maildir mail filenames, except anything after the ":" above? I.e. would it ever change: 1234567890.M20046P2137.mailserver,S=4542,W=4642 ? 5) Would dovecot every copy, truncate+copy or somehow else rewrite files, instead of just moving them (i.e. thus possibly changing inodes, reflinks or that like)? 6) AFAIU, when using maildir, dovecots index files can get completely lost,... and one can fully "recover" all mails and their status. The only "bad" thing that might happen is that MUAs re-load all mail, right? 7) How graceful does dovceot handle the situation when mail is added or removed to/from cur/ by not using dovecot's IMAP,... especially with respect to index files. In the above solution I'd do a lot adding/moving/deleting files in maildirs, which would be done purely at the (file) system level, so the index would quite often get invalidated. How does it recognise that (if at all)? And is this generally stable, or should I rather remove any indexes after syncing and have them rebuilt by that? 8) All the above doesn't handle yet the thing of having yearly/monthly/etc. archive subfolders as described above, like: linux-kernel +--2014 \--2013 possibly on *both* "online" and "offline" server and possibly even in a different layout. I'm not yet sure whether this would be easily doable with the above solution,... my basic idea would be simply consider any such "archive" subfolders as the same than the "base" folder during syncing... and afterwards do re-order the mails on the "offline" and "online" server's folders as necessary. For example, if the "online" server has: linux-kernel +--01 +--02 +--03 ... \--12 in other words, one archive folder for the each of the last 12 months. While the "offline" server has: linux-kernel +--2014 \--2013 The script would then e.g. move the files to their new folders. => does anyone know, whether such thing exists already, i.e. a tool that sorts the mails into maildir subfolders, based on years, months, etc? 9) As said before, I'd like to be able to loose any indexes, but AFAICS, my solution wouldn't anyway work with sdbox. But if there's an sdbox based solution that makes everything much easier, do not hesitate to tell me :) 10) In the long-term I hope to be able to do something that makes server-side search (i.e. through the big fat mail archive) much faster (or is that already supported in dovecot?). I haven't really looked at all into that topic (just know about notmuch[0]). Anyway, if anyone sees problems in my above solution, when that would be used with something that gives me better server side search (i.e. some indexing solution),... I'd be happy to hear about that as well. Thanks a lot so far, Chris. [0] https://notmuchmail.org/

1 0

Possible bug about the SPECIAL-USE / CREATE-SPECIAL-USE extensions
by Jeff Stedfast 20 Dec '15

20 Dec '15

Hi all, I've locally got Dovecot 2.2.21 (5345f22) installed on my MacBook (via homebrew) that I've been using to test my IMAP library against. What is interesting is that even though Dovecot does not advertise CREATE-SPECIAL-USE, it supports creation of special-use folders via the "CREATE FolderName (USE (\Whatever))" syntax. According to rfc6154, a server is supposed to advertise the CREATE-SPECIAL-USE extension if it supports this: An IMAP server that supports this OPTIONAL feature will advertise the "CREATE-SPECIAL-USE" capability string. Clients MUST NOT use the "USE" parameter unless the server advertises the capability. Note that this capability string is different from the "SPECIAL-USE" string defined above, and a server that supports both functions MUST advertise both capability strings. Has this been fixed in a newer version, perhaps? Or could it be a misconfiguration on my part? Attached is my dovecot configuration (gotten via dovecot -n). Thanks, Jeff

2 2

Multiple IPs and hostname
by Thilo Engelbracht 18 Dec '15

18 Dec '15

Hi, I'm running postfix as SMTP-server and dovecot as IMAP-server. The server has multiple IPs, and postfix is configured that every domain is using a separate IPv4-address. Examle: foo1.com --> 11.22.33.44 foo2.com --> 22.33.44.55 foo3.com --> 33.44.55.66 bar.org --> 66.77.88.99 The hostname of the server is: mail.bar.org Now I have a request from one of my customers, who is using 3 different domains: - foo1.com - foo2.com - foo3.com If he receives a mail to "user(a)foo1.com", the header of the mail looks like this: Return-Path: <somebody(a)web.de> Delivered-To: <user(a)foo1.com> (II) Received: from mail.bar.org by mail.bar.org (Dovecot) with LMTP id V[...]wAAvP/pag for <user(a)foo1.com>; Sun, 29 Nov 2015 13:45:37 +0100 (I) Received: from mout.web.de (mout.web.de [212.227.17.11]) (using TLSv1.2 with cipher ECDHE-[...]-SHA384 (256/256 bits)) (No client certificate requested) by mail.foo1.com (Postfix) with ESMTPS id A[...]9 for <user(a)foo1.com>; Sun, 29 Nov 2015 13:45:36 +0100 (CET) As you can see in (I), postfix receives the mail and added a header with the correct hostname: mail.foo1.com But dovecot adds an additional header (see II). Here, the hostname of the system is visible. This is correct, because dovecot -n | grep -i hostname returns: hostname = mail.bar.org But my customer asks me if it's possible that only his domain is visible in the header: So, if somebody sends a mail to "user(a)foo1.com", only the hostname "mail.foo1.com" should be visible in the mail header. And if somebody sends a mail to "user(a)foo2.com", only the hostname "mail.foo2.com" should be visible in the mail header. Is there a way to do this? Thank you in advance. Kind regards Thilo

2 3