- dovecot - dovecot.org

ltmp delivery with ldap and prefetch
by Marc Roos 04 Sep '20

04 Sep '20

I am able to authorize and connect via imap. But I am not able to deliver a message via lmtp I am getting this error message. passdb didn't return userdb entries, trying the next userdb [1] dovecot-ldap.conf.ext pass_attrs = uid=user,userPassword=password,host=host,homeDirectory=userdb_home,uidNu mber=userdb_uid,gidNumber=userdb_gid user_attrs = uid=%{ldap:uidNumber},gid=%{ldap:gidNumber},home=%{ldap:homeDirectory} [2] pass / user db's passdb { args = /etc/dovecot/dovecot-ldap.conf.ext auth_verbose = default default_fields = deny = no driver = ldap master = no mechanisms = name = override_fields = pass = no result_failure = continue result_internalfail = continue result_success = return-ok skip = never username_filter = } userdb { args = /etc/dovecot/special-userdb auth_verbose = default default_fields = driver = passwd-file name = override_fields = result_failure = continue result_internalfail = continue result_success = return-ok skip = never } userdb { args = auth_verbose = default default_fields = driver = prefetch name = override_fields = result_failure = continue result_internalfail = continue result_success = return-ok skip = never }

1 0

Moving Spam to Junk Folder
by bobby 03 Sep '20

03 Sep '20

I am following this tutorial: https://www.linuxbabe.com/redhat/spamassassin-centos-rhel-block-email-spam. I followed the steps in "Move Spam into the Junk Folder". When I send an email from a blacklisted e-mail address, I get a bounce e-mail from my e-mail server. Here is what is in my spamass-milter file: EXTRA_FLAGS="-m -r 8 -R NO_SPAM -i 127.0.0.1 -g sa-milt -- --max-size=5120000" I would prefer it to go into my Junk folder. How can I make this happen?

3 3

Fwd: zlib errors after upgrading to 2.3.11.3
by Robert Nowotny 03 Sep '20

03 Sep '20

Dear Aki, I switched to "gz" now, since "zstd" also gave some errors on writing to files. I dont know if "xz" compression or "zstd" shreddered my MDBOX Files, but I lost 4 days of mail. (a couple of thousand mails). After restoring the backup (what was made after switching to version 2.3.11.3) I still have some broken mdfiles, but not too many. Interestingly always in the "Sent" Mailbox on a number of Users. I just can not go back to 20.08.2020 before I updated to 2.3.11.3 - too many emails would have been lost. So - the current status is 2.3.11.3, with "gz" compression. I force-synced and re-indexed all the mdbox files (250 GB), but still have some broken - please how can I fix those ? You stated : (In theory you could leave the existing mails xz-compressed, but best would be to re-compress everything via dsync so old mails can be read when we eventually remove xz support.) what is the optimal way to do that, expecially not loosing the index (?) for outlook/thunderbird, for third party tools that rely on some index/hash (dunno how exactly that works) ? Sep 3 08:33:25 lxc-imap dovecot: imap(mpaul)<48684><2/9E5mKuAezAqKjk>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 08:35:26 lxc-imap dovecot: imap(mpaul)<48089><YRio1GKuEdHAqKi3>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 08:35:26 lxc-imap dovecot: imap(mpaul)<49228><p/0g9WKuVdHAqKi3>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 08:35:26 lxc-imap dovecot: imap(mpaul)<49229><GMAm9WKuVtHAqKi3>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 08:35:27 lxc-imap dovecot: imap(mpaul)<49230><YYot9WKuV9HAqKi3>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) ... Sep 3 09:04:44 lxc-imap dovecot: imap(cpotzinger)<49040></X+P72KuUNHAqKi3>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:04:51 lxc-imap dovecot: imap(cpotzinger)<54382><eYT1XWOux9HAqKi3>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:04:59 lxc-imap dovecot: imap(cpotzinger)<54396><nKllXmOuyNHAqKi3>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:05:06 lxc-imap dovecot: imap(cpotzinger)<54409><m2TOXmOuydHAqKi3>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:05:12 lxc-imap dovecot: imap(cpotzinger)<54422><IP86X2OuytHAqKi3>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) ... Sep 3 09:08:58 lxc-imap dovecot: imap(cpotzinger)<54867><BtGtbGOu49HAqKi3>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:19:15 lxc-imap dovecot: imap(mpaul)<56030></kHNkWOuAO3AqKjk>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 09:29:41 lxc-imap dovecot: imap(cpotzinger)<57274><c4rCtWOuKuzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:29:47 lxc-imap dovecot: imap(cpotzinger)<57320><K3Uqt2OuLOzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:29:54 lxc-imap dovecot: imap(cpotzinger)<57333><e4aMt2OuLezAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:30:01 lxc-imap dovecot: imap(cpotzinger)<57346><JEfzt2OuLuzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:30:08 lxc-imap dovecot: imap(cpotzinger)<57360><mH1YuGOuMOzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:30:14 lxc-imap dovecot: imap(cpotzinger)<57382><IzLBuGOuMezAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:30:21 lxc-imap dovecot: imap(cpotzinger)<57397><q10kuWOuM+zAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:30:27 lxc-imap dovecot: imap(cpotzinger)<57410><CSuJuWOuNOzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:31:22 lxc-imap dovecot: imap(mpaul)<57525><TfoOvWOuWO3AqKjk>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 09:31:49 lxc-imap dovecot: imap(cpotzinger)<57423><Be/ruWOuNuzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:31:55 lxc-imap dovecot: imap(cpotzinger)<57584><yvXOvmOuQuzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:32:02 lxc-imap dovecot: imap(cpotzinger)<57597><e/8tv2OuQ+zAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:32:08 lxc-imap dovecot: imap(cpotzinger)<57611><BLSOv2OuROzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:32:19 lxc-imap dovecot: imap(cpotzinger)<57633><Ky0twGOuR+zAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:32:26 lxc-imap dovecot: imap(cpotzinger)<57646><ljSRwGOuS+zAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:32:33 lxc-imap dovecot: imap(cpotzinger)<57659><rn75wGOuT+zAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:32:39 lxc-imap dovecot: imap(cpotzinger)<57672><GP9jwWOuUuzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:33:45 lxc-imap dovecot: imap(mpaul)<57795><2CEwxWOubu3AqKjk>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 09:38:29 lxc-imap dovecot: imap(mpaul)<58347><CKyX1mOuie3AqKjk>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 09:38:32 lxc-imap dovecot: imap(mpaul)<58352><osHU1mOuiu3AqKjk>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Sep 3 09:38:44 lxc-imap dovecot: imap(cpotzinger)<54527><5ZpiYmOuBOfAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:38:51 lxc-imap dovecot: imap(cpotzinger)<58378><kAuN12OumuzAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Sep 3 09:38:57 lxc-imap dovecot: imap(cpotzinger)<58392><xJLx12Oum+zAqKiU>: Error: Mailbox Sent: UID=29534: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) ... Am 26.08.2020 um 09:42 schrieb Robert Nowotny: > I had 4 broken mdbox files for 4 different users. > After deleting them, dovecot re-indexed, now everything is ok again. > > I can try to update tonight again to 2.3.11.3 and see if some broken > files will appear again tomorrow. > > Am 25.08.2020 um 14:51 schrieb Aki Tuomi: >>> On 25/08/2020 14:35 Robert Nowotny<rnowotny(a)rotek.at> wrote: >>> >>> >>> I get ZLIB Errors after dovecot upgrade from 2.3.10.1 to 2.3.11.3 >>> >>> >>> Aug 21 15:27:34 lxc-imap dovecot: imap(acsida)<63870><jZk...>: Error: Mailbox Sent: UID=40826: read(zlib(/home/vmail/virtualmailboxes/acsida/storage/m.2409)) failed: read(/home/vmail/virtualmailboxes/acsida/storage/m.2409) failed: Broken pipe (FETCH BODY[]).... >>> >>> many of those entries.... >>> >>> after installing2.3.10.1 those files are still broken - but new files are working again. >>> >>> anything I can tweak or do ? >>> >>> >> Hi! >> >> Is it possible your mdbox file is corrupted? Can you share doveconf -n? >> >> Aki >

1 0

Submission proxy warning connecting to Exim
by Alec Moskvin 03 Sep '20

03 Sep '20

Hello, When connecting to Exim 4.94, dovecot submission logs this warning: dovecot[25235]: submission(alec)<25347><QNydV8+t+oMKAgAF>: Warning: smtp-client: conn localhost:25 ([::1]:25) [1]: Received invalid EHLO response line: Unexpected character in EHLO keyword Looks like it doesn't like the underscore in "X_PIPE_CONNECT", since it's neither "isalnum" nor a dash [1]: 1598480923.401085 220 myhostname ESMTP Exim 4.94 Wed, 26 Aug 2020 18:28:43 -0400 1598480923.401906 250-myhostname Hello localhost [::1] 1598480923.401906 250-SIZE 52428800 1598480923.401906 250-8BITMIME 1598480923.401906 250-PIPELINING 1598480923.401906 250-X_PIPE_CONNECT 1598480923.401906 250-CHUNKING 1598480923.401906 250-STARTTLS 1598480923.401906 250 HELP 1598480923.409242 250 OK 1598480923.434743 250 Accepted 1598480923.578335 250- 511 byte chunk, total 511 1598480923.578335 250 OK id=1kB3uN-0006ar-HU Adding the line "pipelining_connect_advertise_hosts =" to Exim's config fixes the issue by disabling X_PIPE_CONNECT. Is this a bug in Exim or is Dovecot being too strict? Thanks, Alec [1] https://github.com/dovecot/core/blob/2.3.11.3/src/lib-smtp/smtp-syntax.c#L2…

3 2

Sumbission crashes when relaying over TLS
by Alec Moskvin 03 Sep '20

03 Sep '20

Hello, I'm trying to set up the submission proxy, but if I set submission_relay_ssl = starttls, it crashes. Without it, it works. Please find the details below. Thanks, Alec dovecot[256855]: submission-login: Login: user=<alec>, method=PLAIN, rip=::1, lip=::1, mpid=257033, secured, session=<kMd1B9uthLUAAAAAAAAAAAAAAAAAAAAB> dovecot[256855]: submission(alec)<257033><kMd1B9uthLUAAAAAAAAAAAAAAAAAAAAB>: Panic: file smtp-client-connection.c: line 290 (smtp_client_connection_fail): assertion failed: (conn->to_connect == NULL) dovecot[256855]: submission(alec)<257033><kMd1B9uthLUAAAAAAAAAAAAAAAAAAAAB>: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(backtrace_append+0x42) [0x7fbbfb2bcaf2] -> /usr/lib64/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7fbbfb2bcc0e] -> /usr/lib64/dovecot/libdovecot.so.0(+0xf5f09) [0x7fbbfb2c5f09] -> /usr/lib64/dovecot/libdovecot.so.0(+0xf5fa1) [0x7fbbfb2c5fa1] -> /usr/lib64/dovecot/libdovecot.so.0(+0x51eec) [0x7fbbfb221eec] -> /usr/lib64/dovecot/libdovecot.so.0(+0x44d5e) [0x7fbbfb214d5e] -> /usr/lib64/dovecot/libdovecot.so.0(+0x6c431) [0x7fbbfb23c431] -> /usr/lib64/dovecot/libdovecot.so.0(smtp_client_command_input_reply+0x201) [0x7fbbfb235261] -> /usr/lib64/dovecot/libdovecot.so.0(+0x6ccfd) [0x7fbbfb23ccfd] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x68) [0x7fbbfb2dc048] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x132) [0x7fbbfb2dd662] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x51) [0x7fbbfb2dc0f1] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7fbbfb2dc2b0] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fbbfb255ee3] -> dovecot/submission(main+0x4df) [0x561d486b3daf] -> /lib64/libc.so.6(__libc_start_main+0xea) [0x7fbbfb054b8a] -> dovecot/submission(_start+0x2a) [0x561d486b3e3a] dovecot[256855]: submission(alec)<257033><kMd1B9uthLUAAAAAAAAAAAAAAAAAAAAB>: Fatal: master: service(submission): child 257033 killed with signal 6 (core dumped) (gdb) bt #0 0x00007fbbfb069a31 in raise () from /lib64/libc.so.6 #1 0x00007fbbfb0532b6 in abort () from /lib64/libc.so.6 #2 0x00007fbbfb22223c in default_fatal_finish (status=0, type=LOG_TYPE_PANIC) at failures.c:459 #3 fatal_handler_real (ctx=<optimized out>, format=<optimized out>, args=<optimized out>) at failures.c:471 #4 0x00007fbbfb2c5fa1 in i_internal_fatal_handler (ctx=<optimized out>, format=<optimized out>, args=<optimized out>) at failures.c:848 #5 0x00007fbbfb221eec in i_panic (format=format@entry=0x7fbbfb30fd08 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:523 #6 0x00007fbbfb214d5e in smtp_client_connection_fail (status=9002, error=<optimized out>, conn=0x561d491296a8) at smtp-client-connection.c:290 #7 smtp_client_connection_fail (conn=0x561d491296a8, status=<optimized out>, error=<optimized out>) at smtp-client-connection.c:282 #8 0x00007fbbfb23c431 in smtp_client_connection_starttls_cb (reply=0x561d4912d048, conn=0x561d491296a8) at smtp-client-connection.c:788 #9 0x00007fbbfb235261 in smtp_client_command_input_reply (cmd=<optimized out>, reply=reply@entry=0x561d4912d048) at smtp-client-command.c:971 #10 0x00007fbbfb23ccfd in smtp_client_connection_input_reply (reply=0x561d4912d048, conn=0x561d491296a8) at smtp-client-connection.c:1045 #11 smtp_client_connection_input (_conn=<optimized out>) at smtp-client-connection.c:1116 #12 0x00007fbbfb2dc048 in io_loop_call_io (io=0x561d49109cb0) at ioloop.c:713 #13 0x00007fbbfb2dd662 in io_loop_handler_run_internal (ioloop=ioloop@entry=0x561d490fd050) at ioloop-epoll.c:222 #14 0x00007fbbfb2dc0f1 in io_loop_handler_run (ioloop=0x561d490fd050) at ioloop.c:765 #15 0x00007fbbfb2dc2b0 in io_loop_run (ioloop=0x561d490fd050) at ioloop.c:738 #16 0x00007fbbfb255ee3 in master_service_run (service=0x561d490fceb0, callback=<optimized out>) at master-service.c:851 #17 0x0000561d486b3daf in main (argc=<optimized out>, argv=<optimized out>) at main.c:396 $ openssl version OpenSSL 1.1.1g 21 Apr 2020 # doveconf -n # 2.3.11.3 (502c39af9): /etc/dovecot/dovecot.conf # OS: Linux 5.8.4 x86_64 Gentoo Base System release 2.7 # Hostname: haswell listen = localhost mail_location = maildir:~/.maildir namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } protocols = submission ssl_cert = </etc/ssl/dovecot/server.pem ssl_client_ca_file = </etc/dovecot/ca.crt ssl_key = # hidden, use -P to show it submission_relay_host = localhost submission_relay_port = 2525 submission_relay_rawlog_dir = %h/rawlog submission_relay_ssl = starttls userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } protocol submission { rawlog_dir = %h/rawlog }

2 2

submission message quota
by Gerry 03 Sep '20

03 Sep '20

Hello, I am trying to come up with a way to have individual quotas per user for the submission service. Similar to what I could achieve with Postfix and policyd. More specifically, the quota I am most interested in, is limiting the number of messages a single account can send within a given timeframe. Ideally, I'd also like to limit the number of total recipients within a given timeframe, to mitigate the loophole of adding multiple recipients to a single message. Example: account Y is allowed to send 500 messages per 60 minutes, with a maximum of 2000 recipients overall. What would be the best path to take? Thanks! Gerry

4 5

Metric label values truncated when using OpenMetrics endpoint
by Daan van Gorkum 02 Sep '20

02 Sep '20

Hi, Recently we upgraded to Dovecot 2.3.11.3 and configured an example metric like this: metric imap_command { event_name = imap_command_finished group_by = cmd_name tagged_reply_state user remote_ip } And enabled the OpenMetrics listener like this: service stats { inet_listener http { port = 5000 } } While the result is great, I noticed that some metrics are being truncated (three dots at the end of the label value): dovecot_imap_command_duration_usecs_sum{cmd_name="FETCH",tagged_reply_state="OK",user="xxxxxxxxxxx.inbox@xxxxxxxxxxxxx…",remote_ip="XXXX:XXXX:XXXX:XXXX:cc6b:cc7a:d…"} 2346 1598055662473 I couldn't find this information in the documentation so I'm wondering if this is a bug. Thank you for your time. Regards, Daan

2 5

systemctl unit should depend on remote-fs
by Tom Sommer 02 Sep '20

02 Sep '20

The shipped systemctl unit file should have "remote-fs.target" added in "After="? -- Tom

1 0

Mail replication does not work
by Pavel Just 02 Sep '20

02 Sep '20

Hi, I have installed two Debian 10.5 (SSH server) systems and I would like replicate mails between them. But the replication is very unstable. Before start of dovecot Source mailbox: 278 messages, Destination mailbox: 0 messages Another mailbox: 16887 messages, D.. 0 messages After start of dovecot Source mailbox: 278 messages, Destination mailbox: 303 messages Another mailbox: 16887 messages, Dest..: 31904 messages In many cases replication hangs. Error message is in the end of the message. If replication hangs, after restart a lot of mail are copied from destination back to source and mailbox grows without new messages. Postfix was stopped during replication. Please, what is wrong in my configuration (except security :-) ? -- Pavel # 2.3.11.3 (502c39af9): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.11 (6c69c917) # OS: Linux 4.19.0-10-amd64 x86_64 Debian 10.5 # Hostname: mail.simac.cz disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it first_valid_uid = 1001 last_valid_uid = 32000 mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_plugins = " notify replication" mail_privileged_group = mail mail_temp_dir = /var/tmp managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { mail_replica = tcp:192.168.240.2:12345 sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = " imap sieve pop3" service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = root } unix_listener replication-notify { mode = 0666 user = root } } service doveadm { inet_listener { port = 12345 ssl = no } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0666 user = root } } service stats { unix_listener stats-reader { mode = 0666 user = root } unix_listener stats-writer { mode = 0666 user = root } } ssl = no userdb { driver = passwd } protocol lmtp { mail_plugins = " notify replication sieve" } protocol lda { mail_plugins = " notify replication sieve" } ____________________________________ Error message ________________ Sep 2 14:08:00 backup dovecot: doveadm: Panic: file mbox-lock.c: line 796 (mbox_lock): assertion failed: (lock_type == F_RDLCK || mbox->mbox_lock_type != F_RDLCK) Sep 2 14:08:00 backup dovecot: doveadm: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x3d) [0x7f300ef00eed] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7f300ef0100e] -> /usr/lib/dovecot/libdovecot.so.0(+0xf465b) [0x7f300ef0a65b] -> /usr/lib/dovecot/libdovecot.so.0(+0xf46f1) [0x7f300ef0a6f1] -> /usr/lib/dovecot/libdovecot.so.0(+0x514a6) [0x7f300ee674a6] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x438e5) [0x7f300f0108e5] -> /usr/lib/dovecot/libdovecot-storage.so.0(mbox_save_begin+0x6fd) [0x7f300f05fa7d] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_save_begin+0x92) [0x7f300f0299e2] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_storage_copy+0x89) [0x7f300f01c4e9] -> /usr/lib/dovecot/modules/lib15_notify_plugin.so(+0x246f) [0x7f300e71846f] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x5cb3c) [0x7f300f029b3c] -> dovecot/doveadm-server(+0x4f805) [0x556ee519a805] -> dovecot/doveadm-server(+0x501f7) [0x556ee519b1f7] -> dovecot/doveadm-server(dsync_mailbox_import_changes_finish+0x164) [0x556ee519dba4] -> dovecot/doveadm-server(dsync_brain_sync_mails+0x735) [0x556ee5198e85] -> dovecot/doveadm-server(dsync_brain_run+0x218) [0x556ee5194858] -> dovecot/doveadm-server(+0x49eae) [0x556ee5194eae] -> dovecot/doveadm-server(+0x5aa6f) [0x556ee51a5a6f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7f300ef20989] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x131) [0x7f300ef21ed1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7f300ef20a2c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7f300ef20ba0] -> dovecot/doveadm-server(+0x2e20e) [0x556ee517920e] -> dovecot/doveadm-server(+0x2fa56) [0x556ee517aa56] -> dovecot/doveadm-server(+0x45cad) [0x556ee5190cad] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7f300ef20989] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x131) [0x7f300ef21ed1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7f300ef20a2c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7f300ef20ba0] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f300ee9a9c3] Sep 2 14:08:00 backup dovecot: doveadm: Fatal: master: service(doveadm): child 1977 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps)

2 1

Incremental purge?
by Jesus Cea 02 Sep '20

02 Sep '20

I am using mdbox storage format. As far as I know, “doveadm purge” does a complete purge. The time and disk activity would be proportional to the number mdbox files on the storage. The purge time could be huge. I wonder if there is some way of doing a kind of incremental purge, stopping/restarting purging, etc. Purging by file range would be nice too. Advice? Thanks Enviado desde mi iPhone

5 7

Dovecot Proxy
by Thoralf Rickert-Wendt 02 Sep '20

02 Sep '20

Hello everyone, it's my first post here on this mailing list and I hope, I make it right. I posted a question on https://serverfault.com/questions/1031441/dovecot-as-proxy-with-submission and nobody was able to answer it. So I decided to push that question here (I'm talking about any new dovecot version and I've tested it with 2.3.4.1 (f79e8e7e4)). I try to run a dovecot proxy in front of a big number of mail servers (serving SMTP-in, submission, IMAP, POP3, Sieve). I need that proxy, because I run out of IPv4 addresses. Of course I use IPv6 too, but many customers still have problems with there providers and they really don't want to share their mails on a "shared-mailserver". I planed to use Dovecot for IMAPS, POP3S, SMTP-submission(465) and postfix for the rest. If I find a solution for sieve, I would try that too, but that is very optional. With the documentation https://wiki1.dovecot.org/HowTo/ImapProxy (which is really old and should be updated) and some other ascii docs (from an Apple mirror somewhere deep in the web) I was able to build a IMAP/POP3 proxy that forwards requests from outside to a specific backend using SSL (993,995). That works - I think.You can find the config on the serverfault page. In general - all known domains in backend are using SSL and the passdb forwards all requests to the backend via SSL. So - I understand:|| |password_query = SELECT NULL AS password, NULL AS destuser, host, 'Y' AS nologin, 'Y' AS nodelay, 'Y' AS nopassword, 'Y' AS proxy, 'any-cert' AS `ssl` FROM proxy_domain WHERE domain = '%d' | But that is only 50% of the show. The rest ist submission (and maybe sieve). Practically the submission implementation in dovecot works too. But because dovecot by default only opens port 587 (starttls), my passdb setting has a problem. When I try to use that port Dovecot tries to use SSL on the backend/587 too - but that is wrong (it should either use 465 or should try to use starttls). So, I have the following options. - find a way to configure dovecot-proxy to listen on 465 with SSL for submission service and hope that it uses the same port - but I didn't find any documentation for that and need help - find a way to configure dovecot-proxy/passdb to return starttls=y when dovecot-submission is used (use a different passdb) - but I didn't find any documentation for that and I'm not sure, if this worls on service/protocol level - find a way to configure the passdb answer based on the used port/protocol. But I only know the parameter %u, %d and %p. - so it would be nice to find a way to also select the protocol (if already developed) - find a way to make a patch in dovecot (which isn't easy for me, because I don't really know the code) Has somebody an idea, how I can configure the dovecot-proxy in that way. bye Thoralf

3 4

about header address parsing
by TACHIBANA Masashi 02 Sep '20

02 Sep '20

Hi, Is this expected or not? From: user1(a)fuga.example.com <user1(a)example.com> To: user2(a)hoge.example.com <user2(a)example.com> ↓ a uid fetch 43055 (envelope) * 1860 FETCH (UID 43055 ENVELOPE ("Thu, 30 Jul 2020 13:52:59 +0900" "test1" ((NIL NIL "user1" "fuga.example.com")) ((NIL NIL "user1" "fuga.example.com")) ((NIL NIL "user1" "fuga.example.com")) ((NIL NIL "user2" "hoge.example.com")) NIL NIL NIL "<WLXvFmpAZFNeQbPPITjRwimDyamnBm1@test>")) From: "user1(a)fuga.example.com" <user1(a)example.com> To: "user2(a)hoge.example.com" <user2(a)example.com> ↓ a uid fetch 43056 (envelope) * 1861 FETCH (UID 43056 ENVELOPE ("Thu, 30 Jul 2020 13:53:59 +0900" "test1" (("user1(a)fuga.example.com" NIL "user1" "example.com")) (("user1(a)fuga.example.com" NIL "user1" "example.com")) (("user1(a)fuga.example.com" NIL "user1" "example.com")) (("user2(a)hoge.example.com" NIL "user2" "example.com")) NIL NIL NIL "<WLXvFmpAZFNeQbPPITjRwimDyamnBm2@test>")) Regards, -- TACHIBANA Masashi QUALITIA CO., LTD. mailto:tachibana@qualitia.co.jp 株式会社クオリティア https://www.qualitia.co.jp/

3 3

Bug: Dovecot appending "MISSING_DOMAIN" to fetch envelope responses
by Michael Gratton 02 Sep '20

02 Sep '20

Hi all, I'm seeing Dovecot include the string "MISSING_DOMAIN" in fetch envelope requests when an mailbox's `addr-spec` part does not have a `domain` part. For example: > C: a022 uid fetch 40 (envelope rfc822.header) > S: * 5 FETCH (UID 40 ENVELOPE ("Sat, 15 Aug 2020 12:53:05 +1000" > "test {{name}}" (("Michael Gratton" NIL "mike" "vee.net")) (("Michael > Gratton" NIL "mike" "vee.net")) (("Michael Gratton" NIL "mike" > "vee.net")) (("postgres" NIL "postgres" "MISSING_DOMAIN")) NIL NIL > NIL NIL) RFC822.HEADER {literal/244b}) I understand that RFC 5322 require a mailbox to have a domain part, however it is still possible for mail to contain mailboxes that do not. In these cases, Dovecot returning random string such as the above is harmful, since it not possible for IMAP clients to determine whether host name string is actually present or something that the IMAP server has made up. As a result, an IMAP client has two options: Either display the string as-is, preventing it from providing an appropriately localised explanation or visual indication of the problem in the user interface, or simply avoid ever requesting IMAP envelopes from the server and always download and parse the complete RFC822 headers itself (as I believe Thunderbird does). The former is untenable since "MISSING_DOMAIN" does not mean anything to non-English speakers, the latter is untenable since it unnecessarily inflates the of volume of data that must be transmitted over the wire to work around a server bug. In these cases, or when any other part of a required IMAP address structure is missing, Dovecot should return the empty string, e.g. "". Cheers, //Mike Dovecot details: > *REDACTED*:~$ sudo dovecot -n > # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf > # Pigeonhole version 0.5.4 () > # OS: Linux 4.19.0-9-amd64 x86_64 Debian 10.4 ext4 > # Hostname: *REDACTED* > first_valid_uid = *REDACTED* > hostname = *REDACTED* > last_valid_uid = *REDACTED* > lda_original_recipient_header = X-Original-To > lmtp_rcpt_check_quota = yes > lmtp_save_to_detail_mailbox = yes > mail_gid = *REDACTED* > mail_home = *REDACTED* > mail_location = mdbox:*REDACTED*:UTF-8 > mail_privileged_group = mail > mail_server_admin = mailto:*REDACTED* > mail_temp_dir = *REDACTED* > mail_uid = *REDACTED* > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date index ihave duplicate mime foreverypart > extracttext imapsieve vnd.dovecot.imapsieve > namespace inbox { > inbox = yes > location = > mailbox Drafts { > auto = subscribe > special_use = \Drafts > } > mailbox Junk { > auto = subscribe > autoexpunge = 30 days > special_use = \Junk > } > mailbox Sent { > auto = subscribe > special_use = \Sent > } > mailbox Trash { > auto = subscribe > autoexpunge = 30 days > special_use = \Trash > } > prefix = > separator = . > type = private > } > passdb { > args = *REDACTED* > driver = ldap > } > plugin { > imapsieve_mailbox1_before = file:*REDACTED* > imapsieve_mailbox1_causes = COPY > imapsieve_mailbox1_name = Junk > imapsieve_mailbox2_before = file:*REDACTED* > imapsieve_mailbox2_causes = COPY > imapsieve_mailbox2_from = Junk > imapsieve_mailbox2_name = * > recipient_delimiter = + > sieve = file://*REDACTED* > sieve_before = /etc/dovecot/sieve-before.d > sieve_global_extensions = +vnd.dovecot.pipe > +vnd.dovecot.environment +vnd.dovecot.report > sieve_pipe_bin_dir = /etc/dovecot/sieve-scripts > sieve_plugins = sieve_imapsieve sieve_extprograms > sieve_quota_max_storage = 50M > } > postmaster_address = *REDACTED* > protocols = " imap lmtp sieve" > service auth-worker { > user = mail > } > service auth { > unix_listener *REDACTED* { > group = postfix > mode = 0666 > user = postfix > } > } > service lmtp { > process_min_avail = 2 > unix_listener *REDACTED* { > group = postfix > mode = 0600 > user = postfix > } > user = mail > } > ssl = required > ssl_cert = <*REDACTED* > ssl_client_ca_dir = *REDACTED* > ssl_dh = # hidden, use -P to show it > ssl_key = # hidden, use -P to show it > submission_host = *REDACTED*:587 > userdb { > args = uid=8 > driver = static > } > protocol lmtp { > mail_plugins = " sieve" > } > protocol imap { > mail_plugins = " imap_sieve" > } -- Michael Gratton. <https://mjog.vee.net>

2 3

using %d as a variable in the ldap search base
by Luke Schierer 01 Sep '20

01 Sep '20

Hi, I'm trying to configure dovecot to use LDAP authentication directly, and I'm having a bit of trouble. I have a somewhat unusual setup, in that I have an LDAP directory that has 3 OUs each of which have their own set of users, some of which overlap. As I was trying to figure things out, I was setting the search base in my /etc/dovecot/dovecot-ldap.conf.ext file which is referenced by /etc/dovecot/conf.d/auth-ldap.conf.ext file to base = ou=%d,dc=thecrazyguys,dc=net however, the resulting searches against the directory endup just dropping the %d, resulting in ou=,dc=thecrazyguys,dc=net which is invalid. On a whim, I tried a search base of base = dc=%d,dc=thecrazyguys,dc=net and found that it did correctly substitute in the variable, which would be correct, except that my ldap tree is set up with OUs and not an extra DC segment. for whatever reason, it will do variable substitution for dc=%d, but not for ou=%d. this is certainly not documented, and seems like wrong behavior, since having an ou in a search base is valid. I'm including configuration information below. Please let me know if I've missed including information that is required. Thanks! Luke luke@schierer@littera001:/etc/dovecot$ lsb_release -rd Description: Ubuntu 18.04.5 LTS Release: 18.04 luke@schierer@littera001:/etc/dovecot$ luke@schierer@littera001:/etc/dovecot$ dpkg -l | grep -i dovecot ii dovecot-core 1:2.2.33.2-1ubuntu4.6 amd64 ii dovecot-imapd 1:2.2.33.2-1ubuntu4.6 amd64 ii dovecot-ldap 1:2.2.33.2-1ubuntu4.6 amd64 ii dovecot-pop3d 1:2.2.33.2-1ubuntu4.6 amd64 luke@schierer@littera001:/etc/dovecot$ root@littera001:/etc/dovecot# dovecot -n # 2.2.33.2 (d6601f4ec): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.21 (92477967) # OS: Linux 4.15.0-112-generic x86_64 Ubuntu 18.04.5 LTS auth_verbose = yes first_valid_uid = 1001 imap_hibernate_timeout = 100 secs lock_method = dotlock mail_cache_min_mail_count = 15 mail_fsync = always mail_location = maildir:~/Maildir:LAYOUT=fs:INDEX=MEMORY mail_nfs_index = yes mail_nfs_storage = yes mail_privileged_group = mail mailbox_list_index = yes maildir_broken_filename_sizes = yes maildir_very_dirty_syncs = yes mbox_min_index_size = 10 B mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocols = " imap pop3" ssl_cert = </etc/dovecot/dovecot.pem ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol imap { mail_max_userip_connections = 20 } root@littera001:/etc/dovecot# root@littera001:/etc/dovecot# cat /etc/dovecot/dovecot-ldap.conf.ext | grep -v ^# | uniq | more hosts = censor001.plerumque.thecrazyguys.net tls = yes tls_require_cert = allow debug_level = 4 auth_bind = yes base = ou=%d,dc=thecrazyguys,dc=net scope = subtree user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid user_filter = (&(objectClass=posixAccount)(uid=%n)) blocking = no root@littera001:/etc/dovecot#

2 4

Re: replicator: Panic: data stack: Out of memory
by Thomas Tsiakalakis 01 Sep '20

01 Sep '20

So nobody has any idea why this could happen? Let me know if I should provide more Information Thanks Thomas Tsiakalakis Team Applikationsbetrieb GDV Dienstleistungs-GmbH Tel: +49(40)33449-4318 Fax: E-Mail: Thomas.Tsiakalakis(a)gdv-dl.de<mailto:Thomas.Tsiakalakis@gdv-dl.de> GDV Dienstleistungs-GmbH Glockengießerwall 1 D-20095 Hamburg www.gdv-dl.de<http://www.gdv-dl.de> Niederlassungen: Wilhelmstraße 43 / 43 G 10117 Berlin Frankenstraße 18 20097 Hamburg Sitz und Registergericht: Hamburg HRB 145291 USt.-IdNr : DE 205183123 Geschäftsführer: Dr. Jens Bartenwerfer Fred di Giuseppe Chiachiarella Aufsichtsratsvorsitzender: Werner Schmidt ------------------------------------------------------------------ Diese E-Mail und alle Anhänge enthalten vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese E-Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe der E-Mail ist nicht gestattet. This e-mail and any attached files may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorised copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Von: Thomas Tsiakalakis Gesendet: Dienstag, 23. Juni 2020 11:25 An: dovecot(a)dovecot.org Betreff: AW: replicator: Panic: data stack: Out of memory I managed to convince the System that I really want a core dump Von: Thomas Tsiakalakis Gesendet: Donnerstag, 18. Juni 2020 16:26 An: 'dovecot(a)dovecot.org' <dovecot(a)dovecot.org> Betreff: replicator: Panic: data stack: Out of memory Hi, I have set up 2 SLES 15 Hosts with dovecot and replication. Everything seems to work fine but every time a message is delivered, I get an out of memory error in the logs. The Replication itself seems to work fine though. I increased default_vsz_limit to 512M but the only thing that changed was that dovecot was trying to allocate 1073741864 bytes instead of 268435496 As I said, I’m running SLES 15 SP1 and Dovecot 2.3.10 (0da0eff44) (both Hosts have the same version) Each Host currently has 8GB of Memory. # free -h total used free shared buff/cache available Mem: 7.8Gi 278Mi 7.3Gi 47Mi 183Mi 7.2Gi Swap: 4.0Gi 0B 4.0Gi # journalctl -f Jun 18 15:55:48 mail1 postfix/pickup[3457]: 18533C009C8: uid=0 from=<root> Jun 18 15:55:48 mail1 postfix/cleanup[3669]: 18533C009C8: message-id=<20200618135548.18533C009C8@mail1> Jun 18 15:55:48 mail1 postfix/qmgr[3458]: 18533C009C8: from=<root@mail1>, size=431, nrcpt=1 (queue active) Jun 18 15:55:48 mail1 dovecot[1833]: lmtp(3673): Connect from local Jun 18 15:55:48 mail1 dovecot[1833]: replicator: Panic: data stack: Out of memory when allocating 268435496 bytes Jun 18 15:55:48 mail1 dovecot[1833]: replicator: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(backtrace_append+0x42) [0x7f8346d6d262] -> /usr/lib64/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7f8346d6d37e] -> /usr/lib64/dovecot/libdovecot.so.0(+0xebcee) [0x7f8346d77cee] -> /usr/lib64/dovecot/libdovecot.so.0(+0xebd91) [0x7f8346d77d91] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f8346cd1f8b] -> /usr/lib64/dovecot/libdovecot.so.0(t_pop_last_unsafe+0) [0x7f8346d73660] -> /usr/lib64/dovecot/libdovecot.so.0(+0xe7a50) [0x7f8346d73a50] -> /usr/lib64/dovecot/libdovecot.so.0(+0x10eb58) [0x7f8346d9ab58] -> /usr/lib64/dovecot/libdovecot.so.0(+0xe3ac5) [0x7f8346d6fac5] -> /usr/lib64/dovecot/libdovecot.so.0(buffer_write+0xf2) [0x7f8346d6fdb2] -> dovecot/replicator(replicator_queue_push+0xde) [0x5648989a5cde] -> dovecot/replicator(+0x5464) [0x5648989a5464] -> dovecot/replicator(+0x4a0b) [0x5648989a4a0b] -> dovecot/replicator(+0x4c02) [0x5648989a4c02] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7f8346d90c99] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x134) [0x7f8346d92574] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7f8346d90d9c] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f8346d90fc8] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f8346d01873] -> dovecot/replicator(main+0x1a0) [0x5648989a3cb0] -> /lib64/libc.so.6(__libc_start_main+0xea) [0x7f83468f534a] -> dovecot/replicator(_start+0x2a) [0x5648989a3d5a] Jun 18 15:55:48 mail1 dovecot[1833]: lmtp(test(a)example.com)<3673><q5sLCGRy615ZDgAATn/NZw>: Warning: replication(test(a)example.com): Sync failure: Jun 18 15:55:48 mail1 dovecot[1833]: lmtp(test(a)example.com)<3673><q5sLCGRy615ZDgAATn/NZw>: Warning: replication(test(a)example.com): Remote sent invalid input: - Jun 18 15:55:48 mail1 dovecot[1833]: lmtp(test(a)example.com)<3673><q5sLCGRy615ZDgAATn/NZw>: msgid=<20200618135548.18533C009C8@mail1>: saved mail to INBOX Jun 18 15:55:48 mail1 dovecot[1833]: lmtp(3673): Disconnect from local: Client has quit the connection (state=READY) Jun 18 15:55:48 mail1 postfix/lmtp[3672]: 18533C009C8: to=<test(a)example.com>, orig_to=<test(a)domain.com>, relay=mail1[private/dovecot-lmtp], delay=0.17, delays=0.02/0.01/0.02/0.13, dsn=2.0.0, status=sent (250 2.0.0 <test(a)example.com> q5sLCGRy615ZDgAATn/NZw Saved) Jun 18 15:55:48 mail1 postfix/qmgr[3458]: 18533C009C8: removed Jun 18 15:55:48 mail1 dovecot[1833]: replicator: Fatal: master: service(replicator): child 2948 killed with signal 6 (core not dumped - https://dovecot.org/bugreport.html#coredumps - set /proc/sys/fs/suid_dumpable to 2)

2 3

shutdown_clients ignored on Director upgrade/restart
by Tom Sommer 01 Sep '20

01 Sep '20

When yum updates a Dovecot Director, it restarts all processes - however a few imap-login and pop3-login processes hang because they are still proxying connections. It's like the now orphaned processes are stalled, and needs a kill -9 to die. This is a problem because those stalled connections are still connected to backends, so when new connections are made from the same account, they do not end up on the same backend and thus index corruption happens (same account, two different backends). shutdown_clients is enabled but it appears Dovecot restarts regardless of verifying all processes/clients are shut down. Just had this happen upon yum update from dovecot-2.3.11.3-3 to dovecot-2.3.11.3-4 -- Tom

1 0

how fetch zlibed mails with doveadm
by rudolf＠padaru.de 31 Aug '20

31 Aug '20

Hey there, i need some help with an administrativ dovecot question. I run a dovecot 2.2.33 ubuntu server with mdbox and "zlib_save = gz zlib_save_level = 6". Now i want to fetch the mailheader from some mails via doveadm fetch -u <user> hdr uid <uid>. The only thing what i receive is some binary stuff. Is there a way, i can tell the dovemadm fetch command, that he should deflate the output for me? Yours sincerely Pascal Rudolf

2 4

zlib errors after upgrading from 2.3.10.1 to 2.3.11.3
by Robert Nowotny 31 Aug '20

31 Aug '20

I get ZLIB Errors after dovecot upgrade from 2.3.10.1 to 2.3.11.3 Thunderbird reports some error when try to save/open a bigger attachment. Aug 21 15:27:34 lxc-imap dovecot: imap(acsida)<63870><jZk...>: Error: Mailbox Sent: UID=40826: read(zlib(/home/vmail/virtualmailboxes/acsida/storage/m.2409)) failed: read(/home/vmail/virtualmailboxes/acsida/storage/m.2409) failed: Broken pipe (FETCH BODY[]) Aug 21 15:27:34 lxc-imap dovecot: imap(acsida)<63871><9A0...>: Error: Mailbox Sent: UID=40826: read(zlib(/home/vmail/virtualmailboxes/acsida/storage/m.2409)) failed: read(/home/vmail/virtualmailboxes/acsida/storage/m.2409) failed: Broken pipe (FETCH BODY[]) Aug 21 15:30:11 lxc-imap dovecot: imap(cpotzinger)<63889><Ig6...>: Error: Mailbox Sent: UID=29495: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Aug 21 15:30:17 lxc-imap dovecot: imap(cpotzinger)<64169><GXe...>: Error: Mailbox Sent: UID=29495: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Aug anything I can tweak or do ? Robert

3 9

PBKDF2 password hashing as in ASP.NET Core
by Yves Goergen 30 Aug '20

30 Aug '20

Hello, I'm setting up a new server and, again, seek for a decently secure (from a security specialist's POV) way to store and verify user passwords in a database. Additionally now, GDPR requires me to use a solid state-of-the-art solution. My OS is Ubuntu 20.04, Dovecot version 2.3.7, database backend with PostgreSQL 12. Obviously, storing the plaintext password is a terrible idea. SHA-based methods aren't suitable either. bcrypt has been recommended often [1]. PBKDF2 was preferred over bcrypt even more [2]. I'm managing all database contents with an ASP.NET Core application that implements the management user frontend. It's a bit hard to find bcrypt support for .NET (there are a few NuGet packages of unknown quality [3]). .NET does however implement, use and recommend PBKDF2 for its own user management. If this is by far the best way to go, I'm already covered on that side. Now the problem is, once again*, how I can use this in applications to make them as secure. I need a solution for Dovecot and Exim. Exim seems to be able to ask Dovecot (IMAP) for user authentication, so I might try that and only need to solve the problem in Dovecot alone. Dovecot documentation says that PBKDF2 is somewhat possible [4]. It requires the hash in the format "$1$salt$rounds$hash". I guess that "salt", "rounds" and "hash" are the parameters here. But what is their format? The .NET implementation [5] describes its format as "{ 0x01, prf (UInt32), iter count (UInt32), salt length (UInt32), salt, subkey }" with big-edian integers. The result is base64-encoded. Prf is an enum value, describing the HMAC SHA-256. Subkey is the hash value part. I might find a way to convert .NET's format into what Dovecot understands. The hmac is SHA-256, the rounds is 10000. But I wasn't able to get it working. My test password is: simplepassword The .NET hash: AQAAAAEAACcQAAAAEG0Anzb9vKOqsAKxLyhXedCTJoHrP381hKiKBHuPHhMdkjqW8Bks8RFOQZLssJ2grQ== The converted hash for Dovecot: $1$bQCfNv28o6qwArEvKFd50A==$10000$kyaB6z9/NYSoigR7jx4THZI6lvAZLPERTkGS7LCdoK0= I've also found the source code in Dovecot that should verify the hash [6]. It gives some more hints about the expected format that are sadly missing from the documentation, making it almost useless. I also tried with the "{PBKDF2}" prefix, with the base64 padding "=" removed and with the hash part converted from base64 to hex. Nothing works. The source mentions "SHA1" somewhere. Is that all it can accept? No up-to-date SHA-256? So what have I done wrong here? Why can't I authenticate? The Dovecot log isn't helpful, it doesn't even mention the user name I tried to log in with from Thunderbird, most of the time (it's unpredictable). What is the correct usage of Dovecot's PBKDF2 feature? Is it functional at all? There's a test case for it [7] but that's not helpful to me. If Dovecot's PBKDF2 support is not functional or not compatible with ASP.NET Core's parameters, what options do I have? Can I build my own authentication service that Dovecot can communicate with, to fill the gap of missing crypto support? Yves *) During my research today, I find myself finding my own questions from 6 years ago when I did this the last time. The situation hasn't changed much since then. Secure password hashing is still impossible or complicated in server applications like Exim or Dovecot. [1] https://codahale.com/how-to-safely-store-a-password/ (linked from the Dovecot documentation) [2] https://www.reddit.com/r/node/comments/4u1jcn/is_bcrypt_the_best_possible_p… [3] https://nugetmusthaves.com/Tag/bcrypt [4] https://doc.dovecot.org/configuration_manual/authentication/password_scheme… [5] https://github.com/dotnet/AspNetCore/blob/master/src/Identity/Extensions.Co… [6] https://github.com/dovecot/core/blob/81b5b188c478ec36bea8bda8fcad1e5f32ac61… [7] https://github.com/dovecot/core/blob/ff5305136ae747867b6f6af9a1737188ae7b3b…

2 4

moving /var/vmail/vmail1 Maildir tree to another partition?
by Voytek Eymont 30 Aug '20

30 Aug '20

I have a small Postfix/Dovecot/MariaDB on Centos 7 with a dozen virtual domains, I'm running out of space in /var but got space can I simply move the entire /var/vmail/vmail1 tree elsewhere ? like say /vmail/vmail1 is it as simple as shutting Dovecot/Postfix, move tree, edit conf and re start ? or am I missing something ..? or is the a better way of doing this ? # grep 'var/vmail' *.conf dovecot.conf: sieve_global_dir = /var/vmail/sieve dovecot.conf: sieve_global_path = /var/vmail/sieve/dovecot.sieve dovecot.conf:# location = maildir:/var/vmail/public/:CONTROL=~/Maildir/public:INDEX=~/Maildir/public # cd ../postfix # grep 'var/vmail' *.cf main.cf:virtual_mailbox_base = /var/vmail # dovecot --version 2.3.11.3 (502c39af9)

2 1

Rsync the dovecot repo
by Marc Roos 30 Aug '20

30 Aug '20

Is it possible to rsync some repo that has centos7 2.3.11? http://repo.dovecot.org/ce-2.3-latest/centos/7/RPMS/x86_64/2.3.11.3-3_ce/

2 1

Imaptest stalls when removing msgs argument
by Marc Roos 30 Aug '20

30 Aug '20

I have had this imaptest[1] stall. I compared this command with what you have on the website and I was a bit surprised that what is on the website is working. What I have noticed is that if you remove this msgs argument, the imaptest stalls. I am not sure if this is according to design, but one might expect that specifying a secs, makes specifying msgs redundant? [1] imaptest - append=100,0 logout=0 host=xx.local port=143 user=xxx pass=xxx seed=100 secs=240 clients=1 mbox=64kb.mbox box=inbox/test. [2] https://doc.dovecot.org/admin_manual/sample_tests/ Using: dovecot-imaptest-2.3.10-2.x86_64

1 0

lazy_expunge and fts_autoindex
by Gregory Heytings 29 Aug '20

29 Aug '20

Hi list, I have both lazy_expunge and fts_autoindex activated (with fts-xapian), as follows: plugin { lazy_expunge = EXPUNGED/ } plugin { fts = xapian fts_xapian = partial=2 full=20 attachments=1 verbose=0 fts_autoindex = yes fts_enforced = yes fts_autoindex_exclude = EXPUNGED fts_autoindex_exclude2 = EXPUNGED/* } However, I still see "indexer-worker...: Info: Indexed 1 messages in EXPUNGED/..." in the dovecot log each time I expunge an email. I tried various other settings for "fts_autoindex_exclude" (EXPUNGED alone, EXPUNGED + EXPUNGED/ + EXPUNGED/*, ...), but none of them seem to work. Thanks for your help, Gregory

2 2

Expuning & Purging doesn't fully remove emails?
by Zelic Bojan 27 Aug '20

27 Aug '20

Hello, I'm facing an issue where deleted emails keep re-appearing after my mailbox index gets recreated. I'm running version 2.2.36 of dovecot, but I tested the same scenario under 2.3.10. I'm also using mdbox, autoexpunge, and using dovecot replication. I've had several instances now where some expunged emails show up again in a mailbox. I noticed this error: doveadm: Error: Corrupted dbox file /var/mail/virtual/mailbox(a)domain.com/mdbox/storage/m.3228 (around offset=1988744): msg header has bad magic value which caused the index to get rebuilt... however several times now, the indexes got rebuilt even though there doesnt seem like there was any error... so im not sure why that is. lmtp(13910): Warning: fscking index file /var/mail/virtual/mailbox(a)domain/mdbox/storage/dovecot.map.index lmtp(13910): Warning: fscking index file /var/mail/virtual/mailbox(a)domain/mdbox/storage/dovecot.map.index lmtp(13910): Warning: mdbox /var/mail/virtual/mailbox@domain/mdbox/storage: rebuilding indexes I'm not sure why these mails keep coming back though... or if there's anything that I can do to limit the number of emails that get restored. I want to make sure expunged & purged emails stay expunged & purged. If I run a purge and then force index recreation... why would expunged emails come back? Shouldn't I expect them all to be deleted & purged? Does expunge not expunge all emails? (In production, I'm running autoexpunge but this test below shows what happens when I attempt to expunge everything) doveadm search -u email@domain all | wc -l # output 22096 doveadm expunge -u email@domain mailbox '*' all doveadm search -u email@domain all | wc -l # output: 0 doveadm purge -u email@domain doveadm dump /var/mail/virtual/email@domain/mdbox/storage/ | grep -c 'ref.*\b0\b' # output: 0 doveadm force-resync -u email@domain Inbox # output: # doveadm(email@domain): Warning: fscking index file /var/mail/virtual/email(a)domain/mdbox/storage/dovecot.map.index # doveadm(email@domain): Warning: mdbox /var/mail/virtual/email@domain/mdbox/storage: rebuilding indexes # doveadm(email@domain): Warning: fscking index file /var/mail/virtual/email(a)domain/mdbox/storage/dovecot.map.index doveadm search -u email@domain all | wc -l # output: 843 I would expect the output to be 0. Theoretically I deleted all emails and purged all emails. Nothing should be left on the disk? However... I can see there are still m.* files in mdbox/storage for the mailbox. Overall, I'm not sure why the index got recreated... but I'm trying to limit the impact of mailbox corruption so that deleted emails do not come back if the index is somehow recreated again. If I were to re-run expunge, purge, and force-resync a 2nd time... it does get emptied out, but I'm not looking to run force-resync intentially since it causes dataloss with mdbox, and re-running only expunge & purge doesn't seem to do anything. Bojan Zelic Sr. IT Infrastructure Engineer

2 3

when are these conf.ext files loaded
by Marc Roos 27 Aug '20

27 Aug '20

Are these .conf.ext always loaded? Or only when they are configured somewhere else?

2 3