May 2021 - dovecot - dovecot.org

Panic: file ostream-zlib.c
by Bouke J. Henstra | ICT Diensten 24 May '21

24 May '21

Hello, At the moment I am investigating an issue (probably) related to Dovecot /zlib. I am hoping someone is willing to help. Thanks. #Background information Every time a certain user checked her e-mail the following errors would appear in "/var/log/mail.err" (please find the lines below). I am using zlib (please find the details in the configuration below). I did include the Dovecot version, OS details, CPU details and file system details in this message. I did migrate the whole mail domain to Microsoft 365 (as the user has an active subscription and had to move on - for me this was the fastest method to help the user in this case). The mailbox still exists on the server. I suspect there's file corruption somewhere? I am routing mail from the domain to xxxxx-xx.mail.protection.outlook.com (just to be sure that the user doesn't miss anything - mx records have been updated). I did notice that the user had issues with receiving mail. Outlook would keep trying to fetch mail and was hard to close (user had to kill Outlook through the task manager - after migrating to Microsoft 365 this problem is resolved too). I would like to find out what causes/caused this problem. I would like to understand the problem and would like to try to avoid future problems. I did run "dovecot-sysreport" but I did notice the data is not anonymized. I can anonymize the details and submit those if required. #Dovcecot version root@s1:/# dovecot --version 2.3.4.1 (f79e8e7e4) #Operating system root@s1:/# uname -a Linux s1 4.19.0-16-amd64 #1 SMP Debian 4.19.181-1 (2021-03-19) x86_64 GNU/Linux root@s1:/# cat /etc/debian_version 10.9 #CPU Architecture root@s1:/# lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian #File System ext4 errors=continue,usrjquota=quota.user,grpjquota=quota.group,jqfmt=vfsv0 0 1 #Errors May 17 22:37:11 s1 dovecot: imap(xxxx(a)xxxxx.xx)<18570><MxyEjIzCs8pSoYVo>: Panic: file ostream-zlib.c: line 37 (o_stream_zlib_close): assertion failed: (zstream->ostream.finished || zstream->ostream.ostream.stream_errno != 0 || zstream->ostream.error_handling_disabled) May 17 22:37:11 s1 dovecot: imap(xxxx(a)xxxxx.xx)<18570><MxyEjIzCs8pSoYVo>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xdb73b) [0x7f45c109973b] -> /usr/lib/dovecot/libdovecot.so.0(+0xdb7d1) [0x7f45c10997d1] -> /usr/lib/dovecot/libdovecot.so.0(+0x4a199) [0x7f45c1008199] -> /usr/lib/dovecot/modules/lib20_zlib_plugin.so(+0x3c6f) [0x7f45c0dc3c6f] -> /usr/lib/dovecot/libdovecot.so.0(+0xffd5e) [0x7f45c10bdd5e] -> /usr/lib/dovecot/libdovecot.so.0(o_stream_destroy+0x16) [0x7f45c10bdd86] -> /usr/lib/dovecot/libdovecot-storage.so.0(maildir_save_finish+0x18d) [0x7f45c11d046d] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_save_cancel+0x4d) [0x7f45c11a867d] -> dovecot/imap(+0x10f36) [0x5606b3187f36] -> dovecot/imap(+0x10ff3) [0x5606b3187ff3] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x6f) [0x7f45c10afbef] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x136) [0x7f45c10b11e6] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7f45c10afc8c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7f45c10afdf0] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f45c1030123] -> dovecot/imap(main+0x325) [0x5606b3187bf5] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7f45c0e1309b] -> dovecot/imap(_start+0x2a) [0x5606b3187d8a] May 17 22:37:11 s1 dovecot: imap(xxxx(a)xxxxx.xx)<18570><MxyEjIzCs8pSoYVo>: Fatal: master: service(imap): child 18570 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) May 17 22:54:52 s1 dovecot: imap(xxxx(a)xxxxx.xx)<20321><HrbpyozCn8tSoYVo>: Panic: file ostream-zlib.c: line 37 (o_stream_zlib_close): assertion failed: (zstream->ostream.finished || zstream->ostream.ostream.stream_errno != 0 || zstream->ostream.error_handling_disabled) May 17 22:54:52 s1 dovecot: imap(xxxx(a)xxxxx.xx)<20321><HrbpyozCn8tSoYVo>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xdb73b) [0x7fda4a55473b] -> /usr/lib/dovecot/libdovecot.so.0(+0xdb7d1) [0x7fda4a5547d1] -> /usr/lib/dovecot/libdovecot.so.0(+0x4a199) [0x7fda4a4c3199] -> /usr/lib/dovecot/modules/lib20_zlib_plugin.so(+0x3c6f) [0x7fda4a27ec6f] -> /usr/lib/dovecot/libdovecot.so.0(+0xffd5e) [0x7fda4a578d5e] -> /usr/lib/dovecot/libdovecot.so.0(o_stream_destroy+0x16) [0x7fda4a578d86] -> /usr/lib/dovecot/libdovecot-storage.so.0(maildir_save_finish+0x18d) [0x7fda4a68b46d] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_save_cancel+0x4d) [0x7fda4a66367d] -> dovecot/imap(+0x10f36) [0x5559c9e82f36] -> dovecot/imap(+0x10ff3) [0x5559c9e82ff3] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x6f) [0x7fda4a56abef] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x136) [0x7fda4a56c1e6] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7fda4a56ac8c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7fda4a56adf0] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fda4a4eb123] -> dovecot/imap(main+0x325) [0x5559c9e82bf5] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7fda4a2ce09b] -> dovecot/imap(_start+0x2a) [0x5559c9e82d8a] May 17 22:54:52 s1 dovecot: imap(xxxx(a)xxxxx.xx)<20321><HrbpyozCn8tSoYVo>: Fatal: master: service(imap): child 20321 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) #Config root@s1:/opt/scripts# dovecot -n # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.19.0-16-amd64 x86_64 Debian 10.9 # Hostname: s1.xxxxx.xx auth_mechanisms = plain login disable_plaintext_auth = no imap_capability = +SPECIAL-USE XLIST listen = *,[::] lmtp_rcpt_check_quota = yes log_timestamp = "%Y-%m-%d %H:%M:%S " mail_max_userip_connections = 100 mail_plugins = quota zlib mail_privileged_group = vmail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = . } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { quota = dict:user::file:/var/vmail/%d/%n/.quotausage quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is full quota_status_success = DUNNO sieve = /var/vmail/%d/%n/.sieve sieve_after = /var/vmail/%d/%n/.ispconfig.sieve sieve_before = /var/vmail/%d/%n/.ispconfig-before.sieve sieve_max_actions = 100 sieve_max_redirects = 25 sieve_max_script_size = 2M zlib_save = gz zlib_save_level = 9 } protocols = imap pop3 lmtp service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = root } service imap-login { client_limit = 1000 process_limit = 512 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service quota-status { client_limit = 1 executable = quota-status -p postfix unix_listener /var/spool/postfix/private/quota-status { group = postfix mode = 0660 user = postfix } } service stats { unix_listener stats-reader { group = vmail mode = 0660 user = vmail } unix_listener stats-writer { group = vmail mode = 0660 user = vmail } } ssl_cert = </etc/postfix/smtpd.cert ssl_cipher_list = ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384 ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol imap { auth_verbose = yes mail_plugins = quota zlib quota imap_quota } protocol pop3 { auth_verbose = yes mail_plugins = quota zlib quota pop3_uidl_format = %08Xu%08Xv } protocol lda { mail_plugins = quota zlib sieve quota postmaster_address = postmaster(a)xxxxx.xx } protocol lmtp { mail_plugins = quota zlib quota sieve postmaster_address = postmaster(a)xxxxx.xx -- Kind regards, Bouke J. Henstra

2 1

BUG report
by Oleg Pyzin 23 May '21

23 May '21

Hello! I've compiled Dovecot without any error, but couldn't run it. ------------- The error is: ------------- Feb 09 19:22:50 master: Panic: kevent(EV_ADD, READ, 21) failed: Invalid argument Feb 09 19:22:50 master: Error: Raw backtrace: 0 libdovecot.0.dylib 0x00216d58 askpass + 136 -> 1 libdovecot.0.dylib 0x00216e78 backtrace_get + 36 -> 2 libdovecot.0.dylib 0x00221fe8 i_set_debug_file + 248 -> 3 libdovecot.0.dylib 0x002220fc default_fatal_handler + 56 -> 4 dovecot 0x000039a0 dup2_append + 1884 -> 5 libdovecot.0.dylib 0x00221a8c i_panic + 120 -> 6 libdovecot.0.dylib 0x0022c13c io_loop_handle_add + 244 -> 7 libdovecot.0.dylib 0x0022f2c8 timeout_add_short + 416 -> 8 libdovecot.0.dylib 0x0022f574 io_add_to + 76 -> 9 dovecot 0x00007dd4 service_monitor_stop_close + 288 -> 10 dovecot 0x00008cdc services_monitor_start + 840 -> 11 dovecot 0x00004aa4 main + 3336 -> 12 dovecot 0x00003048 start + 68 ------------- dovecot -n: admin.MacPPC[/opt/local/var/log/dovecot] <467-7> $ sudo dovecot -n # 2.3.11.3 (): /opt/local/etc/dovecot/dovecot.conf # OS: Darwin 9.8.0 Power Macintosh hfs # Hostname: MacPPC.Dlink default_client_limit = 50 default_internal_group = _dovecot default_process_limit = 10 default_vsz_limit = 128 M info_log_path = /opt/local/var/log/dovecot/info listen = * log_path = /opt/local/var/log/dovecot/error login_plugin_dir = /opt/local/lib/dovecot mail_location = maildir:/private/srv/mail/%u namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } protocols = imap pop3 service imap { process_limit = 32 } service pop3 { process_limit = 32 } service submission { process_limit = 32 } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = # hidden, use -P to show it userdb { driver = passwd } ------------------------------ Will be a very gratefull for the help, Oleg

1 0

Issue with dovecot v2.3.4.1 and mail_crypt plugin. Email received, but not encrypted on disk until moved out of root Maildir folder. IMAP child crashes when mail is read from folder message was moved to.
by Ben Burk 23 May '21

23 May '21

Recently I attempted enabling the mail_crypt plugin on a per user basis as summarized below: uname -a Linux smtp 4.19.0-16-amd64 #1 SMP Debian 4.19.181-1 (2021-03-19) x86_64 GNU/Linux dovecot --version 2.3.4.1 (f79e8e7e4) $ cat /etc/dovecot/conf.d/10-mailcrypt.conf mail_attribute_dict = file:%h/Maildir/dovecot-attributes mail_plugins = $mail_plugins mail_crypt plugin { mail_crypt_curve = secp521r1 mail_crypt_save_version = 2 mail_crypt_require_encrypted_user_key = yes } cat dovecot-sql.conf.ext driver = mysql connect = host=127.0.0.1 dbname=VVVVV user=WWWWWW password=XXXXXX user_query = SELECT user, \ concat('*:bytes=', quota) AS quota_rule, \ '/mail/%d/%n' AS home, \ uid, gid \ FROM virtual_users WHERE user='%u'; password_query = SELECT user, password, crypt as userdb_mail_crypt_save_version, '%w' AS userdb_mail_crypt_private_password FROM virtual_users WHERE user='%u'; dovecot -n # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.19.0-16-amd64 x86_64 Debian 10.9 # Hostname: smtp.burk.tech auth_verbose = yes mail_attribute_dict = file:%h/Maildir/dovecot-attributes mail_location = maildir:~/Maildir:LAYOUT=fs mail_plugins = quota mail_crypt mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = type = private } passdb { driver = pam } plugin { mail_crypt_curve = secp521r1 mail_crypt_require_encrypted_user_key = # hidden, use -P to show it mail_crypt_save_version = 2 } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0777 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl = required ssl_cert = </etc/dovecot/fullchain.cer.pem ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 ssl_prefer_server_ciphers = yes userdb { driver = passwd } verbose_ssl = yes protocol lmtp { hostname = smtp.burk.tech postmaster_address = postmaster(a)burk.tech } protocol imap { mail_max_userip_connections = 50 } The issue I've noticed occurs when the following settings above are enabled and mail is received into the root of the Maildir fs, then mail is moved to another folder (including a file being deleted). Upon a client's retrieval of messages from the folder where mail was moved to, the log spits out the following: May 22 19:18:15 smtp dovecot: imap-login: Login: user=<ben>, method=PLAIN, rip=XXX.XXX.XXX.XXX, lip=YYY.YYY.YYY.YYY, mpid=31682, TLS, session=<7oASZPPCkN9EdIVp> May 22 19:18:15 smtp dovecot: imap(ben)<31682><7oASZPPCkN9EdIVp>: Error: Mailbox Trash: Saving mail: save: read() failed: read(/mail/burk.tech/ben/Maildir/Receipts/cur/1621724316.M515979P30417.smtp,S=2903,W=2964:2,S) failed: Private key not available: Cannot decrypt key f9e7a6b9dd24535a29915fb46336047b07ae5b9fea5eda8e03cf9a8fa009f88b: Cannot decrypt key 876c6bfbc355a86c3c924a5841bb2e5c6e1b0c5496d5116c4fcb9c511f803c87: Password not available May 22 19:18:15 smtp dovecot: imap(ben)<31682><7oASZPPCkN9EdIVp>: Error: Mailbox Trash: copy: i_stream_read() failed: read(/mail/burk.tech/ben/Maildir/Receipts/cur/1621724316.M515979P30417.smtp,S=2903,W=2964:2,S) failed: Private key not available: Cannot decrypt key f9e7a6b9dd24535a29915fb46336047b07ae5b9fea5eda8e03cf9a8fa009f88b: Cannot decrypt key 876c6bfbc355a86c3c924a5841bb2e5c6e1b0c5496d5116c4fcb9c511f803c87: Password not available May 22 19:18:15 smtp dovecot: imap(ben)<31682><7oASZPPCkN9EdIVp>: Error: Mailbox Trash: Saving mail: read() failed: read(/mail/burk.tech/ben/Maildir/Receipts/cur/1621724316.M515979P30417.smtp,S=2903,W=2964:2,S) failed: Private key not available: Cannot decrypt key f9e7a6b9dd24535a29915fb46336047b07ae5b9fea5eda8e03cf9a8fa009f88b: Cannot decrypt key 876c6bfbc355a86c3c924a5841bb2e5c6e1b0c5496d5116c4fcb9c511f803c87: Password not available (read reason=) May 22 19:18:15 smtp dovecot: imap(ben)<31682><7oASZPPCkN9EdIVp>: Panic: file ostream-encrypt.c: line 631 (o_stream_encrypt_close): assertion failed: (estream->finalized || estream->ctx_sym == NULL || estream->ostream.ostream.stream_errno != 0) May 22 19:18:15 smtp dovecot: imap(ben)<31682><7oASZPPCkN9EdIVp>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xdb73b) [0x7fdb3ce7273b] -> /usr/lib/dovecot/libdovecot.so.0(+0xdb7d1) [0x7fdb3ce727d1] -> /usr/lib/dovecot/libdovecot.so.0(+0x4a199) [0x7fdb3cde1199] -> /usr/lib/dovecot/libdovecot.so.0(+0x49276) [0x7fdb3cde0276] -> /usr/lib/dovecot/libdovecot.so.0(+0xffd5e) [0x7fdb3ce96d5e] -> /usr/lib/dovecot/libdovecot.so.0(o_stream_destroy+0x16) [0x7fdb3ce96d86] -> /usr/lib/dovecot/libdovecot-storage.so.0(maildir_save_finish+0x18d) [0x7fdb3cfa946d] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_save_cancel+0x4d) [0x7fdb3cf8167d] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_storage_copy+0x122) [0x7fdb3cf73f22] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x5badc) [0x7fdb3cf81adc] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_move+0x31) [0x7fdb3cf81c11] -> dovecot/imap(+0x128f5) [0x563bd267a8f5] -> dovecot/imap(command_exec+0x70) [0x563bd2687dc0] -> dovecot/imap(+0x1e3f2) [0x563bd26863f2] -> dovecot/imap(+0x1e494) [0x563bd2686494] -> dovecot/imap(client_handle_input+0x1b5) [0x563bd2686845] -> dovecot/imap(client_input+0x7e) [0x563bd2686d6e] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x6f) [0x7fdb3ce88bef] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x136) [0x7fdb3ce8a1e6] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7fdb3ce88c8c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7fdb3ce88df0] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fdb3ce09123] -> dovecot/imap(main+0x325) [0x563bd2678bf5] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7fdb3cbf209b] -> dovecot/imap(_start+0x2a) [0x563bd2678d8a] May 22 19:18:15 smtp dovecot: imap-login: Debug: SSL alert: close notify May 22 19:18:15 smtp dovecot: imap(ben)<31682><7oASZPPCkN9EdIVp>: Fatal: master: service(imap): child 31682 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) My keys for the user are as follows: ben(a)smtp.burk.tech :~$ doveadm mailbox cryptokey list -U doveadm(ben): Error: net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied Folder Active Public ID yes 876c6bfbc355a86c3c924a5841bb2e5c6e1b0c5496d5116c4fcb9c511f803c87 I've checked and it appears that the mail as its received into the root of Maildir is not initially encrypted. It is stored plaintext. However, when the mail is moved, it is then properly encrypted. ben(a)smtp.burk.tech :~/Maildir/cur$ file 1621726534.31820_1.smtp\:2\, 1621726534.31820_1.smtp:2,: SMTP mail, ASCII text ben(a)smtp.burk.tech :~/Maildir/Receipts/cur$ file 1621726595.M343917P30832.smtp\,S\=2927\,W\=2988\:2\,S 1621726595.M343917P30832.smtp,S=2927,W=2988:2,S: MailCrypt -- Ben Burk BURK.TECH System Administrator

1 0

Dovecot Subfolder Always Uses dot separator with mdbox
by Steven Smith 22 May '21

22 May '21

I’m trying to configure dovecot to use mailbox subfolders that look like both foo/bar on the IMAP client and the filesystem. But the subfolders created all look like foo.bar, not foo/bar with a proper nesting structure. This also happens when the folder ‘bar’ is moved into the folder ‘foo’—A new folder `foo.bar’ is created. I believe that i’m following the configuration rules at https://wiki.dovecot.org/MailboxFormat/Maildir#Maildir_and_filesystems and https://wiki.dovecot.org/MailLocation/Maildir. This issue happens whether I set separator to its default, ‘.’, or ‘/’: > mail_location = mdbox:/private/var/mail/tld.hostname.mail/%Ln/mdbox:LAYOUT=fs > namespace inbox { > … > separator = / > … > } The IMAP client is macOS Mail.app version 14.0. I’d greatly appreciate any help or pointers to fix this configuration issue. dovecot -n > # 2.3.13 (): /opt/local/etc/dovecot/dovecot.conf > # Pigeonhole version 0.5.13 () > # OS: Darwin 20.4.0 x86_64 apfs > # Hostname: mail.hostname.tld > aps_topic = com.apple.mail.XServer.ffffffff-ffff-fff-ffffffffff > auth_cache_size = 10 M > auth_gssapi_hostname = $ALL > auth_krb5_keytab = /opt/local/etc/dovecot/imap.keytab > auth_mechanisms = plain gssapi > auth_realms =mail.hostname.tld > auth_socket_path = /opt/local/var/run/dovecot/auth-userdb > auth_username_format = %Ln > debug_log_path = /opt/local/var/log/mail/mail-debug.log > disable_plaintext_auth = no > first_valid_gid = 6 > first_valid_uid = 6 > imap_id_log = * > imap_id_send = "name" * "version" * > imap_idle_notify_interval = 29 mins > imap_urlauth_submit_user = submit > info_log_path = /opt/local/var/log/mail/mail-info.log > last_valid_gid = 100 > lda_mailbox_autocreate = yes > log_path = /opt/local/var/log/mail/mail-err.log > login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c > mail_access_groups = mail > mail_attachment_dir = /private/var/mail/tld.hostname.mail/attachments > mail_attachment_fs = sis posix:mode=0666 > mail_gid = mail > mail_home = /private/var/mail/tld.hostname.mail > mail_location = mdbox:/private/var/mail/tld.hostname.mail/%Ln/mdbox:LAYOUT=fs > mail_log_prefix = "%s(pid %p user %u): " > mail_plugins = quota zlib acl fts fts_solr fts_lucene > mail_privileged_group = mail > mail_uid = _dovecot > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve > mdbox_rotate_size = 200 M > namespace inbox { > inbox = yes > location = > mailbox Archive { > auto = subscribe > special_use = \Archive > } > mailbox Drafts { > auto = subscribe > special_use = \Drafts > } > mailbox Junk { > auto = create > special_use = \Junk > } > mailbox Notspam_train { > auto = create > special_use = \Junk > } > mailbox Sent { > auto = subscribe > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Spam_train { > auto = create > special_use = \Junk > } > mailbox Trash { > auto = create > special_use = \Trash > } > prefix = > separator = / > } > passdb { > driver = pam > name = pam > } > plugin { > fts = solr > fts_autoindex = yes > fts_autoindex_exclude = \Junk > fts_solr = url=http://127.0.0.1:8983/solr/dovecot/ > imapsieve_mailbox1_before = file:/opt/local/etc/dovecot/sieve/report-spam.sieve > imapsieve_mailbox1_causes = COPY APPEND > imapsieve_mailbox1_name = Spam_train > imapsieve_mailbox2_before = file:/opt/local/etc/dovecot/sieve/report-ham.sieve > imapsieve_mailbox2_causes = COPY APPEND > imapsieve_mailbox2_name = Notspam_train > mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename > mail_log_fields = uid box msgid from subject size flags > quota_grace = 10%% > quota_rule = *:storage=16G > quota_rule2 = Trash:storage=+256M > quota_warning = storage=100%% quota-exceeded %u > quota_warning2 = storage=80%% quota-warning %u > recipient_delimiter = + > sieve = /private/var/mail/tld.hostname.mail/rules/%Ln/dovecot.sieve > sieve_after = /opt/local/etc/dovecot/sieve-after.d > sieve_before = /opt/local/etc/dovecot/sieve-before.d > sieve_dir = /private/var/mail/tld.hostname.mail/rules/%Ln/%u > sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment > sieve_pipe_bin_dir = /opt/local/etc/dovecot/sieve > sieve_plugins = sieve_imapsieve sieve_extprograms > sieve_quota_max_storage = 50M > } > postmaster_address = postmaster(a)hostname.tld > protocols = imap lmtp sieve > quota_full_tempfail = yes > sendmail_path = /opt/local/sbin/sendmail > service auth-worker { > group = mail > user = root > } > service auth { > extra_groups = _keytabusers > idle_kill = 15 mins > unix_listener /opt/local/var/spool/postfix/private/auth { > group = mail > mode = 0660 > user = _postfix > } > } > service dns-client { > unix_listener dns-client { > mode = 0600 > } > } > service imap-login { > inet_listener imap { > address = 127.0.0.1, ::1 > port = 143 > } > inet_listener imaps { > port = 993 > ssl = yes > } > process_min_avail = 6 > service_count = 0 > vsz_limit = 2 G > } > service imap { > process_limit = 200 > process_min_avail = 6 > } > service lmtp { > unix_listener /opt/local/var/spool/postfix/private/dovecot-lmtp { > group = mail > mode = 0660 > user = _postfix > } > } > ssl = required > ssl_ca = </opt/local/etc/certificates/cert.chain.pem > ssl_cert = </opt/local/etc/certificates/cert.cert.pem > ssl_cipher_list = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256 > ssl_dh = # hidden, use -P to show it > ssl_key = # hidden, use -P to show it > ssl_min_protocol = TLSv1.2 > ssl_prefer_server_ciphers = yes > userdb { > driver = passwd > name = passwd > } > verbose_proctitle = yes > protocol lda { > mail_fsync = optimized > mail_plugins = quota zlib acl fts fts_solr fts_lucene sieve push_notify > } > protocol imap { > mail_max_userip_connections = 50 > mail_plugins = quota zlib acl fts fts_solr fts_lucene imap_acl imap_quota imap_zlib imap_sieve > } > protocol lmtp { > mail_fsync = optimized > mail_plugins = quota zlib acl fts fts_solr fts_lucene sieve > }

1 0

Dovecot v2.3.13 reporting (very) incorrect vsize for some maildir folders
by Eirik Rye 22 May '21

22 May '21

Hello! Dovecot v2.3.13. Full, but anonymized, `doveconf -n` attached. We are in the process of migrating maildir-backed users from filesystem quotes, to using dovecot's "count" quota backend. This is not reflected in the doveconf output because we're overriding `quota`, `quota_rule` and `quota_vsizes` in userdb: {"quota":"count:User quota","quota_rule":"*:storage=15G","quota_vsizes":"yes"} Afterwards, for a couple users, we received reports that mail delivery had failed because they were over quota. When looking into it for a specific user, we noticed that the vsize reported for a particular folder (with 47k messages) was reported as being nearly 50 times larger than it's on-disk size: root@mail02:~# doveadm mailbox status -u anonymized_user 'messages recent unseen vsize' 'anonymized/folder/name' [...] anonymized/folder/name messages=47338 recent=0 unseen=0 vsize=14335366070 However, the filesystem itself reports a much smaller size (but correct message count): root@mail02:~# du -hd1 /home/anonymized_user/Maildir/.anonymized.folder.name/ 313M /home/anonymized_user/Maildir/.anonymized.folder.name/cur 36K /home/anonymized_user/Maildir/.anonymized.folder.name/tmp 4.0K /home/anonymized_user/Maildir/.anonymized.folder.name/new 320M /home/anonymized_user/Maildir/.anonymized.folder.name/ root@mail02:~# ls /home/anonymized_user/Maildir/.anonymized.folder.name/cur | wc -l 47338 I have tried: - `doveadm force-resync -u anonymized_user` - deleting the index files in the specific folder, and running `doveadm index -u anonymized_user '*'` as well as `doveadm mailbox status -u anonymized_user vsize '*'` - deleting all `*index*` files in maildir, and running `doveadm index -u anonymized_user '*'` as well as `doveadm mailbox status -u anonymized_user vsize '*'` - comparing all maildir-sizes (S=) with their actual sizes to see if there are discrepancies. There are none. - regardless of the above, set `maildir_broken_filename_sizes = yes`, deleted indexes and reindexed. Even after deleting the list index (dovecot.list.index) as well as the mailbox indexes, recalculating the vsize seems very quick so I feel like the incorrect vsize is being fetched from a cache somewhere instead of being recalculated..? Any idea what is causing dovecot to report this wildly incorrect mailbox size? Are there any other files (not matching `*index*`) that are responsible? I'd rather not touch the user's control files, for obvious reasons. On a side note: We're also experiencing some issues with `mail_vsize_bg_after_count`. It seems to work correctly by returning a temporary error and deferring to a background job, but the indexer-worker job often times never appears to start (or do anything) at all: dovecot 29622 0.0 0.0 4120 1124 ? S 15:22 0:00 dovecot/indexer [0 clients, 0 requests] 266248 29623 0.0 0.0 6008 4604 ? S 15:22 0:00 dovecot/indexer-worker [idling] Waiting a while, and attempting to refetch the quota returns the same temperror with no indexer-worker being started. However, this is a separate issue from the above and can be ignored for now. Best regards, Eirik Rye

4 11

director & backend on same server with director_proxy_maybe
by lucas.raynaud＠ircf.fr 21 May '21

21 May '21

Hello, I've 2 mail servers, as test server, with dovecot as backend and setup pop3 and imap connections and NFS. For prevent issues with simultaneous connections I want to setup up Director on the same VM and on the same instance as the backend Dovecot. I found in the documentation that this configuration is broken, but can work with development, and after research on the internet and on the questions encountered previously as on : https://dovecot.org/pipermail/dovecot/2015-July/101483.html https://dovecot.org/pipermail/dovecot/2011-September/130939.html https://dovecot.org/pipermail/dovecot/2012-June/136535.html I find it quite unclear if this is really possible with director_proxy_maybe, which according to the changelog, allows this setup to work. Today I've reached this level with those errors: IPs : mail4 : XX.XX.111.8 mail3 : XX.XX.51.247 doveadm director ring status director ip port type last failed status // on mail3 XX.XX.51.247 9090 self never synced XX.XX.111.8 9090 l+r never synced doveadm director status lucas.raynaud(a)ircf.fr Current: 5.196.111.8 (expires 2021-05-21 10:48:51) Hashed: 5.196.111.8 Initial config: login with lucas.raynaud(a)ircf.fr on mail4 mail log on mail4 May 21 10:51:52 mail4 dovecot: auth-worker(12917): Debug: sql(lucas.raynaud(a)ircf.fr,XX.XX.111.8,<4h3SK9PCYsIFxG8I>): query: SELECT password, 'y' as proxy_maybe, 'y' as director_proxy_maybe FROM mail_user WHERE (login = 'lucas.raynaud(a)ircf.fr' OR email = 'lucas.raynaud(a)ircf.fr') AND disableimap = 'n' AND server_id = '8' May 21 10:51:52 mail4 dovecot: auth: Debug: client passdb out: OK#0111#011user=lucas.raynaud@ircf.fr#011director_proxy_maybe=y#011lip=XX.XX.111.8#011lport=143#011proxy#011pass=x9Im.bprP!CWzbgQ May 21 10:51:52 mail4 dovecot: imap-login: Debug: Ignoring unknown passdb extra field: director_proxy_maybe May 21 10:51:52 mail4 dovecot: imap-login: Debug: Ignoring unknown passdb extra field: lip May 21 10:51:52 mail4 dovecot: imap-login: Debug: Ignoring unknown passdb extra field: lport May 21 10:51:52 mail4 dovecot: imap-login: Error: proxy: host not given: user=<lucas.raynaud(a)ircf.fr>, method=PLAIN, rip=XX.XX.111.8, lip=XX.XX.111.8, secured, session=<4h3SK9PCYsIFxG8I> May 21 10:51:52 mail4 dovecot: imap-login: Disconnected (internal failure, 1 successful auths): user=<lucas.raynaud(a)ircf.fr>, method=PLAIN, rip=XX.XX.111.8, lip=XX.XX.111.8, secured, session=<4h3SK9PCYsIFxG8I> no log on mail3 login with lucas.raynaud(a)ircf.fr on mail3 mail log on mail3 May 21 10:55:07 mail3 dovecot: auth-worker(19907): Debug: sql(lucas.raynaud(a)ircf.fr,XX.XX.51.247,<Hh5yN9PCRtAFxDP3>): query: SELECT password, 'y' as proxy_maybe, 'y' as director_proxy_maybe FROM mail_user WHERE (login = 'lucas.raynaud(a)ircf.fr' OR email = 'lucas.raynaud(a)ircf.fr') AND disableimap = 'n' AND server_id = '8' May 21 10:55:07 mail3 dovecot: auth: Debug: client passdb out: OK#0111#011user=lucas.raynaud@ircf.fr#011director_proxy_maybe=y#011lip=XX.XX.51.247#011lport=143#011proxy#011pass=x9Im.bprP!CWzbgQ May 21 10:55:07 mail3 dovecot: imap-login: Debug: Ignoring unknown passdb extra field: director_proxy_maybe May 21 10:55:07 mail3 dovecot: imap-login: Debug: Ignoring unknown passdb extra field: lip May 21 10:55:07 mail3 dovecot: imap-login: Debug: Ignoring unknown passdb extra field: lport May 21 10:55:07 mail3 dovecot: imap-login: proxy(lucas.raynaud(a)ircf.fr): Login failed to XX.XX.111.8:143: [UNAVAILABLE] Account is temporarily unavailable.: user=<lucas.raynaud(a)ircf.fr>, method=PLAIN, rip=XX.XX.51.247, lip=XX.XX.51.247, secured, session=<Hh5yN9PCRtAFxDP3> May 21 10:55:07 mail3 dovecot: imap-login: Disconnected (proxy dest auth failed): user=<lucas.raynaud(a)ircf.fr>, method=PLAIN, rip=XX.XX.51.247, lip=XX.XX.51.247, secured, session=<Hh5yN9PCRtAFxDP3> mail log on mail4 May 21 10:55:07 mail4 dovecot: auth-worker(13096): Debug: sql(lucas.raynaud(a)ircf.fr,XX.XX.51.247,<rI10N9PCyKsFxDP3>): query: SELECT password, 'y' as proxy_maybe, 'y' as director_proxy_maybe FROM mail_user WHERE (login = 'lucas.raynaud(a)ircf.fr' OR email = 'lucas.raynaud(a)ircf.fr') AND disableimap = 'n' AND server_id = '8' May 21 10:55:07 mail4 dovecot: auth: Debug: client passdb out: OK#0111#011user=lucas.raynaud@ircf.fr#011director_proxy_maybe=y#011lip=XX.XX.111.8#011lport=143#011proxy#011pass=x9Im.bprP!CWzbgQ May 21 10:55:07 mail4 dovecot: imap-login: Debug: Ignoring unknown passdb extra field: director_proxy_maybe May 21 10:55:07 mail4 dovecot: imap-login: Debug: Ignoring unknown passdb extra field: lip May 21 10:55:07 mail4 dovecot: imap-login: Debug: Ignoring unknown passdb extra field: lport May 21 10:55:07 mail4 dovecot: imap-login: Error: proxy: host not given: user=<lucas.raynaud(a)ircf.fr>, method=PLAIN, rip=XX.XX.51.247, lip=XX.XX.111.8, session=<rI10N9PCyKsFxDP3> May 21 10:55:07 mail4 dovecot: imap-login: Disconnected (internal failure, 1 successful auths): user=<lucas.raynaud(a)ircf.fr>, method=PLAIN, rip=XX.XX.51.247, lip=XX.XX.111.8, session=<rI10N9PCyKsFxDP3> sql query : password_query = SELECT password, 'y' as proxy_maybe, 'y' as director_proxy_maybe FROM mail_user WHERE (login = '%u' OR email = '%u') AND disable%Ls = 'n' AND server_id = '8' It seems that Director is working but not adding the "host" extra field when it should, also it's strange that the director_proxy_maybe is unknown. Is it really possible to configure director on same instance as backend? Is director_proxy_maybe working? Also do you have answers about my problems? Thanks for advance. Lucas ############ mail3's and mail4's configs are the same. dovecot -n # 2.2.27 (): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 () # OS: Linux 4.9.0-14-amd64 x86_64 Debian 9.13 auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login auth_proxy_self = XX.XX.111.8 auth_verbose = yes dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } director_mail_servers = XX.XX.111.8 XX.XX.51.247 director_servers = XX.XX.111.8 XX.XX.51.247 director_user_expire = 5 mins disable_plaintext_auth = no doveadm_port = 24245 mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_plugins = " quota" mail_privileged_group = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:user::file:/var/vmail/%d/%n/.quotausage quota_warning = storage=90%% quota-warning 90 %u sieve = /var/vmail/%d/%n/.sieve sieve_after = /etc/dovecot/sieve/after.d sieve_before = /var/vmail/%d/%n/sieve sieve_dir = /var/vmail/%d/%n/sieve/ sieve_global_dir = /etc/dovecot/sieve } pop3_reuse_xuidl = yes pop3_save_uidl = yes pop3_uidl_format = UID%u-%v protocols = " imap sieve pop3" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service director { fifo_listener login/proxy-notify { mode = 0666 user = $default_login_user } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service doveadm { inet_listener { port = 24245 } } service imap-login { executable = imap-login director inet_listener imap { port = 143 } } service ipc { unix_listener ipc { user = dovecot } } service pop3-login { executable = pop3-login director inet_listener pop3 { port = 110 } } service quota-warning { executable = script /usr/local/bin/mail-scripts/quota-warning.sh unix_listener quota-warning { group = vmail mode = 0660 user = vmail } user = vmail } ssl_cert = </opt/ssl/ircf.crt ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { auth_socket_path = director-userdb } protocol doveadm { auth_socket_path = director-userdb } protocol lda { mail_plugins = " quota sieve quota" } protocol imap { auth_socket_path = director-userdb mail_max_userip_connections = 16 mail_plugins = " quota quota imap_quota" } protocol pop3 { auth_socket_path = director-userdb mail_max_userip_connections = 16 mail_plugins = " quota quota" } local 10.10.10.0/24 { doveadm_password = # hidden, use -P to show it }

1 0

Re: WG: blocking - sql(acastus@example.ch,IP): unknown user
by mauric＠gmx.ch 20 May '21

20 May '21

>Hi! >You can try auth_policy_server such as weakforced. See <https://github.com/PowerDNS/weakforced> https://github.com/PowerDNS/weakforced and <https://doc.dovecot.org/configuration_manual/authentication/auth_policy/#au thentication-auth-policy> https://doc.dovecot.org/configuration_manual/authentication/auth_policy/#aut hentication-auth-policy >Aki And this Weakforced, i dont found for Debian Booster, so I have try to install but this need other meny requirements pakage, so please this are also possible as package for Debian? here any running with this this setup? I don't found any example on the net... or you using fail2ban? Mauri

1 0

Sendmail and Doecot
by Stephane Magnier 20 May '21

20 May '21

Hi, I am using Dovecot v 2.2.32 and Sendmail 8.15.2 My "folder" config on Dovecot is : mbox:/var/spool2/%u/mail:LAYOUT=fs:INBOX=/var/spool/mail/%u But in fact, any email arriving into the mailbox, stays in Sendmail into the file location (by default) : /var/spool/mail/user1 and any folder created, can be reached via Dovecot and is located to /var/spool2/user1 If I want to backup the user1's emails, I need to backup the "dovecot" location ( /var/spool2/user1) but also the Sendmail file location ( /var/spool/mail/user1) If I don't do that, I loose fresh emails which are still into the Sendmail directory, which are not copied into the Dovecot location.. Sounds weird I would expect to backup only my Dovecot directory and my email client should see only the Dovecot directory I would expect to have small sendmail email file user ( /var/spool/mail/user1 ) and i should't car about these files. Everything should be copied automatically into my Dovecot location what do you think ? Best regards, Stéve

3 2

blocking - sql(acastus@example.ch,IP): unknown user
by mauric＠gmx.ch 19 May '21

19 May '21

hello running with following Setup booster, postfix, Dovecot, mysql and meny other nice tools searching for solutions I see meny other running with same questions like this, so please what i need to read or setup that i don't receive so many failed logon try? "unknown user" i see, meny "fix" this fail2ban, but iam shure that Dovecot has also any solution for this? so friendly asking how to close this request from trying for failed logon. May 14 time Dovecot/auth-worker(8075): Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth May 14 time Dovecot/auth-worker(8075): Debug: Module loaded: /usr/lib/dovecot/modules/auth/lib20_auth_var_expand_crypt.so May 14 time Dovecot/auth-worker(8075): Debug: Module loaded: /usr/lib/dovecot/modules/auth/libdriver_mysql.so May 14 time Dovecot/auth-worker(8043): Debug: sql(acastus(a)example.ch,IP): query: SELECT username AS user, password, '/folder/example.ch/acastus' AS userdb_home, '10000' AS userdb_uid, '10000' AS userdb_gid, CONCAT('maildir:storage=', (quota*1024)) as userdb_quota FROM mailbox WHERE username='acastus(a)example.ch' AND active = 1 May 14 time Dovecot/auth-worker(8026): Info: sql(acastus(a)example.ch,IP): unknown user thanks Mauri

3 2

Separating Dovecot and Postfix
by White, Daniel E. (GSFC-770.0)[NICS] 14 May '21

14 May '21

I am struggling to update a very old set of mail servers. Some are supposed to be relays (MTAs by my understanding) while others are where the mailboxes live (MDA) In rebuilding the MDA servers, is postfix required on the same server or can dovecot connect to postfix on a separate MTA server ? The old MDAs have both installed, but the config files are a mess.

3 4