August 2020 - dovecot

Expuning & Purging doesn't fully remove emails?
by Zelic Bojan 27 Aug '20

27 Aug '20

Hello, I'm facing an issue where deleted emails keep re-appearing after my mailbox index gets recreated. I'm running version 2.2.36 of dovecot, but I tested the same scenario under 2.3.10. I'm also using mdbox, autoexpunge, and using dovecot replication. I've had several instances now where some expunged emails show up again in a mailbox. I noticed this error: doveadm: Error: Corrupted dbox file /var/mail/virtual/mailbox(a)domain.com/mdbox/storage/m.3228 (around offset=1988744): msg header has bad magic value which caused the index to get rebuilt... however several times now, the indexes got rebuilt even though there doesnt seem like there was any error... so im not sure why that is. lmtp(13910): Warning: fscking index file /var/mail/virtual/mailbox(a)domain/mdbox/storage/dovecot.map.index lmtp(13910): Warning: fscking index file /var/mail/virtual/mailbox(a)domain/mdbox/storage/dovecot.map.index lmtp(13910): Warning: mdbox /var/mail/virtual/mailbox@domain/mdbox/storage: rebuilding indexes I'm not sure why these mails keep coming back though... or if there's anything that I can do to limit the number of emails that get restored. I want to make sure expunged & purged emails stay expunged & purged. If I run a purge and then force index recreation... why would expunged emails come back? Shouldn't I expect them all to be deleted & purged? Does expunge not expunge all emails? (In production, I'm running autoexpunge but this test below shows what happens when I attempt to expunge everything) doveadm search -u email@domain all | wc -l # output 22096 doveadm expunge -u email@domain mailbox '*' all doveadm search -u email@domain all | wc -l # output: 0 doveadm purge -u email@domain doveadm dump /var/mail/virtual/email@domain/mdbox/storage/ | grep -c 'ref.*\b0\b' # output: 0 doveadm force-resync -u email@domain Inbox # output: # doveadm(email@domain): Warning: fscking index file /var/mail/virtual/email(a)domain/mdbox/storage/dovecot.map.index # doveadm(email@domain): Warning: mdbox /var/mail/virtual/email@domain/mdbox/storage: rebuilding indexes # doveadm(email@domain): Warning: fscking index file /var/mail/virtual/email(a)domain/mdbox/storage/dovecot.map.index doveadm search -u email@domain all | wc -l # output: 843 I would expect the output to be 0. Theoretically I deleted all emails and purged all emails. Nothing should be left on the disk? However... I can see there are still m.* files in mdbox/storage for the mailbox. Overall, I'm not sure why the index got recreated... but I'm trying to limit the impact of mailbox corruption so that deleted emails do not come back if the index is somehow recreated again. If I were to re-run expunge, purge, and force-resync a 2nd time... it does get emptied out, but I'm not looking to run force-resync intentially since it causes dataloss with mdbox, and re-running only expunge & purge doesn't seem to do anything. Bojan Zelic Sr. IT Infrastructure Engineer

2 3

when are these conf.ext files loaded
by Marc Roos 27 Aug '20

27 Aug '20

Are these .conf.ext always loaded? Or only when they are configured somewhere else?

2 3

Environment variable in config
by Marc Roos 26 Aug '20

26 Aug '20

I am getting errors addressing environment variables. Are they supposed to work in such sections. Not really clear from this[1] page. service health-check { # example health-check. executable = script -p /bin/health-check.sh inet_listener health-check { port = env:PORT0 } } [1] https://doc.dovecot.org/configuration_manual/config_file/config_variables/

1 0

userdb_mail_crypt_private_password: unknown variable after upgrading Dovecot
by f_ckmail＠web.de 26 Aug '20

26 Aug '20

1 0

Dovecot 2.3.11.4 - Centos 6&7 packages
by Aki Tuomi 26 Aug '20

26 Aug '20

Hi! We have pushed new versions for these packages that now support tcpwrappers. They were inadvertendly left out from last time, but now they have been restored. Sorry for the inconvenience. Regards, Aki Tuomi Open-Xchange oy

2 1

Dovecot search message problem
by Claudio Junior 25 Aug '20

25 Aug '20

Hi I have a dovecot installed on debian 10 using solr to have a better search speed in emails. I am observing after an upgrade of the debian dovecot package the error below: Aug 25 00:33:45 server01 dovecot: imap(username(a)example.com)<24104><hyaFXqutnqUAAAAAAAAAAAAAAAAAAAAB>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xdb62b) [0x7f0faeaeb62b] -> /usr/lib/dovecot/libdovecot.so.0(+0xdb6c1) [0x7f0faeaeb6c1] -> /usr/lib/dovecot/libdovecot.so.0(+0x4a149) [0x7f0faea5a149] -> /usr/lib/dovecot/libdovecot.so.0(+0x474ac) [0x7f0faea574ac] -> /usr/lib/dovecot/libdovecot.so.0(message_parser_parse_next_block+0x104) [0x7f0faead3844] -> /usr/lib/dovecot/libdovecot.so.0(message_search_msg+0xa8) [0x7f0faead5df8] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xcf89e) [0x7f0faec6e89e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_search_args_foreach+0x45) [0x7f0faebf0445] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xd0774) [0x7f0faec6f774] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xd1a68) [0x7f0faec70a68] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_search_next_nonblock+0x61) [0x7f0faec710e1] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonblock+0x28) [0x7f0faebf9e58] -> dovecot/imap(+0x2691f) [0x55a43cada91f] -> dovecot/imap(command_exec+0x70) [0x55a43cad3d80] -> dovecot/imap(+0x25ed2) [0x55a43cad9ed2] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0x111) [0x7f0faeb018f1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd0) [0x7f0faeb03070] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7f0faeb01b7c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7f0faeb01ce0] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f0faea820d3] -> dovecot/imap(main+0x325) [0x55a43cac4bf5] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7f0fae86809b] -> dovecot/imap(_start+0x2a) [0x55a43cac4d8a] Aug 25 00:33:45 server01 dovecot: imap(username(a)example.com)<24104><hyaFXqutnqUAAAAAAAAAAAAAAAAAAAAB>: Fatal: master: service(imap): child 24104 killed with signal 6 (core dumped) The package versions are: ii dovecot-core 1:2.3.4.1-5+deb10u3 amd64 secure POP3/IMAP server - core files ii dovecot-imapd 1:2.3.4.1-5+deb10u3 amd64 secure POP3/IMAP server - IMAP daemon ii dovecot-lmtpd 1:2.3.4.1-5+deb10u3 amd64 secure POP3/IMAP server - LMTP server ii dovecot-lucene 1:2.3.4.1-5+deb10u3 amd64 secure POP3/IMAP server - Lucene support ii dovecot-mysql 1:2.3.4.1-5+deb10u3 amd64 secure POP3/IMAP server - MySQL support ii dovecot-pop3d 1:2.3.4.1-5+deb10u3 amd64 secure POP3/IMAP server - POP3 daemon ii dovecot-sieve 1:2.3.4.1-5+deb10u3 amd64 secure POP3/IMAP server - Sieve filters support ii dovecot-solr 1:2.3.4.1-5+deb10u3 amd64 secure POP3/IMAP server - Solr support Emails are accessed via roundcube webmail. I am attaching information for a better analysis of the problem. -- Claudio Junior

1 1

Strange index error on NFS after upgrade to CentOS 7 from 6
by Alessio Cecchi 25 Aug '20

25 Aug '20

Hi, I'm running 3 Dovecot director and 6 Dovecot backend on CentOS 6 and Dovecot 2.3.10, with Maildir shared on NFSv3 (on NetApp). Since CentOS 6 will be EOL on November I started to upgrade to CentOS 7 the first Dovecot backend. But after Director start to direct users on new CentOS 7 server some errors, for any users, come up, an example: Aug 21 14:56:41 Error: imap(alessio(a)cecchi.it) session=<11sBlmKtj1qelGW/>: Mailbox INBOX: Broken or unexpectedly changed file /home/vmail/domains/cecchi.it/alessio/Maildir/dovecot-uidlist line 18664: Invalid data: - re-reading from beginning Aug 21 14:56:41 Error: imap(alessio(a)cecchi.it) session=<11sBlmKtj1qelGW/>: Mailbox INBOX: Broken file /home/vmail/domains/cecchi.it/alessio/Maildir/dovecot-uidlist line 18664: Invalid data: Consequently to the error the user see the synchronization of his mailbox again. The Dovecot version and configuration is the same that in CentOS 6, also the mount parameters are the same, so I suspect that could be some difference in the NFS client implementation on CentOS 7. This the entry on fstab: 192.168.1.2:/vmail0 /mnt/vmail0 nfs rw,nfsvers=3,noatime,nodiratime,_netdev,nordirplus 0 0 # Bind for vmail0 /mnt/vmail0/domains /home/vmail/domains none x-systemd.requires=/mnt/vmail0,x-systemd.automount,bind 0 0 In the archive of this list I found a similar issue but without a solutions: https://dovecot.org/pipermail/dovecot/2018-October/113207.html Do you have any suggestions? Thanks -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice

2 2

unsupported auth mechanism
by absolutely_free＠libero.it 25 Aug '20

25 Aug '20

Hi, last friday I upgraded from dovecot-2.3.10.1_2 to dovecot-2.3.11.3 (FreeBSD). Since then, I started having some issues with some customers. The log I see is like: pop3-login: Disconnected (tried to use unsupported auth mechanism): user=<username(a)domain.it>, method=USER, rip=x,y,x,z, lip=x,y,w,z, session=<e5KkR5qto8hPAL9q> While, before upgrade, same user logged succesfully: pop3-login: Login: user=<username(a)domain.it>, method=PLAIN, rip=x,y,x,z, lip=x,y,w,z, mpid=62876, session=<dMRmGWCtFtJtdcy6> This is my conf (unchanged since months): # dovecot -n # 2.3.11.3 (502c39af9): /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 12.1-RELEASE-p8 amd64 zfs # Hostname: server doveconf: Warning: please set ssl_dh=</usr/local/etc/dovecot/dh.pem doveconf: Warning: You can generate it with: dd if=/var/db/dovecot/ssl-parameters.dat bs=1 skip=88 | openssl dhparam -inform der > /usr/local/etc/dovecot/dh.pem auth_mechanisms = plain login digest-md5 cram-md5 auth_verbose = yes auth_verbose_passwords = yes default_client_limit = 2000 default_process_limit = 500 default_vsz_limit = 512 M disable_plaintext_auth = no first_valid_gid = 125 first_valid_uid = 125 imap_id_log = * mail_gid = 1003 mail_location = maildir:/mail/domains mail_privileged_group = postfix mail_uid = 1003 namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } passdb { args = /etc/dovecot/dovecot-sql-crypt.conf.ext driver = sql } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = postfix mode = 0600 user = postfix } } service imap { process_limit = 1024 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl_cert = </etc/ssl/wildcard.domain.net_bundle.crt ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol imap { mail_max_userip_connections = 100 } any help? Thank you!

2 2

Panic with namespace
by Kostya Vasilyev 25 Aug '20

25 Aug '20

Hi, I was trying to configure a namespace like this: 10-mail.conf namespace inbox { prefix = "iNBox." inbox = yes } And when a client connected, I got the panic below. Dovecot version is 2.3.4.1 (f79e8e7e4) on Debian 10. Aug 25 16:18:27 kman dovecot: imap(kman@...)<20082><XOXiibOtpq2y2nhZ>: Panic: file mailbox-list-maildir-iter.c: line 511 (maildir_list_iter_next ): assertion failed: ((ctx->info.flags & MAILBOX_NOCHILDREN) != 0) Aug 25 16:18:27 kman dovecot: imap(kman@...)<20082><XOXiibOtpq2y2nhZ>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xdb62b) [0x7f659 9d9962b] -> /usr/lib/dovecot/libdovecot.so.0(+0xdb6c1) [0x7f6599d996c1] -> /usr/lib/dovecot/libdovecot.so.0(+0x4a149) [0x7f6599d08149] -> /usr/lib/dov ecot/libdovecot-storage.so.0(+0x45b6e) [0x7f6599e92b6e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_list_index_sync+0x1dc) [0x7f6599f08c6c] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_list_index_refresh_force+0xfb) [0x7f6599f0ac3b] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb9274 ) [0x7f6599f06274] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb9467) [0x7f6599f06467] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_exists+0 x68) [0x7f6599ea70e8] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_list_mailbox+0x1f6) [0x7f6599eb6126] -> /usr/lib/dovecot/libdovecot-storage. so.0(mailbox_list_iter_init_namespaces+0x208) [0x7f6599f0be08] -> dovecot/imap(cmd_list_full+0x60e) [0x5654c07d02de] -> dovecot/imap(command_exec+0x70 ) [0x5654c07d9d80] -> dovecot/imap(+0x1e3f2) [0x5654c07d83f2] -> dovecot/imap(+0x1e494) [0x5654c07d8494] -> dovecot/imap(client_handle_input+0x1b5) [0 x5654c07d8845] -> dovecot/imap(client_input+0x7e) [0x5654c07d8d6e] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x6f) [0x7f6599dafadf] -> /usr/ lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x136) [0x7f6599db10d6] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7f65 99dafb7c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7f6599dafce0] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f659 9d300d3] -> dovecot/imap(main+0x325) [0x5654c07cabf5] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7f6599b1b09b] -> dovecot/imap(_sta rt+0x2a) [0x5654c07cad8a] -- K

1 0

doveadm/doveconf as user tries and fails to read host key
by jimc 25 Aug '20

25 Aug '20

Distro: OpenSuSE Tumbleweed for x86_64 Failing version: dovecot23-2.3.11.3-1.1.x86_64 Install Date: 2020-08-18 Reverting to previous version works: dovecot23-2.3.10.1-2.3.x86_64 (Packages downgraded coordinately: dovecot23 dovecot23-backend-sqlite) How to make it fail: As the user, execute doveadm expunge mailbox Spam37 savedbefore 3day #User's actual cmd doveadm who #The simplest possible command, for testing It says: doveconf: Fatal: Error in configuration file /etc/dovecot/conf.d/10-ssl.conf line 12: ssl_cert: Can't open file /etc/ssl/hostcerts/hostw.cia: Permission denied The actual EPERM occurs trying to traverse a directory in /etc/letsencrypt, but the next configuration item to be read (in the SSL section) is the host's private key, and the user is surely not ever going to get permission to read that. (I did test giving the user permission to the 750 directory and it did attempt to read the private key, failing.) If you run it as root, of course it works because root has read permission. The initial failure was seen running as the user from cron. Behavior seen in strace: doveadm execs doveconf; doveconf reads the configuration and saves it somewhere (shared memory?); doveconf execs the next program which in this case is doveadm with its original command line; and doveadm now knows its configuration. I can re-do and post the strace if needed. I don't know why doveconf is reading the SSL keys in 2.3.11.3 when it didn't in 2.3.10.1, but if the idea is to read the complete configuration in case it might be needed in obscure situations, a possible workaround is to not die on unreadable secrets and to report those either as unset or as a new "error" symbol, letting the consumer (doveadm) deal with the fallout, or in this case ignore it. Attached: sysreport.gz ; doveconf-n.out Dovecot's working files and user mailboxes are on ext4 filesystems; NFS is not involved. The mail reader is Roundcube webmail. -- James F. Carter Email: jimc(a)jfcarter.net Web: http://www.math.ucla.edu/~jimc (q.v. for PGP key)

2 2