- dovecot - dovecot.org

doveadm sync duplicating emails
by Paul Robinson 27 Apr '21

27 Apr '21

I'm upgrading a mail server using postfix 2.9.6 and dovecot 2.0.19 to a new server running postfix 3.4.13 and dovecot 2.3.7.2 (using Maildir format on both sides). I had trouble getting doveadm backup to work for one user, while all the others worked straight away. It would complain about problems on the old server in the Maildir/dovecot* files. I managed to get it to work by removing all the dovecot* files under Maildir for that one user (accepting that this would mean clients having to redownload emails). >From that point, a doveadm backup worked. Using roundcube to look at the new mail server, I could see everything arrived. At this point, some mail generated on the new mail server would go to the new mail server, but all other mail is going to the old mail server. I then tried to use doveadm sync -1R to update the new mail server so the new server contains all the mail from the old server plus the new mail that has been locally delivered. What then happened was that all the new emails delivered to the old server since the backup were copied to the new server, and all the emails that were locally delivered on the new mail server were still there. (So far so good). But I also saw that every email that was on the old server before the backup now appeared twice on the new mail server. Running sync again would mean I had three copies of email. The sync ran quickly - much too quickly for these extra copies to have been sent over the network. It appears that the sync made extra local copies on the new server. I see no errors in logs that can explain things. Looking at the email, the files stored in the cur directories are there multiple times; they have different names, but identical content. Trying "doveadm deduplicate -F user-list ALL" ran quickly and to no effect (where user-list is a file with a list of users). Trying "doveadm deduplicate -F user-list -m ALL" takes 100% cpu and a long time to run and still running right now after 4 hours. Judging by the disk space usage freed up, this is going to take a very long time to complete. Has anybody got any idea what might cause this and what change I should make? This was the sync command line: doveadm -vc /etc/dovecot/dovecot-migration.conf sync -1RF user-list imapc: For the backup and sync, I used this as the config: imapc_features = rfc822.size fetch-headers imapc_host = <snipped> imapc_ssl = imaps imapc_port = 993 imapc_user = %n imapc_master_user = master imapc_password = <snipped> mail_prefetch_count = 20 ssl_cipher_list = EECDH+AESGCM+AES128:EECDH+AESGCM+AES256:EECDH+CHACHA20:EDH+AESGCM+AES128:EDH+AESGCM+AES256:EDH+CHACHA20:EECDH+SHA256+AES128:EECDH+SHA384+AES256:EDH+SHA256+AES128:EDH+SHA256+AES256:EECDH+SHA1+AES128:EECDH+SHA1+AES256:EDH +SHA1+AES128:EDH+SHA1+AES256:EECDH+HIGH:EDH+HIGH:AESGCM+AES128:AESGCM+AES256:CHACHA20:SHA256+AES128:SHA256+AES256:SHA1+AES128:SHA1+AES256:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK:!KRB5:!aECDH:!DH mail_location = maildir:~/Maildir mail_home = /var/vmail/%d/%n/ mail_fsync=never Thanks, Paul

2 1

failed: Cached message size smaller than expected
by Michael Grant 27 Apr '21

27 Apr '21

Every few days, my mailbox seizes up. No mail come in to my imap clients. I'm getting these errors over and over with my mailbox: Error: Mailbox INBOX: Deleting corrupted cache record uid=371208: UID 371208: Broken physical size in mailbox INBOX: read(/var/mail/mgrant) failed: Cached message size smaller than expected (17212 < 17222, box=INBOX, UID=371208) Error: Mailbox INBOX: UID=371208: read(/var/mail/mgrant) failed: Cached message size smaller than expected (17212 < 17222, box=INBOX, UID=371208) (FETCH BODY[]) Error: Mailbox INBOX: Deleting corrupted cache record uid=371203: UID 371203: Broken physical size in mailbox INBOX: read(/var/mail/mgrant) failed: Cached message size smaller than expected (3904 < 3914, box=INBOX, UID=371203) Error: Mailbox INBOX: UID=371203: read(/var/mail/mgrant) failed: Cached message size smaller than expected (3904 < 3914, box=INBOX, UID=371203) (FETCH BODY[]) My inbox is an mbox file. I'm running dovecot installed on Debian Bullseye, the dovecot packages are all: 1:2.3.13+dfsg1-1 I am running sendmail and using procmail for local delivery. I suspect, but am not certain, that this may be some locking issue between procmail and dovecot but I have never been able to prove that. The final procmail rule which appends messages to my mailbox looks like this, the trailing ':' causes procmail to use a lockfile: :0: /var/mail/mgrant The locking config lines in 10-mail.conf are commented, but I have also tried uncommenting them, did not help: #mbox_read_locks = fcntl #mbox_write_locks = fcntl dotlock Though sometimes it seems to fix itself after a few hours, the only way I have found to fix this quickly is to manually remove the cache files and restart dovecot: rm ~/mail/.imap/INBOX/* systemctl restart dovecot I am not even sure this is a locking issue. Something definitely gets corrupted though. I do have several IMAP clients hitting the same mailbox (phone, laptop, desktop). On the phone, I run K9 and also the gmail client which talks imap. Also using thunderbird, outlook, and w10 mail, though typically not all at the same time. You could definitely say I am stress testing this setup a bit! Any ideas on how to resolve this? Michael Grant

6 10

Dovecot integration w/ FreeIPA expired password as well as if over quota login notice; local user can't login
by Robert Kudyba 26 Apr '21

26 Apr '21

As I continue to test freeipa-server-4.9.3-1, on Fedora 33 with dovecot-2.3.14-1 I've run into the following issues with web mail and Dovecot integration. 1. I followed https://www.freeipa.org/page/Dovecot_IMAPS_Integration_with_FreeIPA_using_S… but I couldn't get web mail to login until I used the suggestion from https://blog.delouw.ch/2017/02/19/integrate-dovecot-imap-with-freeipa-using… and changed logins auth_mechanisms = plain gssapi login which allowed logins of FreeIPA Kerberos users. 2. even with auth_mechanisms = plain gssapi login, I could then no longer login to SquirrelMail webmail with any local Unix (non-Kerberized) users. The dovecot logs show: auth: Error: policy(localuser(a)ourdomain.edu,127.0.0.1,<r2eFe+PAvut/AAAB>): Policy server HTTP error: connect(x.x.x.x:8084) failed: Connection refused auth: Debug: policy(localuser(a)ourdomain.edu,127.0.0.1,<r2eFe+PAvut/AAAB>): Policy report action finished auth: Debug: http-client[1]: request [Req2: POST https://x.x.x.x:8084/?command=report] Destroy (requests left=1) auth: Debug: http-client[1]: request [Req2: POST https://x.x.x.x:8084/?command=report] Free (requests left=0) auth: Debug: http-client: conn x.x.x.x[2]: Connection close auth: Debug: http-client: conn x.x.x.x[2]: Connection disconnect auth: Debug: http-client: conn x.x.x.x[2]: Disconnected: connect() failed: Connection refused (fd=23) auth: Debug: http-client: conn x.x.x.x[2]: Detached peer auth: Debug: http-client: conn x.x.x.x[2]: Connection destroy auth: Debug: http-client: host x.x.x.x: Idle host timed out auth: Debug: http-client: host x.x.x.x: Host destroy auth: Debug: http-client: host x.x.x.x: Host session destroy auth: Debug: http-client[1]: queue https://x.x.x.x:8084: Destroy auth: Debug: client passdb out: FAIL 1 user=localuser(a)ourdomain.edu original_user=localuser imap-login: Debug: Ignoring unknown passdb extra field: original_user imap-login: Info: Aborted login (auth failed, 1 attempts in 3 secs): user=< localuser(a)ourdomain.edu>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured, session=<r2eFe+PAvut/AAAB> 3. If a user was over quota there was no way to tell on the webmail page that they were over quota but the dovecot logs show imap(ouruser): Error: mkdir(/path/to/ouruser/mail/.imap) failed: Disk quota exceeded. Would there be a security risk if the web page displayed a warning that could be generalized to inform the user to either check their quota or password reset being needed?

1 0

Live migrating index/control files
by Eirik Rye 26 Apr '21

26 Apr '21

Hello, We are running a director-based dovecot cluster where the user's maildir and the indexes/control files are, for legacy reasons, stored on separate NFS-backends, i.e.: mail_location = maildir:~/Maildir:CONTROL=/mail/control/%1u/%2u/%u:INDEX=/mail/index/%1u/%2u/%u Both ~/Maildir and /mail/{control,index}/ are separate NFS-mounts, and we now want to consolidate all files to the user's home directory instead. The end goal is to stop using NFS altogether. I have devised a four step plan for gradually migrating the index/control files into the user's maildir, on a per-user basis: 1. Kick the user in an attempt to flush the user's index files: `doveadm director kick <username>` 2. Rsync the index/control files into the user's maildir: `rsync -uav /mail/{index,control}/${user:0:1}/${user:0:2}/${user}/ /mail/${user:0:1}/${user:0:2}/${user}/Maildir/;` 3. Update the (redis-based) userdb for the specific user to override mail_location to use the default CONTROL/INDEX locations: `{"mail": "maildir:~/Maildir"}` 4. Kick the user one last time so that they reconnect using the new settings. Unfortunately, after running this for a small batch of users, we recieved a couple reports that their clients had started to resynchronize and redownload their mailboxes, which is exactly what we're trying to avoid. There are no index related error messages related to the few users that reported these issues. From my understanding, clients become confused when the control files are changed or lost (`dovecot-uidlist` and `dovecot-uidvalidity`), so I suspect the kick + rsync + kick procedure is not robust enough for this. Does anyone have any experience with performing such a migration without taking systems offline for the duration? Any suggestions to make this process more transparent for our users, so as to not induce panic when their mailboxes abruptly appear empty? Best regards, Eirik

2 5

How to Easily Set Up a Full-Featured Linux Mail Server on Ubuntu 18.04.5 LTS with iRedMail 1.4.0
by Turritopsis Dohrnii Teo En Ming 26 Apr '21

26 Apr '21

Subject: How to Easily Set Up a Full-Featured Linux Mail Server on Ubuntu 18.04.5 LTS with iRedMail 1.4.0 Good day from Singapore, I followed linuxbabe.com's Xiao Guoan's guide and successfully setup a full featured Linux mail server on Ubuntu 18.04.5 LTS with IRedMail 1.4.0. Author: Mr. Turritopsis Dohrnii Teo En Ming (TARGETED INDIVIDUAL) Country: Singapore Date: 25 April 2021 Sunday Type of Publication: PDF Manual Document Version: 20210425.01 (1st release) ***IMPORTANT NOTICE*** Please note that Turritopsis Dohrnii Teo En Ming’s guide is based on Xiao Guoan’s guide at linuxbabe.com. Reference Guide Used by Teo En Ming: How to Easily Set Up a Full-Featured Mail Server on Ubuntu 18.04 with iRedMail Link: https://www.linuxbabe.com/mail-server/ubuntu-18-04-iredmail-email-server Original Author: Xiao Guoan The following is a list of open-source software that will be automatically installed and configured by iRedMail. • Postfix SMTP server • Dovecot IMAP server • Nginx web server to serve the admin panel and webmail • OpenLDAP, MySQL/MariaDB, or PostgreSQL for storing user information • Amavised-new for DKIM signing and verification • SpamAssassin for anti-spam • ClamAV for anti-virus • Roundcube webmail • SOGo groupware, providing webmail, calendar (CalDAV), contacts (CardDAV), tasks and ActiveSync services. • Fail2ban for protecting SSH • mlmmj mailing list manager • Netdata server monitoring • iRedAPD Postfix policy server for greylisting Redundant Download Links for Teo En Ming's PDF Manual: [1] https://drive.google.com/file/d/1un8sLLmNSMIt7V6blWCvJEgwGvxMbd4B/view?usp=… [2] https://drive.google.com/file/d/1i0vY7kfYkobu563qoI3_qCZg7G7BFoYR/view?usp=… [3] https://drive.google.com/file/d/1U9MFN1EklLbA8TMweLV5ntiSJuBBVkpQ/view?usp=… [4] https://www.docdroid.net/dW70KtS/iredmail-setup-1st-release-pdf [5] https://www.mediafire.com/file/evar7j28knqyoj6/IRedMail+Setup+1st+Release.p… [6] https://www.scribd.com/document/504932780/IRedMail-Setup-1st-Release Mr. Turritopsis Dohrnii Teo En Ming, 43 years old as of 26 April 2021, is a TARGETED INDIVIDUAL living in Singapore. He is an IT Consultant with a System Integrator (SI)/computer firm in Singapore. He is an IT enthusiast. -----BEGIN EMAIL SIGNATURE----- The Gospel for all Targeted Individuals (TIs): [The New York Times] Microwave Weapons Are Prime Suspect in Ills of U.S. Embassy Workers Link: https://www.nytimes.com/2018/09/01/science/sonic-attack-cuba-microwave.html ******************************************************************************************** Singaporean Targeted Individual Mr. Turritopsis Dohrnii Teo En Ming's Academic Qualifications as at 14 Feb 2019 and refugee seeking attempts at the United Nations Refugee Agency Bangkok (21 Mar 2017), in Taiwan (5 Aug 2019) and Australia (25 Dec 2019 to 9 Jan 2020): [1] https://tdtemcerts.wordpress.com/ [2] https://tdtemcerts.blogspot.sg/ [3] https://www.scribd.com/user/270125049/Teo-En-Ming -----END EMAIL SIGNATURE-----

1 0

cannot see my mails
by Jean-Max Reymond 26 Apr '21

26 Apr '21

Hi, After a change of server, I cannot see my mails. postfix is OK and I receive emails. If I send emails, with roundcube or thunderbird, I can see these new sent emails. Access rights seems OK. Dovecot with debug trace does not complain. Any tips ? append_dot_mydomain = no biff = no broken_sasl_auth_clients = yes compatibility_level = 2 content_filter = smtp-amavis:[127.0.0.1]:10024 home_mailbox = Maildir/ local_recipient_maps = $virtual_mailbox_maps local_transport = virtual mailbox_command = /usr/lib/dovecot/deliver -c /etc/dovecot/dovecot.conf -m "${EXTENSION}" message_size_limit = 30720000 milter_default_action = accept milter_protocol = 2 mydestination = localhost non_smtpd_milters = inet:localhost:12345 readme_directory = no slow_destination_concurrency_limit = 2 slow_destination_recipient_limit = 20 smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt smtp_use_tls = yes smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) smtpd_milters = inet:localhost:12345 smtpd_recipient_restrictions = reject_unknown_sender_domain, reject_unknown_recipient_domain, reject_unauth_pipelining, permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination smtpd_sasl_auth_enable = yes smtpd_sasl_authenticated_header = yes smtpd_sasl_local_domain = $myhostname smtpd_sasl_path = private/dovecot-auth smtpd_sasl_security_options = noanonymous smtpd_sasl_type = dovecot smtpd_sender_restrictions = reject_unknown_sender_domain smtpd_tls_auth_only = yes smtpd_tls_cert_file = /etc/dovecot/private/dovecot.pem smtpd_tls_key_file = /etc/dovecot/private/dovecot.key smtpd_tls_mandatory_ciphers = medium smtpd_tls_mandatory_protocols = SSLv3, TLSv1 smtpd_tls_received_header = yes smtpd_use_tls = yes tls_random_source = dev:/dev/urandom transport_maps = hash:/etc/postfix/transport vacation_destination_recipient_limit = 1 virtual_alias_maps = proxy:mysql:/etc/postfix/mysql/virtual_alias_maps.cf virtual_gid_maps = static:8 virtual_mailbox_base = /home/Mails virtual_mailbox_domains = proxy:mysql:/etc/postfix/mysql/virtual_domains_maps.cf virtual_mailbox_limit = 204800000 virtual_mailbox_maps = proxy:mysql:/etc/postfix/mysql/virtual_mailbox_maps.cf virtual_minimum_uid = 8 virtual_transport = virtual virtual_uid_maps = static:8 yahoo_destination_concurrency_limit = 4 yahoo_destination_rate_delay = 1s yahoo_destination_recipient_limit = 2 yahoo_initial_destination_concurrency = 1 -- Jean-Max Reymond CKR Solutions Open Source https://www.ckr-solutions.com

3 9

Panic: file ostream.c: assertion failed when using mail_filter on RHEL8
by Martijn Brinkers (list) 24 Apr '21

24 Apr '21

Hi, I installed dovecot on a new up-to-date RHEL8 test system and configured a simple mail_filter plugin which only copies the input back to the output (i.e., the filter does nothing special). On RHEL8 I get the following panic when saving the mail. The Dovecot version that comes with RHEL8 is dovecot-2.3.8-4.el8.x86_64 The mail_filter works correctly on an Ubuntu machine )2.2.33.2- 1ubuntu4.7) Any idea what's causing this panic on RHEL8 and how to fix it? Kind regards, Martijn Brinkers Log output (panic log is at the bottom) Apr 23 10:35:54 ciphermail-webmail dovecot[18680]: master: Dovecot v2.3.8 (9df20d2db) starting up for imap Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: Loading modules from directory: /usr/lib64/dovecot/auth Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/lib20_auth_var_expand_crypt.so Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.so Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: Read auth token secret from /var/run/dovecot/auth-token-secret.dat Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: auth client connected (pid=19771) Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: client in: AUTH 1 PLAIN service=imap secured se ssion=i7avXaHAuLh/AAAB lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=47288 resp=<hidden> Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: dict( test(a)example.com,127.0.0.1,<i7avXaHAuLh/AAAB>): Performing passdb lookup Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth-worker(19774): Debug: Loading modules from directory: /usr/lib64/dovecot/auth Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth-worker(19774): Debug: Module loaded: /usr/lib64/dovecot/auth/lib20_auth_var_expand_crypt.so Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth-worker(19774): Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.so Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth-worker(19774): Debug: conn unix:auth-worker (pid=19773,uid=97): Server accepted connection (fd=13) Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth-worker(19774): Debug: conn unix:auth-worker (pid=19773,uid=97): Sending version handshake Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth-worker(19774): Debug: conn unix:auth-worker (pid=19773,uid=97): auth-worker<1>: Handling PASSV request Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth-worker(19774): Debug: dict(test(a)example.com,127.0.0.1,<i7avXaHAuLh/AAAB>): Performing passdb lookup Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth-worker(19774): Debug: dict(test(a)example.com,127.0.0.1,<i7avXaHAuLh/AAAB>): Lookup: shared/passdb/test(a)example.com/test = {"userdb_email":"test(a)example.com ","userdb_quota_rule":"*:bytes=1073741824","password":"{SSHA256.b64}SF5 hEeUZVO4ydasqNb040KHWGyin791L40BYR8cXf+I0EGeetzq5Cg==","userdb_password ":"test","userdb_home":" /var/vmail/test=40example.com(a)ciphermail.private","user":" test=40example.com(a)ciphermail.private"} Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth-worker(19774): Debug: dict(test(a)example.com,127.0.0.1,<i7avXaHAuLh/AAAB>): username changed test(a)example.com -> test=40example.com(a)ciphermail.private Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth-worker(19774): Debug: dict(test=40example.com(a)ciphermail.private,127.0.0.1,<i7avXaHAuL h/AAAB>): Finished passdb lookup Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth-worker(19774): Debug: conn unix:auth-worker (pid=19773,uid=97): auth-worker<1>: Finished Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: dict( test(a)example.com,127.0.0.1,<i7avXaHAuLh/AAAB>): username changed test(a)example.com -> test=40example.com(a)ciphermail.private Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: dict( test=40example.com(a)ciphermail.private,127.0.0.1,<i7avXaHAuLh/AAAB>): Finished passdb lookup Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: auth( test=40example.com(a)ciphermail.private,127.0.0.1,<i7avXaHAuLh/AAAB>): Auth request finished Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: client passdb out: OK 1 user=test=40example.com(a)ciphermail.private original_user=test(a)example.com Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: master in: REQUEST 3373400065 19771 1 603cbf25103fc9a6 77ceabe45f077e40 session_pid=19776 request_auth_token Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: prefetch(test=40example.com(a)ciphermail.private,127.0.0.1,<i7avXaHAuLh/A AAB>): Performing userdb lookup Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: prefetch(test=40example.com(a)ciphermail.private,127.0.0.1,<i7avXaHAuLh/A AAB>): success Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: prefetch(test=40example.com(a)ciphermail.private,127.0.0.1,<i7avXaHAuLh/A AAB>): Finished userdb lookup Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: master userdb out: USER 3373400065 test=40example.com(a)ciphermail.private email=test(a)example.com quota_rule=*:bytes=1073741824 password=<hidden> home=/var/vmail/test=40example.com(a)ciphermail.private auth_mech= PLAIN auth_token=819d2ad4117e4e4139991fab046e13304880e935 auth_user=test(a)example.com Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap-login: Login: user=<test=40example.com(a)ciphermail.private>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=19776, secured, session=<i7avXaHAuLh/AAAB> Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Loading modules from directory: /usr/lib64/dovecot Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Module loaded: /usr/lib64/dovecot/lib10_mail_filter_plugin.so Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Module loaded: /usr/lib64/dovecot/lib10_quota_plugin.so Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Module loaded: /usr/lib64/dovecot/lib11_imap_quota_plugin.so Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Added userdb setting: plugin/auth_mech=PLAIN Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Added userdb setting: plugin/email=test(a)example.com Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Added userdb setting: plugin/password=<hidden> Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Added userdb setting: plugin/quota_rule=*:bytes=1073741824 Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Effective uid=986, gid=983, home=/var/vmail/test=40example.com(a)ciphermail.private Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: mail_filter: Filtering mail_filter via socket /var/run/dovecot/mail- filter Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: mail_filter: Filtering mail_filter_out via socket /var/run/dovecot/mail-filter-out Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Quota root: name=User quota backend=maildir args= Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Quota rule: root=User quota mailbox=* bytes=1073741824 messages=0 Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Quota rule: root=User quota mailbox=Trash bytes=+107374182 (10%) messages=0 Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Quota grace: root=User quota bytes=107374182 (10%) Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: maildir++: root=/var/vmail/test=40example.com(a)ciphermail.private, index=, indexpvt=, control=, inbox=/var/vmail/test=40example.com(a)ciphermail.private, alt= Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: quota: quota_over_flag check: quota_over_script unset - skipping Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Mailbox Drafts: Mailbox opened because: APPEND Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Panic: file ostream.c: line 201 (o_stream_flush): assertion failed: (stream- >stream_errno != 0) Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0xf895b) [0x7f8f1d3d995b] -> /usr/lib64/dovecot/libdovecot.so.0(+0xf89f7) [0x7f8f1d3d99f7] -> /usr/lib64/dovecot/libdovecot.so.0(+0x55b05) [0x7f8f1d336b05] -> /usr/lib64/dovecot/libdovecot.so.0(+0x5a1bd) [0x7f8f1d33b1bd] -> /usr/lib64/dovecot/libdovecot- storage.so.0(maildir_save_finish+0x99) [0x7f8f1d723fa9] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0xfcee) [0x7f8f1bd5acee] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_save_finish+0x7d) [0x7f8f1d6fafdd] -> dovecot/imap(+0x132fd) [0x564a032332fd] -> dovecot/imap(command_exec+0x78) [0x564a03241cd8] -> dovecot/imap(+0x128ad) [0x564a032328ad] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x73) [0x7f8f1d3f10c3] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x135) [0x7f8f1d3f2775] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x50) [0x7f8f1d3f1170] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x48) [0x7f8f1d3f12d8] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x17) [0x7f8f1d368eb7] -> dovecot/imap(main+0x335) [0x564a03232445] -> /lib64/libc.so.6(__libc_start_main+0xf3) [0x7f8f1ca5b803] -> dovecot/imap(_start+0x2e) [0x564a032325fe] Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap(test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Fatal: master: service(imap): child 19776 killed with signal 6 (core not dumped - https://dovecot.org/bugreport.html#coredumps - set /proc/sys/fs/suid_dumpable to 2)

1 1

Why was mail-filter plugin removed in version 2.3.14?
by Martijn Brinkers (list) 23 Apr '21

23 Apr '21

Hi, Mail-filter plugin was removed in version 2.3.14. Any idea why was this removed? Is there a replacement for the mail-filter plugin? Kind regards, Martijn Brinkers

1 0

Size of transaction log file is increasing continuously if dsync is failed
by Duc Anh Do 22 Apr '21

22 Apr '21

Hi Dovecot community, I would like to share a problem I saw a few days ago. Any comment is highly appreciated. *Steps to reproduce* 0. Version of Dovecot I use is v2.3.11.3. 1. Prepare 2 Dovecot backends: - One backend is used as the source of dsync command. Other backend is used as the destination of the dsync command. - Mailbox format is Maildir. Mail dir of two test accounts is empty. 2. Prepare the test account in the source of dsync: - Send a test message to the test account. For example, I use swaks: swaks --server 127.0.0.1 --port 24 --protocol LMTP --to user1@local - Use doveadm expunge command to clean mail dir of the test account: doveadm expunge -u user1@local mailbox INBOX all 3. Prepare the test account in the destination of dsync: - Log in test account via IMAP then select INBOX first: telnet 127.0.0.1 imap ... s login user1@local 1 ... s select INBOX ... s logout - Then perform dsync: doveadm -o imapc_host=backend2.local -o imapc_port=143 -o imapc_user=user1@local -o imapc_password=1 backup -s '' -R -u user1@local imapc: - Confirm error occurs when using dsync: dsync(user1@local): Info: imapc(backend2.local:143): Connected to 192.168.100.2:143 (local 192.168.100.27:58850) dsync(user1@local): Error: Mailbox INBOX sync: mailbox_delete failed: INBOX can't be deleted. AQAAAHm4+Jk= 4. Log in the test account in dsync destination then select INBOX. Repeat many times. I can see the size of the transaction log file is increasing continuously. *Before*: -rw-------. 1 vmail vmail 456 Apr 22 10:54 dovecot.index.log *After*: -rw-------. 1 vmail vmail 544 Apr 22 10:59 dovecot.index.log -rw-------. 1 vmail vmail 632 Apr 22 11:00 dovecot.index.log -rw-------. 1 vmail vmail 720 Apr 22 11:00 dovecot.index.log *Problems* - Size of the transaction log file is increasing continuously. Do you think this is an issue of Dovecot? - The value of *mail_index_log_rotate_max_size* is 1M by default. However the transaction log file dovecot.index.log is not rotated when its size exceeds limit. - Because the size of the transaction log file is too big and it is not rotated, I think "out of memory" error will occur when I try to use IMAP SELECT or POP3 STAT command (if the virtual size limit is small enough). - If "size of the transaction log file is increasing continuously" is an issue, what should I do to avoid it or is there any workaround? I understand I should fix the error *Error: Mailbox INBOX sync: mailbox_delete failed: INBOX can't be deleted* first before doing anything next. I think removing the transaction log file is an understandable workaround. *Investigaton* - About the error *Error: Mailbox INBOX sync: mailbox_delete failed: INBOX can't be deleted*, I can understand the cause after reading this thread https://dovecot.org/pipermail/dovecot/2014-April/095588.html 2) Maildir + INBOX + backup/sync/replicate > In the test scenarios where the INBOX on one side was to be completely > removed, e.g. doveadm backup -R the dsync failed and > nothing was synced to the target. This is because before moving the source > mails to the mailbox, dsync cleans out the old ones ( > -R preserves nothing) and in Maildir the INBOX can not be removed. This is > a feature/not easily solvable, because in Maildir > INBOX is different from other folders. > - The transaction log file is rotated whenever there's any change in index file. I can confirm by sending a test message to the test account in dsync destination. I can see that logic in *src/lib-index/mail-index-sync.c* and *src/lib-index/mail-index-write.c*. After that, the size of the new transaction log file is not increasing anymore. - When the error *Error: Mailbox INBOX sync: mailbox_delete failed: INBOX can't be deleted* occurred, I tried to dump the transaction log file and I saw these records: record: offset=432, type=index-deleted (ext), size=12 record: offset=444, type=index-undeleted (ext), size=12 - Do you think these records relate to the issue that the size of the transaction log file is increasing continuously? - Did the error *Error: Mailbox INBOX sync: mailbox_delete failed: INBOX can't be deleted* make content of transaction log file incomplete/incorrect? Regards, Anh Do

1 0

CA certs for Dovecot-as-client (proxy)
by Peter Mogensen 21 Apr '21

21 Apr '21

Hi, When using proxy=y, ssl=yes (Dovecot 2.3.13) I consistently get this logged when trying to validate the remote server cert. "Disconnected by server: Connection closed: Received invalid SSL certificate: unable to get local issuer certificate: /C=BE/O=GlobalSign nv-sa/CN=AlphaSSL CA - SHA256 - G2 (check ssl_client_ca_* settings?)" As I read the 2.3.x documentation (and the error logged) Dovecot needs to have the trusted CA cert with ssl_client_ca_file or ssl_client_ca_dir. So, I've tried every combination of putting the cert (and the GlobalSign root CA signing it) in ssl_client_ca_dir and individually and as a bundle in ssl_client_ca_file without luck. But even though I can verify the cert with "openssl s_client -connect" and with "openssl verify", no matter what I put in the ssl_client_ca_* settings it seems Dovecot just ignores it. It does complain though, if I point it to a non-existent file, but not if I just fill the file with invalid cert data which can't be parsed. I end up getting in doubt whether it consults the cert data at all. I'm a bit at loss on how to debug this further, short of running it in gdb. "verbose_ssl" doesn't really say anything about the process of find a CA cert to check with. Have I misunderstood the config? /Peter

3 2

systemd timeout on startup after upgrade
by Felix Zielcke 21 Apr '21

21 Apr '21

Hello *, I have upgraded today a Debian buster (stable) system to bullseye (still testing). After upgrade I did a complete reboot of the VM. Dovecot version changed from 1:2.3.4.1-5+deb10u6 to 1:2.3.13+dfsg1-1 `systemctl start dovecot` now complains Job for dovecot.service failed because a timeout was exceeded. As a workaround I set "TimeoutStartSec=infinity" in dovecot.service. But `systemctl status` now says: Active: activating (start) since Sat 2021-04-17 17:53:58 CEST; 1h 23min ago Dovecot itself is working fine without problems. LMTP accepts new mails and IMAP works. It's just that systemd doestn't notice it running. Any idea what causes this? Config is unchanged from the working 2.3.4.1 system. # doveconf -n # 2.3.13 (89f716dc2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.13 (cdd19fe3) # OS: Linux 5.10.0-6-cloud-amd64 x86_64 Debian bullseye/sid # Hostname: daedalus.z-51.de auth_master_user_separator = * auth_mechanisms = plain login auth_ssl_username_from_cert = yes auth_verbose = yes auth_verbose_passwords = plain default_vsz_limit = 2 G first_valid_uid = 8 import_environment = TZ last_valid_uid = 8 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c ssl_security=%k login_trusted_networks = 127.0.0.1 ::1 mail_gid = mail mail_location = maildir:~/Maildir mail_plugins = acl zlib quota fts fts_xapian mail_privileged_group = mail mail_uid = mail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded- character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { special_use = \Junk } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = scheme=CRYPT username_format=%Lu /etc/dovecot/users driver = passwd-file } plugin { acl = vfile:/etc/dovecot/global-acls:cache_secs=300 fts = xapian fts_autoindex = yes fts_autoindex_exclude = \Junk fts_autoindex_exclude2 = \Trash fts_xapian = partial=3 full=20 attachments=0 verbose=0 imapsieve_mailbox1_before = file:/var/mail/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Spam imapsieve_mailbox2_before = file:/var/mail/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Spam imapsieve_mailbox2_name = * sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /var/mail/sieve/before.sieve sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute sieve_pipe_bin_dir = /var/mail/sieve sieve_plugins = sieve_imapsieve sieve_extprograms sieve_redirect_envelope_from = recipient sieve_vacation_default_period = 1d stats_refresh = 30 secs stats_track_cmds = yes } postmaster_address = postmaster(a)z-51.de protocols = " imap lmtp sieve" service auth-worker { user = $default_internal_user } service auth { unix_listener auth-client { group = Debian-exim mode = 0777 user = Debian-exim } unix_listener auth-userdb { group = Debian-exim mode = 0777 user = Debian-exim } vsz_limit = 2 G } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service indexer-worker { vsz_limit = 0 } service lmtp { user = mail } service managesieve-login { inet_listener sieve { address = 127.0.0.1 ::1 port = 4190 } service_count = 1 } service managesieve { process_limit = 1024 } service stats { fifo_listener stats-mail { mode = 0600 user = mail } unix_listener stats-writer { user = mail } } ssl = required ssl_cert = </etc/letsencrypt/live/daedalus.z-51.de/fullchain.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes userdb { args = username_format=%Lu /etc/dovecot/users default_fields = uid=mail gid=mail home=/var/mail/%d/%n driver = passwd-file } verbose_proctitle = yes protocol lmtp { auth_username_format = %Lu mail_plugins = acl zlib quota fts fts_xapian sieve } protocol lda { mail_plugins = acl zlib quota fts fts_xapian sieve } protocol imap { mail_plugins = acl zlib quota fts fts_xapian imap_zlib imap_acl imap_quota imap_sieve }

5 9

Planning on setting up Postfix + Dovecot + Webmail + Apache + MariaDB + PHP using Webmin Control Panel in the Cloud
by Turritopsis Dohrnii Teo En Ming 21 Apr '21

21 Apr '21

Subject: Planning on setting up Postfix + Dovecot + Webmail + Apache + MariaDB + PHP using Webmin Control Panel in the Cloud Good day from the Republic of Singapore, I am planning to setup Postfix Email Server + Dovecot IMAP/POP3 Incoming Mail Server + Webmail (Roundcube or Squirrelmail) + Apache Web Server + MariaDB Database Server + PHP using Webmin Control Panel specifically, in the cloud, preferably Amazon EC2. Are there any very good and well written guides on doing this type of setup? I am looking forward to your reply. Thank you very much. Mr. Turritopsis Dohrnii Teo En Ming, 43 years old as of 21 April 2021 Wed, is a TARGETED INDIVIDUAL (TI) living in Singapore. He is an IT Consultant with a System Integrator (SI)/computer firm in Singapore. He is an IT enthusiast. -----BEGIN EMAIL SIGNATURE----- The Gospel for all Targeted Individuals (TIs): [The New York Times] Microwave Weapons Are Prime Suspect in Ills of U.S. Embassy Workers Link: https://www.nytimes.com/2018/09/01/science/sonic-attack-cuba-microwave.html ******************************************************************************************** Singaporean Targeted Individual Mr. Turritopsis Dohrnii Teo En Ming's Academic Qualifications as at 14 Feb 2019 and refugee seeking attempts at the United Nations Refugee Agency Bangkok (21 Mar 2017), in Taiwan (5 Aug 2019) and Australia (25 Dec 2019 to 9 Jan 2020): [1] https://tdtemcerts.wordpress.com/ [2] https://tdtemcerts.blogspot.sg/ [3] https://www.scribd.com/user/270125049/Teo-En-Ming -----END EMAIL SIGNATURE-----

1 0

Dovecot discards mail in a certain situation
by Aleš Pečnik 21 Apr '21

21 Apr '21

Hi all: I'm using Dovecot with Postfix (via LMTP). Because Postfix doesn't de-duplicate recipients, I'm using sieve to drop duplicates, following this recipe: https://serverfault.com/questions/112958/postfix-aliases-and-duplicate-e-ma… A "sieve_before" script contains: require ["duplicate"]; if duplicate { discard; stop; } It works well, except when the recipient is over quota. Dovecot is configured for tempfail in this case: quota_full_tempfail = yes When a mail arrives for a user who's over quota, Postfix logs this: AF4481700032: to=<?@?>, relay=?[private/dovecot-lmtp], delay=0.05, delays=0.05/0/0/0, dsn=4.2.2, status=deferred (host ?[private/dovecot-lmtp] said: 452 4.2.2 <?@?> Quota exceeded (mailbox for user is full) (in reply to end of DATA command)) Approximately 10 minutes later, Postfix retries, and log shows: dovecot: lmtp(?)<1742912><+EYHBXjidmBAmBoAcAY70w>: sieve: msgid=<489132c2-7b77-cd2f-ce60-439f6558ddcf@?>: marked message to be discarded if not explicitly delivered (discard action) postfix/lmtp[1742691]: AF4481700032: to=<?@?>, relay=?[private/dovecot-lmtp], delay=431, delays=431/0/0/0, dsn=2.0.0, status=sent (250 2.0.0 <?@?> +EYHBXjidmBAmBoAcAY70w Saved) postfix/qmgr[421343]: AF4481700032: removed The message is then nowhere to be found: it's not delivered to mailbox, bounce isn't sent to the sender, mail is not in postfix queue. Apparently it's been discarded. I'd say that at first LMTP delivery attempt, sieve records message as seen, even though it is eventually rejected (452 over quota). During second attempt sieve "recognizes" the message as seen before and discards it. (This is just my interpretation of what could be happening.) Is this a bug or have I misconfigured something? My dovecot -n output is below. Thanks for help, Aleš Pečnik # dovecot -n # 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7.2 () # OS: Linux 5.4.0-62-generic x86_64 Ubuntu 20.04.1 LTS auth_mechanisms = plain login auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@: auth_username_format = %Ln auth_worker_max_count = 3 default_vsz_limit = 1 G first_valid_gid = 12 first_valid_uid = 8 hostname = something.org last_valid_gid = 12 last_valid_uid = 8 listen = * mail_access_groups = mail mail_location = maildir:/var/dovecot/%u mail_plugins = " quota mailbox_alias mail_log notify acl" mail_privileged_group = mail namespace { location = maildir:/var/dovecot/public:INDEXPVT=/var/dovecot/%n/public prefix = Public. separator = . subscriptions = no type = public } namespace inbox { inbox = yes location = mailbox Drafts { auto = no special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Trash { auto = no special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql-master.conf.ext driver = sql master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile mailbox_alias_new = Sent Messages mailbox_alias_new2 = Sent Items mailbox_alias_new3 = Deleted Items mailbox_alias_new4 = Deleted Messages mailbox_alias_old = Sent mailbox_alias_old2 = Sent mailbox_alias_old3 = Trash mailbox_alias_old4 = Trash quota = maildir:User quota quota_rule = *:storage=200M quota_rule2 = Trash:storage=+50M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = file:/var/dovecot/%n/sieve;active=/var/dovecot/%n/sieve/active.sieve sieve_before = /var/lib/dovecot/sieve.d/ sieve_extensions = +notify +imapflags +editheader +duplicate } postmaster_address = postmaster(a)something.org protocols = " imap lmtp" quota_full_tempfail = yes service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 0 } process_min_avail = 12 service_count = 0 } service imap-postlogin { executable = script-login /etc/dovecot/postlogin.sh user = $default_internal_user } service imap { executable = imap imap-postlogin } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { port = 0 } } ssl_cert = </etc/ssl/my.crt ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = " quota mailbox_alias mail_log notify acl sieve" } protocol lda { mail_plugins = " quota mailbox_alias mail_log notify acl sieve" } protocol imap { mail_max_userip_connections = 30 mail_plugins = " quota mailbox_alias mail_log notify acl imap_quota imap_acl" }

2 2

IMAP session hangs on 8k-long commands if COMPRESS=DEFLATE is enabled
by Guilhem Moulin 21 Apr '21

21 Apr '21

Hi there, Dovecot 2.3.7 appears to hang when the client sends a long command after enabling the IMAP COMPRESS extension [RFC 4978]. PoC script attached along with the doveconf(1) output. Without COMPRESS=DEFLATE, and with the default ‘imap_max_line_length’ value (64k) I'm able send commands up to 65539 bytes long (that's 3 bytes more than 2¹⁶, so maybe the leading tag and the trailing CRLF aren't counted), after which Dovecot rightfully replies with tagged BAD responses. $ COMPRESS=n /tmp/many-fetch.pl 65539 […] C: x UID FETCH 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,… [65539 bytes] S: x OK Fetch completed (0.005 + 0.001 + 0.004 secs). C: x UID FETCH 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,… [65540 bytes] S: x BAD Error in IMAP command UID FETCH: IMAP command line too large (0.001 + 0.001 secs). C: x UID FETCH 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,… [65541 bytes] S: x BAD Error in IMAP command UID FETCH: IMAP command line too large (0.001 + 0.001 secs). With COMPRESS=DEFLATE however, the server never sends anything back when the command size exceeds 8192 bytes: $ COMPRESS=y /tmp/many-fetch.pl 8192 […] S: x OK [READ-ONLY] Examine completed (0.001 + 0.000 secs). C: x UID FETCH 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,… [8191 bytes] S: x OK Fetch completed (0.001 + 0.000 secs). C: x UID FETCH 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,… [8192 bytes] S: x OK Fetch completed (0.001 + 0.000 secs). C: x UID FETCH 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,… [8193 bytes] [ _hangs until the server terminates the session for inactivity_ ] S: * BYE Disconnected for inactivity. strace(1) show that Dovecot didn't send the response, even though the server log indicates that it's able to fully read the command. Interestingly, when setting ‘imap_max_line_length’ to 4096, the server also hangs after receiving a command of size >4096 bytes (instead of replying with a tagged BAD response). $ COMPRESS=y /tmp/many-fetch.pl 4096 C: x UID FETCH 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,… [4095 bytes] S: x OK Fetch completed (0.001 + 0.000 secs). C: x UID FETCH 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,… [4096 bytes] S: x OK Fetch completed (0.001 + 0.000 secs). C: x UID FETCH 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,… [4097 bytes] [ _hangs until the server terminates the session for inactivity_ ] I'm also able to reproduce the buggy behavior against Dovecot versions found in other Debian releases. It's identical with 2.3.4 (Buster), the server also hang past 8192 bytes. 2.2.27 (Stretch) and 2.2.13 (Jessie) are able to send replies to 8k-long commands, but hang past 65536 bytes. For all 3 releases, without COMPRESS=DEFLATE Dovecot doesn't hang and rightfully sends tagged BAD responses to commands of size >65539 bytes. So while the hanging behavior appears to be a long-standing bug, the fact that's it's now hit at min($imap_max_line_length, 8192) instead of $imap_max_line_length seems to be a regression from somewhere between 2.2.27 and 2.3.4. Cheers, -- Guilhem.

3 2

Quota warning invoked twice when moving messages to Trash folder
by Dan Conway 20 Apr '21

20 Apr '21

Hello all, I currently have quota warnings enabled through the following configuration on Dovecot 2.2.19: plugin { quota = dict:User quota::file:%h/Maildir/dovecot-quota quota_warning = storage=100%% quota-warning 100 %u %{userdb:quota_rule} %i quota_warning2 = storage=95%% quota-warning 95 %u %{userdb:quota_rule} %i quota_warning3 = storage=90%% quota-warning 90 %u %{userdb:quota_rule} %i } service quota-warning { executable = script /usr/local/bin/quota-warning.pl group = mailusers unix_listener quota-warning { group = mailusers mode = 0775 } } When using Thunderbird as a client for testing, I fill up an inbox until one of the warnings is invoked. (90%). The script runs, and the mailbox sits at about 93%. When moving a handful of messages to the Trash folder, the 100% quota-warning service is invoked. But getting the quota through 'doveadm quota get -u user' yields that same 93% used. I ran strace on the main process that is taking in IMAP commands from the Thunderbird client. Two lines are worth noting from that output: 134 uid store 2311:2312,2314:2317 +Flags (\\Seen)\r\n", 3619 137 uid move 2311:2312,2314:2317 \"INBOX.Trash\"\r\n", 3553 The first line sets the "Seen" flag on a list of messages, while the second line moves the messages to the "INBOX.Trash" folder. My guess as to what is happening is when the "uid move" command is received, The messages are copied first, then removed from the server, causing the quota to spike for a very short amount of time. Are those suspicions correct? After looking at RFC 6851, Section 3.3 (Semantics of MOVE and UID MOVE), it states: "Each message included in the set is moved, rather than copied ... ". Any idea why the spike in quota usage occurs? Thanks.

1 0

POP3 server fails to replicate deletions
by Stephen Gildea 20 Apr '21

20 Apr '21

When I delete messages with POP3, the change doesn't get replicated. To reproduce: run the pop3 server from the command line, type this to it: DELE 1 QUIT This correctly deletes the first message on the local server, but it does not send a replication notify message. Poking around a bit with gdb, it appears that the pop3 server decides incorrectly that the changes it is making are not worth notifying about. It is hard to find something that doesn't happen, and I would appreciate suggestions for further debugging this case. I do get the expected replication when new mail arrives and also when I manipulate my mailbox with IMAP. I have tested this with Dovecot 2.3.7.2 (on Ubuntu 20.04) and with the git HEAD, 2.3.14+. What may be the same bug was reported on this list last August against Dovecot 2.3.8, but it got no reply then. My mail store is on a local ext4 file system. I have replication enabled in my configuration: $ doveconf mail_plugins mail_plugins = mail_crypt notify replication $ doveconf -f protocol=pop3 mail_plugins mail_plugins = mail_crypt notify replication

2 1

fts_enforced=yes or body return NO [SERVERBUG] Internal error occurred when SEARCH
by Alessio Cecchi 20 Apr '21

20 Apr '21

Hi, we have some issues related to the fts_enforced option in dovecot config. Following the table in the docs (https://doc.dovecot.org/settings/plugin/fts-plugin/#plugin-fts-setting-fts-…) we cannot figure out the current behavior of the flag. In our case we have dovecot up and running, dovecot indexes updated using doveadm command (fts rescan/index), but solr instance offline for testing. We try to search using header subject. We tried "yes" and "body" fts_enforced flag's options but we get in telnet session: C: a search header subject "test" S: a NO [SERVERBUG] Internal error occurred. Refer to server log for more information. [2021-04-20 15:52:57] (0.005 + 0.000 + 0.004 secs). But dovecot-log don't include any specific error, except for a generic: Apr 20 15:52:57 imap dovecot: imap(test(a)emailtest.net) session=<t7RIwxyaxMxbj8Qu>: Error: fts_solr: Lookup failed: connect(10.0.0.2:8983) failed: Connection refused Is there any flag or method to check if a solr instance is up and running and, eventually, switching to the internal dovecot indexes instead ? Using the "no" option, for enforced flag, for us is unavailable because the related search could be too long and we could have timeout waiting the results. We aim to achieve the following result: if our solr instance is unavailable, we should switch the imap searches from full-text to headers only. Could you suggest something in this direction? We running doveoct 2.3.14. Thanks -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice

1 0

Replication between three servers
by Miro Igov 20 Apr '21

20 Apr '21

Hello, I have dovecot replication working fine between 2 servers with LDA configured to sync mail coming from Postfix. These 2 servers are used in master/slave mode - only the one server is used by IMAP clients and the second server is just a backup in case of failure. Second server is also MX backup and it happens that incoming mails come from it via Postfix and get synchronized with the first server. I would like to include third server as backup. How could be done while mail_replica accepts only one host?

2 4

What is the proper <luceneMatchVersion> value in solrconfig.xml for dovecot?
by Steve Dondley 19 Apr '21

19 Apr '21

I'm looking at config documentation for solr on dovecot: https://doc.dovecot.org/configuration_manual/fts/solr/ In the suggested solrconfig.xml file (https://raw.githubusercontent.com/dovecot/core/master/doc/solr-config-7.7.0…) it has the following line: <luceneMatchVersion>7.7.0</luceneMatchVersion> I'm running solr version 8.8.1, however. I'm wondering if I should change this line to: <luceneMatchVersion>8.8.1</luceneMatchVersion> Things seems to work fine with the 7.7.0 value but there is a comment in the config file that says:  I'm not familiar with Lucene or Solr so I'm uncertain as to what to set this to. Thanks.

1 0

Sending alert to client after quota has reached 100%
by Dan Conway 19 Apr '21

19 Apr '21

Hello all, I hope you are doing well. In regards to the quota_warning configuration, more specifically when an email is sent to the user when quota has passed a certain percentage, i.e.: quota_warning = <limit configuration> <quota-warning socket name> <parameters> #quota_warning2 = ... #quota_warning3 = ..etc.. Is it possible to communicate with the client by sending an ALERT IMAP message to stdout much like post-login scripting, rather than sending an email to the user? i.e: #!/bin/sh # Message goes to stdout printf "* OK [ALERT] This is an alert\r\n" exec "$@" Thank you.

2 1

Only INBOX is searched when using gmail with dovecot FTS and solr
by hi-dovecot＠mcarle.io 16 Apr '21

16 Apr '21

Hi, I am using https://github.com/docker-mailserver/docker-mailserver to setup my mailserver and added solr for full-text search. Nearly everything is working as expected. But I have a problem with the full-text search, as I have a lot of folders in my mailbox and I just can't find any mail in any of these folders except INBOX when I search with the Gmail android app. After analyzing the logs, I assume, that the bug is in the Gmail app, as it is only searching in the INBOX folder. I searched a bit and found some reports of other users confirming that. As this "design decision" (or bug) is quite old, I assume that Google won't fix it. But as many people are using Gmail, I assume that there must be a way to search all folders with Gmail. Sadly, I haven't found any possibility yet. As a workaround one may put a proxy in between and simply remove the "+box:123" filter from the path (e.g. with a small regex). With that, solr will return all found mails from that user, regardless of the folder. This may be a quite simple, but very dirty hack. Hence, I hope that you guys know a better way, a good way to overcome this issue and enable all Gmail users to search for email in all folders. Best regards, Marcel

3 2

mail_max_userip_connections per remote IP not working
by Alessio Cecchi 16 Apr '21

16 Apr '21

Hi, I'm tryng to set a specific mail_max_userip_connections for a remote IP (webmail IMAP software), but it seems not working: remote 1.2.3.4 { protocol imap { mail_max_userip_connections = 100 } } and also this isn't working remote 1.2.3.4 { mail_max_userip_connections = 100 } I insert it at the end of 20-imap.conf file. Is something wrong or is not supported? I'm running dovecot 2.3.14. Thanks -- Alessio Cecchi Postmaster @http://www.qboxmail.it https://www.linkedin.com/in/alessice

2 1

altmove reverse doesn't work
by Zdeněk Zámečník 14 Apr '21

14 Apr '21

I am having troubles with moving emails from alternative storage to primary storage. If I call command like: doveadm altmove -r -u myuser(a)mydomain.yyy ALL it is doing something (it took almost a minute at first time) but there are still emails in alternative storage. There is not reported any error. Below is my config: # Pigeonhole version 0.5.14 (1b5c82b2) # OS: Linux 5.3.18-3-pve x86_64 Debian 10.9 # Hostname: mail.mydomain.yyy auth_cache_size = 2 M auth_cache_ttl = 5 mins auth_cache_verify_password_with_worker = yes auth_master_user_separator = * auth_mechanisms = plain login dict { acl = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext quota = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no imap_client_workarounds = delay-newmail imap_hibernate_timeout = 5 secs imap_id_log = * imap_id_retain = yes login_trusted_networks = 127.0.0.0/8 mail_gid = 2000 mail_home = /var/vmail/%d/%n mail_location = mdbox:~/mdbox:ALT=/var/vmail-archive/%d/%n/mdbox mail_max_userip_connections = 60 mail_plugins = acl zlib fts quota mail_uid = 2000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vacation-seconds editheader imapsieve vnd.dovecot.imapsieve mdbox_rotate_interval = 1 days mdbox_rotate_size = 16 M namespace { inbox = yes location = mailbox "Deleted Items" { auto = no special_use = \Trash } mailbox "Deleted Messages" { auto = no special_use = \Trash } mailbox Drafts { auto = no special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox "Junk E-mail" { auto = no special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Items" { auto = no special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Spam { auto = no special_use = \Junk } mailbox Trash { special_use = \Trash } mailbox virtual/All { auto = no special_use = \All } prefix = separator = / type = private } namespace { inbox = no list = children location = mdbox:/var/vmail/%%d/%%n/mdbox:ALT=/var/vmail-archive/%%d/%%n/mdbox prefix = shared/%%u/ separator = / subscriptions = no type = shared } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } passdb { args = /etc/dovecot/masters.db driver = passwd-file master = yes pass = yes } plugin { acl = vfile acl_shared_dict = proxy::acl imapsieve_mailbox1_before = file:/var/vmail/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Spam imapsieve_mailbox2_before = file:/var/vmail/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Spam imapsieve_mailbox2_name = * imapsieve_mailbox3_before = file:/var/vmail/sieve/report-spam.sieve imapsieve_mailbox3_causes = COPY imapsieve_mailbox3_name = Junk imapsieve_mailbox4_before = file:/var/vmail/sieve/report-ham.sieve imapsieve_mailbox4_causes = COPY imapsieve_mailbox4_from = Junk imapsieve_mailbox4_name = * mailbox_alias_new = Sent Messages mailbox_alias_new2 = Sent Items mailbox_alias_new3 = Deleted Items mailbox_alias_old = Sent mailbox_alias_old2 = Sent mailbox_alias_old3 = Trash quota = dict:User quota::proxy::quota quota_grace = 10%% quota_rule2 = Trash:ignore quota_rule3 = Junk:ignore quota_warning = storage=80%% quota-warning 90 %u quota_warning2 = storage=85%% quota-warning 95 %u quota_warning3 = storage=95%% quota-warning 105 %u sieve = /var/vmail/%d/%n/sieve/.sieve sieve_after = /var/vmail/%d/%n/sieve/autoreply.sieve sieve_before = /var/vmail/sieve/global.sieve sieve_dir = /var/vmail/%d/%n/sieve sieve_extensions = +editheader +vacation-seconds sieve_global_dir = /var/vmail/sieve/ sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment sieve_max_redirects = 20 sieve_max_script_size = 2M sieve_pipe_bin_dir = /usr/lib/dovecot sieve_plugins = sieve_imapsieve sieve_extprograms sieve_vacation_send_from_recipient = yes } protocols = imap sieve lmtp pop3 service auth-worker { unix_listener auth-worker { user = vmail } user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } user = vmail } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-hibernate { unix_listener imap-hibernate { group = $default_internal_group mode = 0660 } } service imap-login { process_min_avail = 10 service_count = 0 vsz_limit = 512 M } service imap { executable = imap process_limit = 3500 unix_listener imap-master { user = $default_internal_user } vsz_limit = 2 G } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } service managesieve { process_limit = 400 } service pop3-login { process_min_avail = 3 service_count = 0 vsz_limit = 320 M } service pop3 { process_limit = 200 vsz_limit = 320 M } service quota-warning { executable = script /etc/dovecot/quota_warning.sh unix_listener quota-warning { mode = 0666 user = vmail } user = vmail } ssl_cert = </etc/ssl/private/dovecot.pem ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it stats_writer_socket_path = userdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } protocol sieve { managesieve_implementation_string = Dovecot Pigeonhole managesieve_max_compile_errors = 5 } protocol imap { imap_max_line_length = 6 M mail_plugins = quota imap_quota fts mailbox_alias imap_acl acl imap_zlib imap_sieve } protocol lmtp { mail_plugins = quota sieve acl }

3 3

Auth Cache for checkpassword
by Amol Kulkarni 14 Apr '21

14 Apr '21

Hello, I use the checkpassword driver for imap/pop authentication. To improve the performance I would like to enable authentication caching for it. However just adding the auth_cache_size and auth_cache_ttl doesn't work. No entries are cached. If I add a cache directive in the passdb, then the service doesnt start. I was wondering if the authentication caching is supported for the checkpassword driver. Please point me in the right direction. Thanks in advance, Amol

1 0

Tuning pop3-login client_limit
by Martin Olsen 13 Apr '21

13 Apr '21

Good day / evening / morning good dovecot people: Overview: We are seeing the following issue on one of our servers: pop3-login: Error: master(pop3): net_connect_unix(pop3) failed: Resource temporarily unavailable - http://wiki2.dovecot.org/SocketUnavailable (client-pid=5418, client-id=1, rip=__REMOVED__, created 534 msecs ago, received 0/4 bytes) We tried tuning the pop3-login -> client_limit parameter. However, as soon as we increased this parameter (and after restarting dovecot), we saw an onslaught of lmtp connection errors in postfix (unable to deliver via lmtp to dovecot). This led us to set the pop3-login -> client_limit back to 1000 again and the lmtp errors disappeared. This leads me to think that we should tune some other parameters as well. Increase overall limits or such. It seems like when increasing the client_limit for pop3-logins that somehow affected the lmtp deliverability. If anyone has any input on this that would be much appreciated. Cheers, Martin -- Version and config: # dovecot --version 2.2.36 (1f10bfa63) # dovecot -n # 2.2.36 (1f10bfa63): # OS: Centos 7.7 # Hostname: auth_mechanisms = auth_verbose = yes base_dir = __REMOVED__ default_client_limit = 2500 disable_plaintext_auth = no doveadm_password = first_valid_uid = 89 last_valid_uid = 89 lmtp_rcpt_check_quota = yes login_greeting = login_trusted_networks = mail_gid = 89 mail_plugins = " notify replication quota" mail_uid = 89 mbox_write_locks = fcntl namespace inbox { inbox = yes location = prefix = } passdb { args = driver = ldap } plugin { mail_replica = tcps: quota = maildir:User quota quota_rule = } protocols = pop3 lmtp replication_full_sync_interval = 3 hours replication_max_conns = 20 service aggregator { fifo_listener replication-notify-fifo { group = postfix mode = 0640 user = postfix } unix_listener replication-notify { group = postfix mode = 0640 user = postfix } } service auth { unix_listener auth { mode = 0660 } } service doveadm { inet_listener { port = 12345 ssl = yes } } service lmtp { process_min_avail = 10 unix_listener lmtp { group = postfix mode = 0640 user = postfix } } service pop3-login { client_limit = 1000 process_limit = 2048 process_min_avail = 10 service_count = 1 } service pop3 { client_limit = 1 process_limit = 2048 process_min_avail = 0 service_count = 1 }

1 0