- dovecot - dovecot.org

Live migrating index/control files
by Eirik Rye 26 Apr '21

26 Apr '21

Hello, We are running a director-based dovecot cluster where the user's maildir and the indexes/control files are, for legacy reasons, stored on separate NFS-backends, i.e.: mail_location = maildir:~/Maildir:CONTROL=/mail/control/%1u/%2u/%u:INDEX=/mail/index/%1u/%2u/%u Both ~/Maildir and /mail/{control,index}/ are separate NFS-mounts, and we now want to consolidate all files to the user's home directory instead. The end goal is to stop using NFS altogether. I have devised a four step plan for gradually migrating the index/control files into the user's maildir, on a per-user basis: 1. Kick the user in an attempt to flush the user's index files: `doveadm director kick <username>` 2. Rsync the index/control files into the user's maildir: `rsync -uav /mail/{index,control}/${user:0:1}/${user:0:2}/${user}/ /mail/${user:0:1}/${user:0:2}/${user}/Maildir/;` 3. Update the (redis-based) userdb for the specific user to override mail_location to use the default CONTROL/INDEX locations: `{"mail": "maildir:~/Maildir"}` 4. Kick the user one last time so that they reconnect using the new settings. Unfortunately, after running this for a small batch of users, we recieved a couple reports that their clients had started to resynchronize and redownload their mailboxes, which is exactly what we're trying to avoid. There are no index related error messages related to the few users that reported these issues. From my understanding, clients become confused when the control files are changed or lost (`dovecot-uidlist` and `dovecot-uidvalidity`), so I suspect the kick + rsync + kick procedure is not robust enough for this. Does anyone have any experience with performing such a migration without taking systems offline for the duration? Any suggestions to make this process more transparent for our users, so as to not induce panic when their mailboxes abruptly appear empty? Best regards, Eirik

2 5

How to Easily Set Up a Full-Featured Linux Mail Server on Ubuntu 18.04.5 LTS with iRedMail 1.4.0
by Turritopsis Dohrnii Teo En Ming 26 Apr '21

26 Apr '21

Subject: How to Easily Set Up a Full-Featured Linux Mail Server on Ubuntu 18.04.5 LTS with iRedMail 1.4.0 Good day from Singapore, I followed linuxbabe.com's Xiao Guoan's guide and successfully setup a full featured Linux mail server on Ubuntu 18.04.5 LTS with IRedMail 1.4.0. Author: Mr. Turritopsis Dohrnii Teo En Ming (TARGETED INDIVIDUAL) Country: Singapore Date: 25 April 2021 Sunday Type of Publication: PDF Manual Document Version: 20210425.01 (1st release) ***IMPORTANT NOTICE*** Please note that Turritopsis Dohrnii Teo En Ming’s guide is based on Xiao Guoan’s guide at linuxbabe.com. Reference Guide Used by Teo En Ming: How to Easily Set Up a Full-Featured Mail Server on Ubuntu 18.04 with iRedMail Link: https://www.linuxbabe.com/mail-server/ubuntu-18-04-iredmail-email-server Original Author: Xiao Guoan The following is a list of open-source software that will be automatically installed and configured by iRedMail. • Postfix SMTP server • Dovecot IMAP server • Nginx web server to serve the admin panel and webmail • OpenLDAP, MySQL/MariaDB, or PostgreSQL for storing user information • Amavised-new for DKIM signing and verification • SpamAssassin for anti-spam • ClamAV for anti-virus • Roundcube webmail • SOGo groupware, providing webmail, calendar (CalDAV), contacts (CardDAV), tasks and ActiveSync services. • Fail2ban for protecting SSH • mlmmj mailing list manager • Netdata server monitoring • iRedAPD Postfix policy server for greylisting Redundant Download Links for Teo En Ming's PDF Manual: [1] https://drive.google.com/file/d/1un8sLLmNSMIt7V6blWCvJEgwGvxMbd4B/view?usp=… [2] https://drive.google.com/file/d/1i0vY7kfYkobu563qoI3_qCZg7G7BFoYR/view?usp=… [3] https://drive.google.com/file/d/1U9MFN1EklLbA8TMweLV5ntiSJuBBVkpQ/view?usp=… [4] https://www.docdroid.net/dW70KtS/iredmail-setup-1st-release-pdf [5] https://www.mediafire.com/file/evar7j28knqyoj6/IRedMail+Setup+1st+Release.p… [6] https://www.scribd.com/document/504932780/IRedMail-Setup-1st-Release Mr. Turritopsis Dohrnii Teo En Ming, 43 years old as of 26 April 2021, is a TARGETED INDIVIDUAL living in Singapore. He is an IT Consultant with a System Integrator (SI)/computer firm in Singapore. He is an IT enthusiast. -----BEGIN EMAIL SIGNATURE----- The Gospel for all Targeted Individuals (TIs): [The New York Times] Microwave Weapons Are Prime Suspect in Ills of U.S. Embassy Workers Link: https://www.nytimes.com/2018/09/01/science/sonic-attack-cuba-microwave.html ******************************************************************************************** Singaporean Targeted Individual Mr. Turritopsis Dohrnii Teo En Ming's Academic Qualifications as at 14 Feb 2019 and refugee seeking attempts at the United Nations Refugee Agency Bangkok (21 Mar 2017), in Taiwan (5 Aug 2019) and Australia (25 Dec 2019 to 9 Jan 2020): [1] https://tdtemcerts.wordpress.com/ [2] https://tdtemcerts.blogspot.sg/ [3] https://www.scribd.com/user/270125049/Teo-En-Ming -----END EMAIL SIGNATURE-----

1 0

cannot see my mails
by Jean-Max Reymond 26 Apr '21

26 Apr '21

Hi, After a change of server, I cannot see my mails. postfix is OK and I receive emails. If I send emails, with roundcube or thunderbird, I can see these new sent emails. Access rights seems OK. Dovecot with debug trace does not complain. Any tips ? append_dot_mydomain = no biff = no broken_sasl_auth_clients = yes compatibility_level = 2 content_filter = smtp-amavis:[127.0.0.1]:10024 home_mailbox = Maildir/ local_recipient_maps = $virtual_mailbox_maps local_transport = virtual mailbox_command = /usr/lib/dovecot/deliver -c /etc/dovecot/dovecot.conf -m "${EXTENSION}" message_size_limit = 30720000 milter_default_action = accept milter_protocol = 2 mydestination = localhost non_smtpd_milters = inet:localhost:12345 readme_directory = no slow_destination_concurrency_limit = 2 slow_destination_recipient_limit = 20 smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt smtp_use_tls = yes smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) smtpd_milters = inet:localhost:12345 smtpd_recipient_restrictions = reject_unknown_sender_domain, reject_unknown_recipient_domain, reject_unauth_pipelining, permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination smtpd_sasl_auth_enable = yes smtpd_sasl_authenticated_header = yes smtpd_sasl_local_domain = $myhostname smtpd_sasl_path = private/dovecot-auth smtpd_sasl_security_options = noanonymous smtpd_sasl_type = dovecot smtpd_sender_restrictions = reject_unknown_sender_domain smtpd_tls_auth_only = yes smtpd_tls_cert_file = /etc/dovecot/private/dovecot.pem smtpd_tls_key_file = /etc/dovecot/private/dovecot.key smtpd_tls_mandatory_ciphers = medium smtpd_tls_mandatory_protocols = SSLv3, TLSv1 smtpd_tls_received_header = yes smtpd_use_tls = yes tls_random_source = dev:/dev/urandom transport_maps = hash:/etc/postfix/transport vacation_destination_recipient_limit = 1 virtual_alias_maps = proxy:mysql:/etc/postfix/mysql/virtual_alias_maps.cf virtual_gid_maps = static:8 virtual_mailbox_base = /home/Mails virtual_mailbox_domains = proxy:mysql:/etc/postfix/mysql/virtual_domains_maps.cf virtual_mailbox_limit = 204800000 virtual_mailbox_maps = proxy:mysql:/etc/postfix/mysql/virtual_mailbox_maps.cf virtual_minimum_uid = 8 virtual_transport = virtual virtual_uid_maps = static:8 yahoo_destination_concurrency_limit = 4 yahoo_destination_rate_delay = 1s yahoo_destination_recipient_limit = 2 yahoo_initial_destination_concurrency = 1 -- Jean-Max Reymond CKR Solutions Open Source https://www.ckr-solutions.com

3 9

Panic: file ostream.c: assertion failed when using mail_filter on RHEL8
by Martijn Brinkers (list) 24 Apr '21

24 Apr '21

Hi, I installed dovecot on a new up-to-date RHEL8 test system and configured a simple mail_filter plugin which only copies the input back to the output (i.e., the filter does nothing special). On RHEL8 I get the following panic when saving the mail. The Dovecot version that comes with RHEL8 is dovecot-2.3.8-4.el8.x86_64 The mail_filter works correctly on an Ubuntu machine )2.2.33.2- 1ubuntu4.7) Any idea what's causing this panic on RHEL8 and how to fix it? Kind regards, Martijn Brinkers Log output (panic log is at the bottom) Apr 23 10:35:54 ciphermail-webmail dovecot[18680]: master: Dovecot v2.3.8 (9df20d2db) starting up for imap Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: Loading modules from directory: /usr/lib64/dovecot/auth Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/lib20_auth_var_expand_crypt.so Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.so Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: Read auth token secret from /var/run/dovecot/auth-token-secret.dat Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: auth client connected (pid=19771) Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: client in: AUTH 1 PLAIN service=imap secured se ssion=i7avXaHAuLh/AAAB lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=47288 resp=<hidden> Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: dict( test(a)example.com,127.0.0.1,<i7avXaHAuLh/AAAB>): Performing passdb lookup Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth-worker(19774): Debug: Loading modules from directory: /usr/lib64/dovecot/auth Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth-worker(19774): Debug: Module loaded: /usr/lib64/dovecot/auth/lib20_auth_var_expand_crypt.so Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth-worker(19774): Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.so Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth-worker(19774): Debug: conn unix:auth-worker (pid=19773,uid=97): Server accepted connection (fd=13) Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth-worker(19774): Debug: conn unix:auth-worker (pid=19773,uid=97): Sending version handshake Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth-worker(19774): Debug: conn unix:auth-worker (pid=19773,uid=97): auth-worker<1>: Handling PASSV request Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth-worker(19774): Debug: dict(test(a)example.com,127.0.0.1,<i7avXaHAuLh/AAAB>): Performing passdb lookup Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth-worker(19774): Debug: dict(test(a)example.com,127.0.0.1,<i7avXaHAuLh/AAAB>): Lookup: shared/passdb/test(a)example.com/test = {"userdb_email":"test(a)example.com ","userdb_quota_rule":"*:bytes=1073741824","password":"{SSHA256.b64}SF5 hEeUZVO4ydasqNb040KHWGyin791L40BYR8cXf+I0EGeetzq5Cg==","userdb_password ":"test","userdb_home":" /var/vmail/test=40example.com(a)ciphermail.private","user":" test=40example.com(a)ciphermail.private"} Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth-worker(19774): Debug: dict(test(a)example.com,127.0.0.1,<i7avXaHAuLh/AAAB>): username changed test(a)example.com -> test=40example.com(a)ciphermail.private Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth-worker(19774): Debug: dict(test=40example.com(a)ciphermail.private,127.0.0.1,<i7avXaHAuL h/AAAB>): Finished passdb lookup Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth-worker(19774): Debug: conn unix:auth-worker (pid=19773,uid=97): auth-worker<1>: Finished Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: dict( test(a)example.com,127.0.0.1,<i7avXaHAuLh/AAAB>): username changed test(a)example.com -> test=40example.com(a)ciphermail.private Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: dict( test=40example.com(a)ciphermail.private,127.0.0.1,<i7avXaHAuLh/AAAB>): Finished passdb lookup Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: auth( test=40example.com(a)ciphermail.private,127.0.0.1,<i7avXaHAuLh/AAAB>): Auth request finished Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: client passdb out: OK 1 user=test=40example.com(a)ciphermail.private original_user=test(a)example.com Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: master in: REQUEST 3373400065 19771 1 603cbf25103fc9a6 77ceabe45f077e40 session_pid=19776 request_auth_token Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: prefetch(test=40example.com(a)ciphermail.private,127.0.0.1,<i7avXaHAuLh/A AAB>): Performing userdb lookup Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: prefetch(test=40example.com(a)ciphermail.private,127.0.0.1,<i7avXaHAuLh/A AAB>): success Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: prefetch(test=40example.com(a)ciphermail.private,127.0.0.1,<i7avXaHAuLh/A AAB>): Finished userdb lookup Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: auth: Debug: master userdb out: USER 3373400065 test=40example.com(a)ciphermail.private email=test(a)example.com quota_rule=*:bytes=1073741824 password=<hidden> home=/var/vmail/test=40example.com(a)ciphermail.private auth_mech= PLAIN auth_token=819d2ad4117e4e4139991fab046e13304880e935 auth_user=test(a)example.com Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap-login: Login: user=<test=40example.com(a)ciphermail.private>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=19776, secured, session=<i7avXaHAuLh/AAAB> Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Loading modules from directory: /usr/lib64/dovecot Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Module loaded: /usr/lib64/dovecot/lib10_mail_filter_plugin.so Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Module loaded: /usr/lib64/dovecot/lib10_quota_plugin.so Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Module loaded: /usr/lib64/dovecot/lib11_imap_quota_plugin.so Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Added userdb setting: plugin/auth_mech=PLAIN Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Added userdb setting: plugin/email=test(a)example.com Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Added userdb setting: plugin/password=<hidden> Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Added userdb setting: plugin/quota_rule=*:bytes=1073741824 Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Effective uid=986, gid=983, home=/var/vmail/test=40example.com(a)ciphermail.private Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: mail_filter: Filtering mail_filter via socket /var/run/dovecot/mail- filter Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: mail_filter: Filtering mail_filter_out via socket /var/run/dovecot/mail-filter-out Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Quota root: name=User quota backend=maildir args= Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Quota rule: root=User quota mailbox=* bytes=1073741824 messages=0 Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Quota rule: root=User quota mailbox=Trash bytes=+107374182 (10%) messages=0 Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Quota grace: root=User quota bytes=107374182 (10%) Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: maildir++: root=/var/vmail/test=40example.com(a)ciphermail.private, index=, indexpvt=, control=, inbox=/var/vmail/test=40example.com(a)ciphermail.private, alt= Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: quota: quota_over_flag check: quota_over_script unset - skipping Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Debug: Mailbox Drafts: Mailbox opened because: APPEND Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Panic: file ostream.c: line 201 (o_stream_flush): assertion failed: (stream- >stream_errno != 0) Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap( test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0xf895b) [0x7f8f1d3d995b] -> /usr/lib64/dovecot/libdovecot.so.0(+0xf89f7) [0x7f8f1d3d99f7] -> /usr/lib64/dovecot/libdovecot.so.0(+0x55b05) [0x7f8f1d336b05] -> /usr/lib64/dovecot/libdovecot.so.0(+0x5a1bd) [0x7f8f1d33b1bd] -> /usr/lib64/dovecot/libdovecot- storage.so.0(maildir_save_finish+0x99) [0x7f8f1d723fa9] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0xfcee) [0x7f8f1bd5acee] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_save_finish+0x7d) [0x7f8f1d6fafdd] -> dovecot/imap(+0x132fd) [0x564a032332fd] -> dovecot/imap(command_exec+0x78) [0x564a03241cd8] -> dovecot/imap(+0x128ad) [0x564a032328ad] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x73) [0x7f8f1d3f10c3] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x135) [0x7f8f1d3f2775] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x50) [0x7f8f1d3f1170] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x48) [0x7f8f1d3f12d8] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x17) [0x7f8f1d368eb7] -> dovecot/imap(main+0x335) [0x564a03232445] -> /lib64/libc.so.6(__libc_start_main+0xf3) [0x7f8f1ca5b803] -> dovecot/imap(_start+0x2e) [0x564a032325fe] Apr 23 10:36:17 ciphermail-webmail dovecot[18682]: imap(test=40example.com(a)ciphermail.private)<19776><i7avXaHAuLh/AAAB>: Fatal: master: service(imap): child 19776 killed with signal 6 (core not dumped - https://dovecot.org/bugreport.html#coredumps - set /proc/sys/fs/suid_dumpable to 2)

1 1

Why was mail-filter plugin removed in version 2.3.14?
by Martijn Brinkers (list) 23 Apr '21

23 Apr '21

Hi, Mail-filter plugin was removed in version 2.3.14. Any idea why was this removed? Is there a replacement for the mail-filter plugin? Kind regards, Martijn Brinkers

1 0

Size of transaction log file is increasing continuously if dsync is failed
by Duc Anh Do 22 Apr '21

22 Apr '21

Hi Dovecot community, I would like to share a problem I saw a few days ago. Any comment is highly appreciated. *Steps to reproduce* 0. Version of Dovecot I use is v2.3.11.3. 1. Prepare 2 Dovecot backends: - One backend is used as the source of dsync command. Other backend is used as the destination of the dsync command. - Mailbox format is Maildir. Mail dir of two test accounts is empty. 2. Prepare the test account in the source of dsync: - Send a test message to the test account. For example, I use swaks: swaks --server 127.0.0.1 --port 24 --protocol LMTP --to user1@local - Use doveadm expunge command to clean mail dir of the test account: doveadm expunge -u user1@local mailbox INBOX all 3. Prepare the test account in the destination of dsync: - Log in test account via IMAP then select INBOX first: telnet 127.0.0.1 imap ... s login user1@local 1 ... s select INBOX ... s logout - Then perform dsync: doveadm -o imapc_host=backend2.local -o imapc_port=143 -o imapc_user=user1@local -o imapc_password=1 backup -s '' -R -u user1@local imapc: - Confirm error occurs when using dsync: dsync(user1@local): Info: imapc(backend2.local:143): Connected to 192.168.100.2:143 (local 192.168.100.27:58850) dsync(user1@local): Error: Mailbox INBOX sync: mailbox_delete failed: INBOX can't be deleted. AQAAAHm4+Jk= 4. Log in the test account in dsync destination then select INBOX. Repeat many times. I can see the size of the transaction log file is increasing continuously. *Before*: -rw-------. 1 vmail vmail 456 Apr 22 10:54 dovecot.index.log *After*: -rw-------. 1 vmail vmail 544 Apr 22 10:59 dovecot.index.log -rw-------. 1 vmail vmail 632 Apr 22 11:00 dovecot.index.log -rw-------. 1 vmail vmail 720 Apr 22 11:00 dovecot.index.log *Problems* - Size of the transaction log file is increasing continuously. Do you think this is an issue of Dovecot? - The value of *mail_index_log_rotate_max_size* is 1M by default. However the transaction log file dovecot.index.log is not rotated when its size exceeds limit. - Because the size of the transaction log file is too big and it is not rotated, I think "out of memory" error will occur when I try to use IMAP SELECT or POP3 STAT command (if the virtual size limit is small enough). - If "size of the transaction log file is increasing continuously" is an issue, what should I do to avoid it or is there any workaround? I understand I should fix the error *Error: Mailbox INBOX sync: mailbox_delete failed: INBOX can't be deleted* first before doing anything next. I think removing the transaction log file is an understandable workaround. *Investigaton* - About the error *Error: Mailbox INBOX sync: mailbox_delete failed: INBOX can't be deleted*, I can understand the cause after reading this thread https://dovecot.org/pipermail/dovecot/2014-April/095588.html 2) Maildir + INBOX + backup/sync/replicate > In the test scenarios where the INBOX on one side was to be completely > removed, e.g. doveadm backup -R the dsync failed and > nothing was synced to the target. This is because before moving the source > mails to the mailbox, dsync cleans out the old ones ( > -R preserves nothing) and in Maildir the INBOX can not be removed. This is > a feature/not easily solvable, because in Maildir > INBOX is different from other folders. > - The transaction log file is rotated whenever there's any change in index file. I can confirm by sending a test message to the test account in dsync destination. I can see that logic in *src/lib-index/mail-index-sync.c* and *src/lib-index/mail-index-write.c*. After that, the size of the new transaction log file is not increasing anymore. - When the error *Error: Mailbox INBOX sync: mailbox_delete failed: INBOX can't be deleted* occurred, I tried to dump the transaction log file and I saw these records: record: offset=432, type=index-deleted (ext), size=12 record: offset=444, type=index-undeleted (ext), size=12 - Do you think these records relate to the issue that the size of the transaction log file is increasing continuously? - Did the error *Error: Mailbox INBOX sync: mailbox_delete failed: INBOX can't be deleted* make content of transaction log file incomplete/incorrect? Regards, Anh Do

1 0

CA certs for Dovecot-as-client (proxy)
by Peter Mogensen 21 Apr '21

21 Apr '21

Hi, When using proxy=y, ssl=yes (Dovecot 2.3.13) I consistently get this logged when trying to validate the remote server cert. "Disconnected by server: Connection closed: Received invalid SSL certificate: unable to get local issuer certificate: /C=BE/O=GlobalSign nv-sa/CN=AlphaSSL CA - SHA256 - G2 (check ssl_client_ca_* settings?)" As I read the 2.3.x documentation (and the error logged) Dovecot needs to have the trusted CA cert with ssl_client_ca_file or ssl_client_ca_dir. So, I've tried every combination of putting the cert (and the GlobalSign root CA signing it) in ssl_client_ca_dir and individually and as a bundle in ssl_client_ca_file without luck. But even though I can verify the cert with "openssl s_client -connect" and with "openssl verify", no matter what I put in the ssl_client_ca_* settings it seems Dovecot just ignores it. It does complain though, if I point it to a non-existent file, but not if I just fill the file with invalid cert data which can't be parsed. I end up getting in doubt whether it consults the cert data at all. I'm a bit at loss on how to debug this further, short of running it in gdb. "verbose_ssl" doesn't really say anything about the process of find a CA cert to check with. Have I misunderstood the config? /Peter

3 2

systemd timeout on startup after upgrade
by Felix Zielcke 21 Apr '21

21 Apr '21

Hello *, I have upgraded today a Debian buster (stable) system to bullseye (still testing). After upgrade I did a complete reboot of the VM. Dovecot version changed from 1:2.3.4.1-5+deb10u6 to 1:2.3.13+dfsg1-1 `systemctl start dovecot` now complains Job for dovecot.service failed because a timeout was exceeded. As a workaround I set "TimeoutStartSec=infinity" in dovecot.service. But `systemctl status` now says: Active: activating (start) since Sat 2021-04-17 17:53:58 CEST; 1h 23min ago Dovecot itself is working fine without problems. LMTP accepts new mails and IMAP works. It's just that systemd doestn't notice it running. Any idea what causes this? Config is unchanged from the working 2.3.4.1 system. # doveconf -n # 2.3.13 (89f716dc2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.13 (cdd19fe3) # OS: Linux 5.10.0-6-cloud-amd64 x86_64 Debian bullseye/sid # Hostname: daedalus.z-51.de auth_master_user_separator = * auth_mechanisms = plain login auth_ssl_username_from_cert = yes auth_verbose = yes auth_verbose_passwords = plain default_vsz_limit = 2 G first_valid_uid = 8 import_environment = TZ last_valid_uid = 8 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c ssl_security=%k login_trusted_networks = 127.0.0.1 ::1 mail_gid = mail mail_location = maildir:~/Maildir mail_plugins = acl zlib quota fts fts_xapian mail_privileged_group = mail mail_uid = mail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded- character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { special_use = \Junk } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = scheme=CRYPT username_format=%Lu /etc/dovecot/users driver = passwd-file } plugin { acl = vfile:/etc/dovecot/global-acls:cache_secs=300 fts = xapian fts_autoindex = yes fts_autoindex_exclude = \Junk fts_autoindex_exclude2 = \Trash fts_xapian = partial=3 full=20 attachments=0 verbose=0 imapsieve_mailbox1_before = file:/var/mail/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Spam imapsieve_mailbox2_before = file:/var/mail/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Spam imapsieve_mailbox2_name = * sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /var/mail/sieve/before.sieve sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute sieve_pipe_bin_dir = /var/mail/sieve sieve_plugins = sieve_imapsieve sieve_extprograms sieve_redirect_envelope_from = recipient sieve_vacation_default_period = 1d stats_refresh = 30 secs stats_track_cmds = yes } postmaster_address = postmaster(a)z-51.de protocols = " imap lmtp sieve" service auth-worker { user = $default_internal_user } service auth { unix_listener auth-client { group = Debian-exim mode = 0777 user = Debian-exim } unix_listener auth-userdb { group = Debian-exim mode = 0777 user = Debian-exim } vsz_limit = 2 G } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service indexer-worker { vsz_limit = 0 } service lmtp { user = mail } service managesieve-login { inet_listener sieve { address = 127.0.0.1 ::1 port = 4190 } service_count = 1 } service managesieve { process_limit = 1024 } service stats { fifo_listener stats-mail { mode = 0600 user = mail } unix_listener stats-writer { user = mail } } ssl = required ssl_cert = </etc/letsencrypt/live/daedalus.z-51.de/fullchain.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes userdb { args = username_format=%Lu /etc/dovecot/users default_fields = uid=mail gid=mail home=/var/mail/%d/%n driver = passwd-file } verbose_proctitle = yes protocol lmtp { auth_username_format = %Lu mail_plugins = acl zlib quota fts fts_xapian sieve } protocol lda { mail_plugins = acl zlib quota fts fts_xapian sieve } protocol imap { mail_plugins = acl zlib quota fts fts_xapian imap_zlib imap_acl imap_quota imap_sieve }

5 9

Planning on setting up Postfix + Dovecot + Webmail + Apache + MariaDB + PHP using Webmin Control Panel in the Cloud
by Turritopsis Dohrnii Teo En Ming 21 Apr '21

21 Apr '21

Subject: Planning on setting up Postfix + Dovecot + Webmail + Apache + MariaDB + PHP using Webmin Control Panel in the Cloud Good day from the Republic of Singapore, I am planning to setup Postfix Email Server + Dovecot IMAP/POP3 Incoming Mail Server + Webmail (Roundcube or Squirrelmail) + Apache Web Server + MariaDB Database Server + PHP using Webmin Control Panel specifically, in the cloud, preferably Amazon EC2. Are there any very good and well written guides on doing this type of setup? I am looking forward to your reply. Thank you very much. Mr. Turritopsis Dohrnii Teo En Ming, 43 years old as of 21 April 2021 Wed, is a TARGETED INDIVIDUAL (TI) living in Singapore. He is an IT Consultant with a System Integrator (SI)/computer firm in Singapore. He is an IT enthusiast. -----BEGIN EMAIL SIGNATURE----- The Gospel for all Targeted Individuals (TIs): [The New York Times] Microwave Weapons Are Prime Suspect in Ills of U.S. Embassy Workers Link: https://www.nytimes.com/2018/09/01/science/sonic-attack-cuba-microwave.html ******************************************************************************************** Singaporean Targeted Individual Mr. Turritopsis Dohrnii Teo En Ming's Academic Qualifications as at 14 Feb 2019 and refugee seeking attempts at the United Nations Refugee Agency Bangkok (21 Mar 2017), in Taiwan (5 Aug 2019) and Australia (25 Dec 2019 to 9 Jan 2020): [1] https://tdtemcerts.wordpress.com/ [2] https://tdtemcerts.blogspot.sg/ [3] https://www.scribd.com/user/270125049/Teo-En-Ming -----END EMAIL SIGNATURE-----

1 0

Dovecot discards mail in a certain situation
by Aleš Pečnik 21 Apr '21

21 Apr '21

Hi all: I'm using Dovecot with Postfix (via LMTP). Because Postfix doesn't de-duplicate recipients, I'm using sieve to drop duplicates, following this recipe: https://serverfault.com/questions/112958/postfix-aliases-and-duplicate-e-ma… A "sieve_before" script contains: require ["duplicate"]; if duplicate { discard; stop; } It works well, except when the recipient is over quota. Dovecot is configured for tempfail in this case: quota_full_tempfail = yes When a mail arrives for a user who's over quota, Postfix logs this: AF4481700032: to=<?@?>, relay=?[private/dovecot-lmtp], delay=0.05, delays=0.05/0/0/0, dsn=4.2.2, status=deferred (host ?[private/dovecot-lmtp] said: 452 4.2.2 <?@?> Quota exceeded (mailbox for user is full) (in reply to end of DATA command)) Approximately 10 minutes later, Postfix retries, and log shows: dovecot: lmtp(?)<1742912><+EYHBXjidmBAmBoAcAY70w>: sieve: msgid=<489132c2-7b77-cd2f-ce60-439f6558ddcf@?>: marked message to be discarded if not explicitly delivered (discard action) postfix/lmtp[1742691]: AF4481700032: to=<?@?>, relay=?[private/dovecot-lmtp], delay=431, delays=431/0/0/0, dsn=2.0.0, status=sent (250 2.0.0 <?@?> +EYHBXjidmBAmBoAcAY70w Saved) postfix/qmgr[421343]: AF4481700032: removed The message is then nowhere to be found: it's not delivered to mailbox, bounce isn't sent to the sender, mail is not in postfix queue. Apparently it's been discarded. I'd say that at first LMTP delivery attempt, sieve records message as seen, even though it is eventually rejected (452 over quota). During second attempt sieve "recognizes" the message as seen before and discards it. (This is just my interpretation of what could be happening.) Is this a bug or have I misconfigured something? My dovecot -n output is below. Thanks for help, Aleš Pečnik # dovecot -n # 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7.2 () # OS: Linux 5.4.0-62-generic x86_64 Ubuntu 20.04.1 LTS auth_mechanisms = plain login auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@: auth_username_format = %Ln auth_worker_max_count = 3 default_vsz_limit = 1 G first_valid_gid = 12 first_valid_uid = 8 hostname = something.org last_valid_gid = 12 last_valid_uid = 8 listen = * mail_access_groups = mail mail_location = maildir:/var/dovecot/%u mail_plugins = " quota mailbox_alias mail_log notify acl" mail_privileged_group = mail namespace { location = maildir:/var/dovecot/public:INDEXPVT=/var/dovecot/%n/public prefix = Public. separator = . subscriptions = no type = public } namespace inbox { inbox = yes location = mailbox Drafts { auto = no special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Trash { auto = no special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql-master.conf.ext driver = sql master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile mailbox_alias_new = Sent Messages mailbox_alias_new2 = Sent Items mailbox_alias_new3 = Deleted Items mailbox_alias_new4 = Deleted Messages mailbox_alias_old = Sent mailbox_alias_old2 = Sent mailbox_alias_old3 = Trash mailbox_alias_old4 = Trash quota = maildir:User quota quota_rule = *:storage=200M quota_rule2 = Trash:storage=+50M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = file:/var/dovecot/%n/sieve;active=/var/dovecot/%n/sieve/active.sieve sieve_before = /var/lib/dovecot/sieve.d/ sieve_extensions = +notify +imapflags +editheader +duplicate } postmaster_address = postmaster(a)something.org protocols = " imap lmtp" quota_full_tempfail = yes service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 0 } process_min_avail = 12 service_count = 0 } service imap-postlogin { executable = script-login /etc/dovecot/postlogin.sh user = $default_internal_user } service imap { executable = imap imap-postlogin } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { port = 0 } } ssl_cert = </etc/ssl/my.crt ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = " quota mailbox_alias mail_log notify acl sieve" } protocol lda { mail_plugins = " quota mailbox_alias mail_log notify acl sieve" } protocol imap { mail_max_userip_connections = 30 mail_plugins = " quota mailbox_alias mail_log notify acl imap_quota imap_acl" }

2 2