January 2023 - dovecot

How do you scale dovecot for good performance with Roundcube webmailer in front? (hitting limits without exhausting resources)
by Malte Schmidt 27 Jan '23

27 Jan '23

Good day, I am currently setting up/debugging a webmailer-only setup using Roundcube (latest version) with Dovecot (2.3.20, latest as of now). Generally speaking the setup performs OK, I wrote a JMeter Roundcube script to test the Roundcube. This includes actions like listing mails and fetching them. I am hitting a somehow dubious limit of 600 req/s though. (This includes all kinds of HTTP calls to Roundcube, not only the ones triggering IMAP). When the dovecot becomes unavailable, the performance without mails jumps to 7700 req/s. Testing is done with 4 JMeter-servers and one client. My setup is a Dovecot with MySQL (Percona XtraDB) backend. Mails are encrypted with mail_crypt and EC keys. Authentication is done in parallel (auth_cache_verify_with_workers=yes). Where possible min_available_processes have been set equal to the threads available on the Dovecot machine (64). Hardware is a 64 thread Xeon CPU at 2.10 GHz, 96 GB RAM, SSDs as backing storage IOPS read 40000/write 13000. What settings do you recommend and how was your experience with Roundcube and its performance in general (what should be possible with that kind of HW?)? Best regards and thanks in advance!

3 3

[REVISIONS] How to install SSL Certificate on Dovecot and Postfix
by Turritopsis Dohrnii Teo En Ming 27 Jan '23

27 Jan '23

Subject: [REVISIONS] How to install SSL Certificate on Dovecot and Postfix Good day from Singapore, Author: Mr. Turritopsis Dohrnii Teo En Ming (TARGETED INDIVIDUAL) Country: Singapore Date of Implementation: 6 Jan 2023 Friday SECTION ON DOVECOT =================== # cd /etc/dovecot/conf.d # nano 10-ssl.conf ssl = yes ssl_cert = </home/teo-en-ming-corp/2023.crt ssl_key = </home/teo-en-ming-corp/2023.key ssl_ca = </home/teo-en-ming-corp/inter2023.crt # nano /etc/dovecot/dovecot.conf protocols = imap pop3 !include conf.d/*.conf SECTION ON POSTFIX =================== # nano /etc/postfix/main.cf smtpd_use_tls = yes smtpd_tls_auth_only = yes smtpd_tls_key_file = /etc/postfix/2023.key smtpd_tls_cert_file = /etc/postfix/chain2023.crt smtpd_tls_CAfile = /etc/postfix/inter2023.crt Note: chain2023.crt is made by combining 2023.crt and inter2023.crt # nano /etc/postfix/master.cf submission inet n - n - - smtpd smtps inet n - n - - smtpd tlsmgr unix - - n 1000? 1 tlsmgr That's all. Regards, Mr. Turritopsis Dohrnii Teo En Ming Targeted Individual in Singapore Blogs: https://tdtemcerts.blogspot.com https://tdtemcerts.wordpress.com

1 0

LDAP scheme for Dovecot
by Daniel Betschart 24 Jan '23

24 Jan '23

I use an LDAP server for user administration, the users can manage themselves with the LDAP Account Manager (LAM). I would like to give the possibility to be able to define the quota per user. LAM provides support for certain schemes, but the qmail scheme is missing in the free variant. So I used the Courier scheme. Now, while LAM correctly stores the value in bytes, it appends the letter S. Unfortunately, the mail server Dovecot cannot do anything with this letter S. Is there a better scheme for Dovecot? I haven't found any until now. Or can I remove the letter S at the end in the Dovecot configuration file via regex?

1 0

Why does doveadm always run doveconf?
by Bert Van de Poel 21 Jan '23

21 Jan '23

Dear fellow Dovecot users, I've recently been aiding in the fixing up of the old dovecot_stats_ munin plugin. Currently, it still parses the output of doveadm oldstats, as it is quite an old script. Regardless of any feelings we may all have about oldstats, I was quite surprised to find that doveadm requires quite broad privileges (in my case root privileges) to function properly. It seems that any call to doveadm, even for "doveadm oldstats dump domain" runs doveconf and will attempt to fully parse my configuration including trying to read SSL/TLS certificates. Due to this choice, the fact the FIFO of oldstats can be given low privileges, doveadm still has to be invoked with high privileges or it will fail at the stage of verifying configuration. Now I'm wondering why it's the case that a command such as "doveadm oldstats dump domain" is invoking doveconf and therefore has these kinds of limitations. While for basically all non-stats functions of doveadm, running as root (or similar) makes a lot of sense, I'd argue it doesn't for oldstats (and maybe also the new stats?), which simply talk to a socket. Is there a certain reasoning behind this, or is this accidental behaviour because of the standard operations doveadm always performs? Any elaboration would be more than welcome! Kind regards, Bert

2 3

Different username for passdb and userdb
by support+dovecot＠shellworld.net 21 Jan '23

21 Jan '23

Hello I am attempting to use an external IMAP server for authentication, on Dovecot 2.3.16 (7e2e900c1a), running on Ubuntu 22.04. The external server requires usernames of the format: user@domain However, my local users, where the mail is actually drawn from, are of the format: user (passwd file driven). I was able to log in with just "user", and have it both authenticate remotely, and access local mail, using: passdb { driver = imap args = host=remotehost.com ssl=imaps port=993 user=%n@%d ssl_ca_dir=/etc/ssl/certs } userdb { driver = passwd args = blocking=no } However, the remote SMTP server, which I will not be proxying, requires the same user@domain format for usernames. I would rather not ask users to use "user@domain" for SMTP, and "user" for IMAP. They should only have to use "user@domain" for login everywhere. Is there a way to do this? The docs suggest so (AuthDatabase.PasswdFile.txt), but if so I must be botching it. Things I have tried: passdb { driver = imap args = host=remotehost.com ssl=imaps port=993 ssl_ca_dir=/etc/ssl/certs } userdb { driver = passwd args = blocking=no override_fields = username_format=%n } I also tried: override_fields = user=%n Or, alternatively to any override_fields: args = blocking=no username_format=%n Lastly, I even tried this whole different approach: userdb { driver = passwd-file args = blocking=no username_format=%n /etc/passwd } The following is logged no matter which of those I try: auth: imapc(remotehost.com:993): Connected to #.#.#.#:993 (local #.#.#.#:35088) auth: Error: passwd(user(a)domain,#.#.#.#,<...>): user not found from userdb imap(274049): Error: auth-master: login: request [...]: Login auth request failed: Authenticated user not found from userdb, auth lookup id=... (auth connected 10 msecs ago, request took 1 msecs, client-pid=274038 client-id=1) imap-login: Disconnected: Internal login failure (pid=274038 id=1): user=<user@domain>, method=PLAIN, rip=#.#.#.#, lip=#.#.#.#, mpid=274049, TLS, session=<...> (The last attempt got a slightly varied log message, but quite similar: auth: Error: passwd-file(user(a)domain#.#.#.#,<...>): user not found from userdb ) Any help with this would be much appreciated. Thanks! Luke

1 1

submission_host auth
by k v 18 Jan '23

18 Jan '23

When using dovecot container with sieve plugin there is no sendmail to use for sending email for sieve redirect action for example. We can use submission_host instead https://doc.dovecot.org/settings/core/#core_setting-submission_host but there is no way to specify credentials for auth in remote MTA. Submission_relay_* settings e.g. submission_relay_master_user relate to dovecot submission service. Using something like permit_mynetworks in remote MTA is not acceptable for security reasons. Is it possible to add authorization in the remote MTA using submission_host?

2 4

dovecot replication - new and cur folders on mx1 and mx2
by Tomaz Kavcic 17 Jan '23

17 Jan '23

Hello, I have a question in regards to specific dovecot replication behaviour and I'm just wondering if this is actually an expected/normal behaviour, or just a version issue. I'm using dovecot 2.3.16 which is packed by default with latest Ubuntu 22.04.1 LTS server release. I setup dovecot replication pair (mx1 - mx2) which is working ok. MX1 has priority 10, MX2 has priority 20. I use maildir (postfix + dovecot lmtp). The "strange" behaviour is this. When new mail arrives, it's by default delivered into "new" folder inside user directory. This email is then replicated to both servers (mx1 and mx2). When I login to mx1 via IMAP client (roundcube, outlook, etc.) that specific email is moved from "new" to "cur" folder on server mx1 and it's flagged also with "S", which probably means read flag. On server mx2, that email filename is also flagged with "S", but the email stays inside the "new" folder and it's not moved to "cur". If I want this email to be moved to "cur" on mx2 server, I have to login to that IMAP server as well, click on that email (which is already flagged as read), and after click, the email is also moved to "cur" on server mx2. Simply said, all new mails on mx1 server are moved to "cur" when accessed, but the stay in "new" folder on server mx2 until they're physically accessed there as well. Is this normal behaviour? I tried setup with TCP and SSH replication, and the situation is the same in all cases. Lastly, I tried TCPS (with SSL) as well, but that option has issues in 2.3.16, which is probably known already as I found multiple posts about these issues in this version. Thank you in advance for your answer and kind regards, Tomaz Kavcic. --- As suggested by mr. Timo Sarainen, this should be synced, so I'm posting doveconf -n as attachment for both servers as well.

3 3

Quota Clone Plugin
by MK 17 Jan '23

17 Jan '23

Hello Everybody, just a short question, maybe someone used this plugin and can give me a short answer. I have configured dovecot Quota Clone Plugin to copy quota information into a MySQL Database. Works finde, but dovecot writes the quota information only to the database if the quota is triggert by new email or something else like this. So I don't have all my user in the MySQL Database. Is there any command I can use that can trigger dovecot to write the quota information of all the users to the database one time, so that I have all my user quota information copied in the database? I have tried "doveadm quota recalc -A" but it took quite a long time and there is a very high I/O Load on the server, so I stopped it. I don't think this is what I want to do. Maybe there is another way. For all of you who don't know this plugin. https://doc.dovecot.org/configuration_manual/quota_clone_plugin/ Greetings, Oliver

3 3

hostname in director_mail_servers setting
by k v 16 Jan '23

16 Jan '23

director_mail_servers setting uses IP Adresses as values https://doc.dovecot.org/settings/core/#core_setting-director_mail_servers As described in the documentation, hostnames are converted to addresses when the configuration is read https://doc.dovecot.org/settings/types/#ip-addresses This is acceptable in host or virtual machine environment, but becomes a problem in a containerized environment. When any backend container is restarted, its address will change and users related to this backend can no longer connect, but backend (available with new IP address) can process request for example, director_mail_servers = dovecot-1 dovecot-2 dovecot-3 becomes in runtime (with one successfully connectetd user foo(a)example.com) doveadm director status mail server ip tag vhosts state state changed users 10.0.71.3 100 up - 0 10.0.71.4 100 up - 1 10.0.71.5 100 up - 0 where dovecot-1 10.0.71.3, dovecot-2 10.0.71.4, dovecot-3 10.0.71.5 When dovevecot-2 backend container restarts, and becomes new IP address, user can't connect to related backend: imap-login: Info: proxy(foo@example.com,10.0.71.4:143): Started proxying to <10.0.71.4> (<dovecot-2>) (0.009 secs) imap-login: Error: proxy(foo@example.com,10.0.71.4:143): connect(10.0.71.4, 143) failed: No route to host (after 28 secs, 3 reconnects, local=10.0.71.13:41066) My suggestion to developers: resolve backend hostnames to IP address on every request to director_mail_servers It's not about dovemon service with active health checks from dovecot pro, just dynamic hostname resolve

2 1

Error: Mailbox INBOX: file_dotlock_create in directory /var/mail failed: Permission denied
by manmatha 16 Jan '23

16 Jan '23

Hi All, We are running Round cube on top of dovecot and postfix as our email echo system. Main problem we are facing is that, roundcube loads really slow for big inboxes (~5000 emails). I found that, while it loads, imap process goes high in terms of CPU utilization(~90%). I asked it in some roundcube support group, They believe that it is related to dovecot not roundcube. Can you shed any light on this issue. Thanks in advance. Here is $dovecot -n$ output xyz@roundcube-internal-server-1:/etc/dovecot$ dovecot -n # 2.3.16 (7e2e900c1a): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.16 (09c29328) # OS: Linux 5.15.0-1024-kvm x86_64 Ubuntu 22.04.1 LTS # Hostname: roundcube-internal-server-1 auth_mechanisms = plain login disable_plaintext_auth = no listen = * lock_method = dotlock mail_always_cache_fields = date.received mail_cache_min_mail_count = 5000 mail_location = mbox:~/mail:INBOX=/var/mail/%u:INDEX=/var/indexes/isical.ac.in/%n mail_privileged_group = mail mail_sort_max_read_count = 100 mailbox_list_index_include_inbox = yes mailbox_list_index_very_dirty_syncs = yes maildir_very_dirty_syncs = yes mbox_dotlock_change_timeout = 1 mins mbox_lazy_writes = no mbox_write_locks = dotlock fcntl namespace inbox { inbox = yes location = prefix = } passdb { driver = pam } protocols = imap pop3 ssl_cert = </etc/dovecot/private/dovecot.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { driver = passwd } protocol !indexer-worker { mail_vsize_bg_after_count = 100 } On a different note we are getting error occasionally of the following type. I dont know if they are related. Jan 16 15:36:13 roundcube-internal-server-1 dovecot: imap(xyz)<361270><aqauu17ycsvAqL8/>: Error: unlink(/var/mail/xyz.lock) failed: Permission denied (in file-dotlock.c:195) Jan 16 15:36:13 roundcube-internal-server-1 dovecot: imap(xyz)<361270><aqauu17ycsvAqL8/>: Error: unlink(xyz.lock) failed: Permission denied (in file-dotlock.c:195) Jan 16 15:36:13 roundcube-internal-server-1 dovecot: imap(xyz)<361270><aqauu17ycsvAqL8/>: Error: Mailbox INBOX: file_dotlock_create(xyz) in directory /var/mail failed: Permission denied (euid=10865(xyz) egid=8(mail) missing +w perm: /var/mail, UNIX perms appear ok (ACL/MAC wrong?), dir owned by 0:8 mode=0771)

2 1