- dovecot - dovecot.org

Exim / Dovecot v2.4.0 authentication patch
by Timo Sirainen 04 Feb '25

04 Feb '25

Hi, Dovecot v2.4.0 changed authentication protocol slightly to allow new functionality (SCRAM TLS channel binding). It attempted to preserve backwards compatibility by checking client-provided VERSION first before sending data that the client wouldn't handle correctly. However, Exim's Dovecot authenticator doesn't send VERSION until Dovecot has sent the whole authentication handshake. This causes Exim to get stuck when trying to authenticate. I guess we'll provide some kind of a workaround for v2.4.1, but this should get fixed on Exim side as well. Attached a patch that I tested works (against 4.97-4ubuntu4).

2 1

OpenLDAP Pass-through Authentication
by Dino Edwards 04 Feb '25

04 Feb '25

Hi, Trying to get pass-through authentication working however, I'm running to the following error in OpenLDAP: openldap | 679ceede.3aa31e0a 0x7f2ff617e6c0 conn=1004 op=1 SRCH attr=uid mail displayName openldap | 679ceede.3aa4b816 0x7f2ff617e6c0 conn=1004 op=1 SEARCH RESULT tag=101 err=0 qtime=0.000019 etime=0.000172 nentries=1 text= openldap | 679ceede.3aaafd88 0x7f2ff597d6c0 conn=1005 fd=14 ACCEPT from IP=172.16.32.1:47082 (IP=0.0.0.0:1389) openldap | 679ceede.3aab8f34 0x7f2ff597d6c0 conn=1005 op=0 BIND dn="cn=username(a)domain.tld,ou=users,dc=domain,dc=local" method=128 openldap | 679ceede.3aac9267 0x7f2ff597d6c0 SASL [conn=1005] Failure: cannot connect to saslauthd server: No such file or directory openldap | 679ceede.3aad25b9 0x7f2ff597d6c0 conn=1005 op=0 RESULT tag=97 err=49 qtime=0.000005 etime=0.000120 text= openldap | 679ceede.3ab0191f 0x7f2ff617e6c0 conn=1005 fd=14 closed (connection lost) openldap | 679ceede.3ab3d76a 0x7f2ff597d6c0 conn=1004 fd=13 closed (connection lost) I followed the admin docs at: https://www.openldap.org/doc/admin26/guide.html#Pass-Through%20authenticatio n Here's what I did: Created /usr/lib/sasl2/slapd.conf with the following content: mech_list: plain pwcheck_method: saslauthd saslauthd_path: /var/run/sasl2/mux Created /etc/saslauthd.conf with the following content: ldap_servers: ldap://192.168.xxx.xxx.xxx ldap_search_base: OU=Users,DC=domain,DC=tld ldap_filter: (uid=%u) ldap_bind_dn: CN=saslauthd,CN=Users,DC=domain,DC=tld ldap_password: somepassword Added a user that already exists in domain.tld in openldap with the following password: {SASL}username(a)domain.tld I would appreciate some help on this. Thanks

1 1

IMAP dovecot\postgres low authentication performance
by Anatoliy Zhestov 04 Feb '25

04 Feb '25

Hi. We have a performance problem with imap authentication through postgresql. Our servers(modoboa based) have a big amount of permanent imap connections(5000-50000). Current performance is about 3000 successful authentications per hour. No visible reasons for such low speed. Accordingly, after a network failure or server restart, all clients try to reconnect, but restoring the connection pool takes hours and even tens of hours. Judging by the logs after the restart, a huge number of auth requests closed by timeout after 70-90 seconds. The postgresql database is not overloaded at the restore connections process and the postgresql connection pool (100) does not overflow. Manually started sql auth queries work fast, tables have indexes. So I guess there is a bottleneck somewhere in dovecot auth service or postgresql driver. I couldn't find any settings in the documentation that are directly responsible for regulating the number of connections to PostgreSQL from the auth service or performance of the driver. Is there any way to manage this? Does it make sense to use pgbouncer in front of the base? What else can be a bottleneck in our configuration and how to regulate the number of possible simultaneous authentications via PostgreSQL? I would be happy for any advice on how to increase the performance to at least 100 authentications per second. Thanks in advance for your help. Anatoliy Zhestov. ---------------- configs of server with 48 core, 184 gb mem --------------- dovecot 2.3.16 # 2.3.16 (7e2e900c1a): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.16 (09c29328) # OS: Linux 5.15.0-126-generic x86_64 Ubuntu 22.04.5 LTS # Hostname: imap.ourcompany.net auth_cache_negative_ttl = 0 auth_cache_size = 20 M auth_cache_ttl = 3 hours auth_master_user_separator = * auth_mechanisms = plain login default_client_limit = 131072 default_process_limit = 131072 dict { quota = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext } mail_location = maildir:~/Maildir mail_plugins = quota managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mai lbox date index ihave duplicate mime foreverypart extracttext editheader imapflags notify vnd.dovecot.pipe vnd.dovecot.execute namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } passdb { args = /etc/dovecot/dovecot-sql-master.conf.ext driver = sql master = yes pass = yes } plugin { quota = dict:User quota::proxy::quota sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_execute_bin_dir = /usr/lib/dovecot/sieve-execute sieve_execute_socket_dir = . sieve_extensions = +notify +imapflags +editheader +body +vnd.dovecot.execute +vnd.dovecot.pipe sieve_pipe_bin_dir = /usr/lib/dovecot/sieve-execute sieve_pipe_socket_dir = . sieve_plugins = sieve_extprograms } protocols = " imap lmtp sieve" service anvil { client_limit = 393219 } service auth { client_limit = 524288 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-radicale { group = radicale mode = 0666 user = radicale } unix_listener auth-userdb { user = vmail } } service config { vsz_limit = 1 G } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap { executable = imap postlogin process_limit = 131072 vsz_limit = 1 G } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } vsz_limit = 512 M } service pop3 { executable = pop3 postlogin } service postlogin { executable = script-login /usr/local/bin/postlogin.sh service_count = 1000 process_min_avail = 5 user = modoboa } service send-to-click-frontapp1 { executable = script /usr/lib/dovecot/sieve-execute/clickhouse_log_frontapp1.sh unix_listener send-to-click-frontapp1 { group = vmail mode = 0666 user = vmail } user = dovenull vsz_limit = 2 G } service send-to-click { executable = script /usr/lib/dovecot/sieve-execute/clickhouse_log.sh unix_listener send-to-click { group = vmail mode = 0666 user = vmail } user = dovenull vsz_limit = 2 G } service stats { unix_listener stats-reader { group = vmail mode = 0660 user = vmail } unix_listener stats-writer { group = vmail mode = 0660 user = vmail } vsz_limit = 2 G } ssl_cert = </etc/letsencrypt/live/imap.ourcompany.net/fullchain.pem ssl_cipher_list = EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS:!RC4 ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1 stats_writer_socket_path = userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = quota sieve postmaster_address = postmaster(a)imap.ourcompany.net } protocol imap { mail_max_userip_connections = 50 mail_plugins = quota imap_quota } /etc/dovecot/dovecot-sql-master.conf.ex driver = pgsql connect = host=127.0.0.1 port=5432 dbname=modoboa user=modoboa password=XXXXXXXXXXXXXXXXX password_query = SELECT email AS user, password FROM core_user WHERE email='%u' and is_active and master_user /etc/dovecot/dovecot-sql.conf.ext driver = pgsql connect = host=127.0.0.1 port=5432 dbname=modoboa user=modoboa password=XXXXXXXXXXXXXXXXX user_query = SELECT '/srv/vmail/%d/%n' AS home, 1003 as uid, 1003 as gid, '*:bytes=' || mb.quota || 'M' AS quota_rule FROM admin_mailbox mb INNER JOIN admin_domain dom ON mb.domain_id=dom.id INNER JOIN core_user u ON u.id=mb.user_id WHERE (mb.is_send_only IS NOT TRUE OR '%s' NOT IN ('imap', 'pop3', 'lmtp')) AND mb.address='%n' AND dom.name='%d' password_query = SELECT email AS user, password, '/srv/vmail/%d/%n' AS userdb_home, 1003 AS userdb_uid, 1003 AS userdb_gid, CONCAT('*:bytes=', mb.quota, 'M') AS userdb_quota_rule FROM core_user u INNER JOIN admin_mailbox mb ON u.id=mb.user_id INNER JOIN admin_domain dom ON mb.domain_id=dom.id WHERE (mb.is_send_only IS NOT TRUE OR '%s' NOT IN ('imap', 'pop3')) AND email='%u' AND is_active AND dom.enabled iterate_query = SELECT email AS user FROM core_user /usr/local/bin/postlogin.sh #!/bin/sh PATH="/usr/bin:/usr/local/bin:/bin" psql -c "UPDATE core_user SET last_login=now() WHERE username='$USER'" > /dev/null exec "$@" /usr/lib/dovecot/sieve-execute/clickhouse_log_frontapp1.sh #!/bin/bash cd /tmp logger -i -t SIEVE_TEST "$1,$2,$3,$4" clickhouse-client -h XXXXXXXXXXXXXXXXX -d default -u some_user --password XXXXXXXXXXXXXXXXX \ --param_origin_from="$1" \ --param_origin_to="$2" \ --param_message_id="$3" \ --param_forward="$4" \ -q "INSERT INTO frontapp1_redirects (origin_from_email,origin_to_email,message_id,forward_email) VALUES ({origin_from:String},{origin_to:String},{message_id:String},{forward:String})" /usr/lib/dovecot/sieve-execute/clickhouse_log.sh #!/bin/bash cd /tmp logger -i -t SIEVE_TEST "$1,$2,$3,$4" clickhouse-client -h XXXXXXXXXXXXXXXXX -d default -u some_user --password XXXXXXXXXXXXXXXXX \ --param_origin_from="$1" \ --param_origin_to="$2" \ --param_message_id="$3" \ --param_forward="$4" \ -q "INSERT INTO frontapp_redirects (origin_from_email,origin_to_email,message_id,forward_email) VALUES ({origin_from:String},{origin_to:String},{message_id:String},{forward:String})"

3 12

Problem email client iPhone ios18.2
by andrzej.milewski＠gmail.com 04 Feb '25

04 Feb '25

Hello, I have a fairly new installation of the Dovecot server version 2.3.19.1 on Debian 12.8. In general, the server works correctly. It sends and receives emails without issues. The clients primarily use Thunderbird on desktops and Gmail on Android. However, some clients are using iPhones with various hardware versions. Those with iOS below version 18 work correctly without any issues. Those who have updated their phones to version 18 and now even 18.2 are experiencing problems with mailbox synchronization. On version 18.2, the issue appears as follows: When you open the app, you can read several messages. They load correctly, but, for instance, attachments do not. If I wait for some time, the attachment eventually loads. During this time, I observe the server logs, and it shows that a certain session on the Dovecot server has ended. Dec 17 13:13:01 iredmail0 dovecot[408]: imap(xxx(a)xxx.com)<44785><1QGZCHcp/+3AqM2O>: Disconnected: Connection closed (UID SEARCH finished 92.295 secs ago) in=2972 out=6076956 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=36 body_bytes=6059064 Dec 18 08:40:41 iredmail0 dovecot[408]: imap(xxx(a)xxx.com)<177412>: Disconnected: Connection closed (UID FETCH finished 90.079 secs ago) in=5126 out=1304237 deleted=0 expunged=0 trashed=0 hdr_count=94 hdr_bytes=289688 body_count=89 body_bytes=969615 Dec 18 08:42:19 iredmail0 dovecot[408]: imap(xxx(a)xxx.com)<177410><D5mqU4cpZ+nAqM2O>: Disconnected: Connection closed (UID FETCH finished 95.051 secs ago) in=4475 out=1162785 deleted=0 expunged=0 trashed=0 hdr_count=95 hdr_bytes=281674 body_count=61 body_bytes=839545 Dec 18 08:44:17 iredmail0 dovecot[408]: imap(edyta.kowalewska(a)laktopol.com.pl)<177918>: Disconnected: Connection closed (UID FETCH finished 92.132 secs ago) in=29399 out=30266644 deleted=0 expunged=0 trashed=0 hdr_count=565 hdr_bytes=1726228 body_count=628 body_bytes=28109455 After the session is closed, the attachment loads immediately. From the logs and router analysis, it appears that the download happens in another previously opened session. After this incident, I go to another message with an attachment, and the situation repeats. There are also situations where the email body does not load, and the app waits for the server until the session closes and the content is fetched in another session. It does not matter whether I connect to the server via the internet or through a local network. On an iPhone 7 with iOS 15.8.3, everything works smoothly. Attachments open without issues. On iOS 18.1.1, attachments do not load, and the session does not close on its own. It takes forever unless the app is closed. In the Gmail app, everything works correctly without any issues. This is clearly a problem with the mail client on iPhone. However, is there anything I can adjust in my server settings to mitigate this issue for users? My dovecot configuration https://pastebin.com/UPNQ5dZz

20 43

Re: [OFFLIST] Re: IMAP dovecot\postgres low authentication performance
by Anatoliy Zhestov 03 Feb '25

03 Feb '25

> > Long ago in MagicMail we recognized that there was a problem with > Postgres direct back addressing a large number of AUTH attempts at once, > and we went down the road to using a high performace DBFILE mechanisms, > with real time updates from the Postgres database.. allowed us to scale > greatly.. Thank you, this looks interesting but I don't really understand how to organize it correctly. Do you have a link to some manual that could serve as an example? On Mon, Feb 3, 2025 at 8:39 PM Michael Peddemors <michael(a)linuxmagic.com> wrote: > Long ago in MagicMail we recognized that there was a problem with > Postgres direct back addressing a large number of AUTH attempts at once, > and we went down the road to using a high performace DBFILE mechanisms, > with real time updates from the Postgres database.. allowed us to scale > greatly.. > > Don't know much about you, your product or use case, just thought it > might help.. > > On 2025-02-02 21:05, Anatoliy Zhestov via dovecot wrote: > > Hi. We have a performance problem with imap authentication through > > postgresql. > > Our servers(modoboa based) have a big amount of permanent imap > > connections(5000-50000). > > Current performance is about 3000 successful authentications per hour. No > > visible reasons for such low speed. Accordingly, after a network failure > or > > server restart, all clients try to reconnect, but restoring the > connection > > pool takes hours and even tens of hours. Judging by the logs after the > > restart, a huge number of auth requests closed by timeout after 70-90 > > seconds. The postgresql database is not overloaded at the restore > > connections process and the postgresql connection pool (100) does not > > overflow. Manually started sql auth queries work fast, tables have > indexes. > > So I guess there is a bottleneck somewhere in dovecot auth service or > > postgresql driver. > > > > I couldn't find any settings in the documentation that are directly > > responsible for regulating the number of connections to PostgreSQL from > the > > auth service or performance of the driver. Is there any way to manage > this? > > Does it make sense to use pgbouncer in front of the base? What else can > be > > a bottleneck in our configuration and how to regulate the number of > > possible simultaneous authentications via PostgreSQL? I would be happy > for > > any advice on how to increase the performance to at least 100 > > authentications per second. > > > > > > Thanks in advance for your help. > > Anatoliy Zhestov. > > > > ---------------- configs of server with 48 core, 184 gb mem > --------------- > > > > dovecot 2.3.16 > > # 2.3.16 (7e2e900c1a): /etc/dovecot/dovecot.conf > > # Pigeonhole version 0.5.16 (09c29328) > > # OS: Linux 5.15.0-126-generic x86_64 Ubuntu 22.04.5 LTS > > # Hostname: imap.ourcompany.net > > auth_cache_negative_ttl = 0 > > auth_cache_size = 20 M > > auth_cache_ttl = 3 hours > > auth_master_user_separator = * > > auth_mechanisms = plain login > > default_client_limit = 131072 > > default_process_limit = 131072 > > dict { > > quota = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext > > } > > mail_location = maildir:~/Maildir > > mail_plugins = quota > > managesieve_notify_capability = mailto > > managesieve_sieve_capability = fileinto reject envelope encoded-character > > vacation subaddress comparator-i;ascii-numeric relational regex > imap4flags > > copy include variables body enotify environment mai > > lbox date index ihave duplicate mime foreverypart extracttext editheader > > imapflags notify vnd.dovecot.pipe vnd.dovecot.execute > > namespace inbox { > > inbox = yes > > location = > > mailbox Drafts { > > auto = subscribe > > special_use = \Drafts > > } > > mailbox Junk { > > auto = subscribe > > special_use = \Junk > > } > > mailbox Sent { > > auto = subscribe > > special_use = \Sent > > } > > mailbox Trash { > > auto = subscribe > > special_use = \Trash > > } > > prefix = > > } > > passdb { > > args = /etc/dovecot/dovecot-sql.conf.ext > > driver = sql > > } > > passdb { > > args = /etc/dovecot/dovecot-sql-master.conf.ext > > driver = sql > > master = yes > > pass = yes > > } > > plugin { > > quota = dict:User quota::proxy::quota > > sieve = ~/.dovecot.sieve > > sieve_dir = ~/sieve > > sieve_execute_bin_dir = /usr/lib/dovecot/sieve-execute > > sieve_execute_socket_dir = . > > sieve_extensions = +notify +imapflags +editheader +body > > +vnd.dovecot.execute +vnd.dovecot.pipe > > sieve_pipe_bin_dir = /usr/lib/dovecot/sieve-execute > > sieve_pipe_socket_dir = . > > sieve_plugins = sieve_extprograms > > } > > protocols = " imap lmtp sieve" > > service anvil { > > client_limit = 393219 > > } > > service auth { > > client_limit = 524288 > > unix_listener /var/spool/postfix/private/auth { > > group = postfix > > mode = 0666 > > user = postfix > > } > > unix_listener auth-radicale { > > group = radicale > > mode = 0666 > > user = radicale > > } > > unix_listener auth-userdb { > > user = vmail > > } > > } > > service config { > > vsz_limit = 1 G > > } > > service dict { > > unix_listener dict { > > mode = 0600 > > user = vmail > > } > > } > > service imap { > > executable = imap postlogin > > process_limit = 131072 > > vsz_limit = 1 G > > } > > service lmtp { > > unix_listener /var/spool/postfix/private/dovecot-lmtp { > > group = postfix > > mode = 0600 > > user = postfix > > } > > vsz_limit = 512 M > > } > > service pop3 { > > executable = pop3 postlogin > > } > > service postlogin { > > executable = script-login /usr/local/bin/postlogin.sh > > service_count = 1000 > > process_min_avail = 5 > > user = modoboa > > } > > > > service send-to-click-frontapp1 { > > executable = script > > /usr/lib/dovecot/sieve-execute/clickhouse_log_frontapp1.sh > > unix_listener send-to-click-frontapp1 { > > group = vmail > > mode = 0666 > > user = vmail > > } > > user = dovenull > > vsz_limit = 2 G > > } > > service send-to-click { > > executable = script /usr/lib/dovecot/sieve-execute/clickhouse_log.sh > > unix_listener send-to-click { > > group = vmail > > mode = 0666 > > user = vmail > > } > > user = dovenull > > vsz_limit = 2 G > > } > > service stats { > > unix_listener stats-reader { > > group = vmail > > mode = 0660 > > user = vmail > > } > > unix_listener stats-writer { > > group = vmail > > mode = 0660 > > user = vmail > > } > > vsz_limit = 2 G > > } > > ssl_cert = </etc/letsencrypt/live/imap.ourcompany.net/fullchain.pem > > ssl_cipher_list = > > > EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS:!RC4 > > ssl_client_ca_dir = /etc/ssl/certs > > ssl_dh = # hidden, use -P to show it > > ssl_key = # hidden, use -P to show it > > ssl_min_protocol = TLSv1 > > stats_writer_socket_path = > > userdb { > > driver = prefetch > > } > > userdb { > > args = /etc/dovecot/dovecot-sql.conf.ext > > driver = sql > > } > > protocol lmtp { > > mail_plugins = quota sieve > > postmaster_address = postmaster(a)imap.ourcompany.net > > } > > protocol imap { > > mail_max_userip_connections = 50 > > mail_plugins = quota imap_quota > > } > > > > > > > > > > /etc/dovecot/dovecot-sql-master.conf.ex > > driver = pgsql > > connect = host=127.0.0.1 port=5432 dbname=modoboa user=modoboa > > password=XXXXXXXXXXXXXXXXX > > password_query = SELECT email AS user, password FROM core_user WHERE > > email='%u' and is_active and master_user > > > > > > /etc/dovecot/dovecot-sql.conf.ext > > driver = pgsql > > connect = host=127.0.0.1 port=5432 dbname=modoboa user=modoboa > > password=XXXXXXXXXXXXXXXXX > > user_query = SELECT '/srv/vmail/%d/%n' AS home, 1003 as uid, 1003 as gid, > > '*:bytes=' || mb.quota || 'M' AS quota_rule FROM admin_mailbox mb INNER > > JOIN admin_domain dom ON mb.domain_id=dom.id INNER JOIN core_user u ON > > u.id=mb.user_id > > WHERE (mb.is_send_only IS NOT TRUE OR '%s' NOT IN ('imap', 'pop3', > 'lmtp')) > > AND mb.address='%n' AND dom.name='%d' > > password_query = SELECT email AS user, password, '/srv/vmail/%d/%n' AS > > userdb_home, 1003 AS userdb_uid, 1003 AS userdb_gid, CONCAT('*:bytes=', > > mb.quota, 'M') AS userdb_quota_rule FROM core_user u INNER JOIN > > admin_mailbox mb ON u.id=mb.user_id INNER JOIN admin_domain dom ON > > mb.domain_id=dom.id WHERE (mb.is_send_only IS NOT TRUE OR '%s' NOT IN > > ('imap', 'pop3')) AND email='%u' AND is_active AND dom.enabled > > iterate_query = SELECT email AS user FROM core_user > > > > > > > > /usr/local/bin/postlogin.sh > > #!/bin/sh > > PATH="/usr/bin:/usr/local/bin:/bin" > > psql -c "UPDATE core_user SET last_login=now() WHERE username='$USER'" > > > /dev/null > > exec "$@" > > > > > > /usr/lib/dovecot/sieve-execute/clickhouse_log_frontapp1.sh > > #!/bin/bash > > cd /tmp > > logger -i -t SIEVE_TEST "$1,$2,$3,$4" > > clickhouse-client -h XXXXXXXXXXXXXXXXX -d default -u some_user --password > > XXXXXXXXXXXXXXXXX \ > > --param_origin_from="$1" \ > > --param_origin_to="$2" \ > > --param_message_id="$3" \ > > --param_forward="$4" \ > > -q "INSERT INTO frontapp1_redirects > > (origin_from_email,origin_to_email,message_id,forward_email) VALUES > > > ({origin_from:String},{origin_to:String},{message_id:String},{forward:String})" > > > > /usr/lib/dovecot/sieve-execute/clickhouse_log.sh > > #!/bin/bash > > cd /tmp > > logger -i -t SIEVE_TEST "$1,$2,$3,$4" > > clickhouse-client -h XXXXXXXXXXXXXXXXX -d default -u some_user --password > > XXXXXXXXXXXXXXXXX \ > > --param_origin_from="$1" \ > > --param_origin_to="$2" \ > > --param_message_id="$3" \ > > --param_forward="$4" \ > > -q "INSERT INTO frontapp_redirects > > (origin_from_email,origin_to_email,message_id,forward_email) VALUES > > > ({origin_from:String},{origin_to:String},{message_id:String},{forward:String})" > > _______________________________________________ > > dovecot mailing list -- dovecot(a)dovecot.org > > To unsubscribe send an email to dovecot-leave(a)dovecot.org > > > -- > "Catch the Magic of Linux..." > ------------------------------------------------------------------------ > Michael Peddemors, President/CEO LinuxMagic Inc. > Visit us at http://www.linuxmagic.com @linuxmagic > A Wizard IT Company - For More Info http://www.wizard.ca > "LinuxMagic" a Reg. TradeMark of Wizard Tower TechnoServices Ltd. > ------------------------------------------------------------------------ > 604-682-0300 Beautiful British Columbia, Canada > >

2 1

Dovecot autoexpunge lock errors - Couldn't create dovecot.autoexpunge.lock
by mihai.aurel88＠gmail.com 03 Feb '25

03 Feb '25

Hello, We are seeing lots of errors like this in the logs: "dovecot: imap(user@domain)<31219><Dly3bTyt4tesENjH>: Error: autoexpunge: Couldn't create dovecot.autoexpunge.lock lock: file_create_locked(/home/virtual/domain/user/dovecot.autoexpunge.lock) failed: fstat(/home/virtual/domain/user/dovecot.autoexpunge.lock) failed: Stale file handle" We don't understand why Dovecot is trying to create the dovecot.autoexpunge.lock file in the mail home directory instead of mail location directory. For some accounts, the dovecot.autoexpunge.lock file is correctly created inside the mail home directory. Any ideas on what could be causing this behavior? # doveadm user user@domain field value uid 150 gid 8 home /home/virtual/domain/user mail maildir:~/Maildir quota_rule *:bytes=8192000000 # doveconf -n # 2.3.21.1 # Pigeonhole version 0.5.21.1 # OS: Linux 6.1.0-29-amd64 x86_64 Debian 12.9 auth_cache_verify_password_with_worker = yes auth_mechanisms = plain login auth_verbose = yes auth_worker_max_count = 256 default_client_limit = 10000 default_process_limit = 10000 default_vsz_limit = 1 G dict { lastlogin = mysql:/etc/dovecot/mysql/dovecot-dict-lastlogin.conf quotadict = mysql:/etc/dovecot/mysql/dovecot-dict-quota.conf } disable_plaintext_auth = no first_valid_uid = 100 imap_hibernate_timeout = 30 secs log_timestamp = "%Y-%m-%d %H:%M:%S " login_log_format_elements = user=<%u> method=%m rip=%r lip=%l pid=%p %c login_trusted_networks = mail_fsync = always mail_gid = 8 mail_home = /home/virtual/%d/%n mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = zlib quota mail_log notify mail_privileged_group = mail mail_uid = 150 maildir_stat_dirs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave metric imap_command { filter = event=imap_command_finished AND tagged_reply_state=OK group_by = cmd_name } metric imap_select_no { filter = event=imap_command_finished AND cmd_name=SELECT AND tagged_reply_state=NO } metric imap_select_no_notfound { filter = event=imap_command_finished AND cmd_name=SELECT AND tagged_reply="NO*Mailbox doesn't exist:*" } metric storage_http_gets { filter = event=http_request_finished AND category=storage AND method=get } mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { auto = subscribe autoexpunge = 30 days special_use = \Junk } mailbox Trash { auto = subscribe autoexpunge = 60 days special_use = \Trash } mailbox virtual/All { special_use = \All } prefix = } passdb { args = /etc/dovecot/mysql/dovecot-sql.conf driver = sql } plugin { last_login_dict = proxy::lastlogin last_login_key = # hidden, use -P to show it mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = dict:user::proxy::quotadict quota_rule2 = Trash:ignore quota_rule3 = Spam:ignore quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 The email account that you tried to reach is over quota quota_status_success = DUNNO quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_before = /etc/dovecot/sieve/default.sieve sieve_dir = ~/sieve sieve_vacation_send_from_recipient = yes trash = /etc/dovecot/dovecot-trash.conf zlib_save = gz zlib_save_level = 6 } protocols = imap sieve pop3 lmtp service anvil { client_limit = 50000 unix_listener anvil-auth-penalty { mode = 00 } } service auth { client_limit = 50000 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = mail mode = 0660 user = vmail } user = root vsz_limit = 1 G } service dict { unix_listener dict { mode = 0660 user = vmail } } service imap-hibernate { unix_listener imap-hibernate { group = $default_internal_group mode = 0660 user = vmail } } service imap-login { inet_listener imap { address = * port = 143 } inet_listener imaps { address = * port = 993 } process_min_avail = 1 service_count = 0 } service imap { extra_groups = $default_internal_group process_min_avail = 16 unix_listener imap-master { user = $default_internal_user } vsz_limit = 2 G } service lmtp { process_min_avail = 16 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } vsz_limit = 1 G } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 16 service_count = 1 } service managesieve { vsz_limit = 1 G } service pop3-login { inet_listener pop3 { address = * port = 110 } inet_listener pop3s { address = * port = 995 } process_min_avail = 16 service_count = 1 } service pop3 { process_min_avail = 16 vsz_limit = 1 G } service quota-status { executable = /usr/lib/dovecot/quota-status -p postfix process_min_avail = 16 unix_listener /var/spool/postfix/private/quota-status { group = postfix mode = 0666 user = postfix } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { mode = 0660 user = vmail } user = vmail } service stats { unix_listener stats-reader { mode = 0660 user = vmail } unix_listener stats-writer { mode = 0666 user = vmail } } ssl_cert = </etc/dovecot/ssl/cert.pem ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1 userdb { args = /etc/dovecot/mysql/dovecot-sql.conf driver = sql } verbose_proctitle = yes protocol imap { imap_max_line_length = 64 k mail_max_userip_connections = 16 mail_plugins = zlib quota mail_log notify trash imap_quota } protocol pop3 { mail_max_userip_connections = 10 mail_plugins = zlib quota mail_log notify pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_fast_size_lookups = yes pop3_no_flag_updates = yes pop3_uidl_format = %08Xu%08Xv } protocol sieve { mail_max_userip_connections = 10 managesieve_max_compile_errors = 5 } protocol lda { auth_socket_path = /var/run/dovecot/auth-master mail_plugins = zlib quota mail_log notify sieve sendmail_path = /usr/lib/sendmail syslog_facility = mail } protocol lmtp { mail_plugins = zlib quota mail_log notify sieve syslog_facility = mail }

1 0

Clarification Question about migrating_mailboxes and broken doc link
by Aleks 02 Feb '25

02 Feb '25

Hi. I have read the https://doc.dovecot.org/2.3/admin_manual/migrating_mailboxes/#id7 and there is one part which is not 100% clear to me. We have 2 Dovecot servers mail1 (2.2.36.4) and mail2 ( 2.3.21 ). Now I have understood that I need to add this to mail1 ``` service doveadm { inet_listener { port = 12354 } } doveadm_password = supersecret ``` and this on mail2 ``` doveadm_password = supersecret ``` The question is for me now: I think that this command `doveadm backup -Ru username tcp:host:port` should be executed on the mail2. Is this assumption right? What I also found is that on that Page https://doc.dovecot.org/2.3/admin_manual/doveadm_mailbox_commands/ the wiki is linked for the manpages but the wiki link shows this. ``` The old wiki has been closed Our primary documentation is now at https://doc.dovecot.org, please update your links. ``` What's the right link for the manpages in the web? Best regards Aleks

2 2

[Off-topic] ANN: Nauthilus
by Christian Rößner 01 Feb '25

01 Feb '25

Today I am pleased to announce the Nauthilus software. Nauthilus (N-Auth-ilus) is a centralized authentication server with a comprehensive policy engine. The idea behind this project is to connect services centrally in one place and perform extensive tests during authentication. # What can Nauthilus do and what problems does it solve? Many services on the Internet require authentication. Each application must then be connected to databases and must implement its own protective measures to ward off attackers. Every installation carries the risk of compromisation and therefore also access to the databases. With Nauthilus, the task is centralized in one place. Nauthilus is essentially an HTTP REST server that can be accessed by any application. It takes on the role of a guardian. Nauthilus integrates very well with Dovecot and Postfix. # Authentication process Nauthilus uses several authentication steps: ## 1. features Features include tests such as TLS verification, relay domains (is the system even responsible for the requested domain?), blocklists, RBLs and freely definable Lua features. Nauthilus has a powerful brute force concept with buckets to detect even slow attacks over days and weeks. It offers a bucket system for this purpose. ## 2. backend authentication Nauthilus includes LDAP support and Lua to perform the authentication itself. A large library of predefined functions is available in Lua, including SQL support. ## 3 Policies Policies are run through after authentication. Despite a successful login, the system can reject the login (or, conversely, allow it!). There is also space here for GeoIP lookups, etc. ## 4. Post-processing After the 3rd point, authentication has been completed, but at this point further tests can run in the background such as: * Check password policy and take action * Consult the Haveibeenpwnd network * GeoIP tracking across national borders and much more. ## Miscellaneous Nauthilus allows the free definition of so-called hooks. Each hook listens for a specific URI (callback) in the HTTP request. These callbacks are written by the administrator in Lua. In an initial proof of concept, Nauthilus can take on the role of a Dovecot director. This has already been tested with version 2.4.0. Currently, Nauthilus can dynamically delegate incoming connections to backends. The hooks concept is used here as an example. # Final words By integrating a Lua VM into the server, Nauthilus can be integrated and customized in almost any setup. See also the other Nauthilus-related projects listed in the appendix. To enable single sign-on (SSO), it can be operated with an Ory-Hydra server or the sister project nauthilus-keycloak can be used as a custom authenticator in Keycloak. 100% Open-Source 100% Community # Project https://github.com/croessner/nauthilus # Sub projects https://github.com/croessner/nauthilus-demo https://github.com/croessner/nauthilus-keycloak https://github.com/croessner/pfxhttp https://github.com/croessner/geoip-policyd # Mailing lists: https://lists.nauthilus.org N.B.: In the future, announcements are sent over the nauthilus-announce ML. This is just a hello world! Christian Rößner -- Rößner-Network-Solutions Zertifizierter ITSiBe / CISO Marburger Str. 70a, 36304 Alsfeld Fax: +49 6631 78823409, Mobil: +49 171 9905345 USt-IdNr.: DE225643613, https://roessner.website PGP fingerprint: 658D 1342 B762 F484 2DDF 1E88 38A5 4346 D727 94E5

2 2

Replication ade
by Hanns Mattes 01 Feb '25

01 Feb '25

Hei, mit 2.4 verschwindet ja, wenn ich das richtig verstanden habe, die Replication aus dem Werkzeugkasten von Dovecot. Obwohl ich nur ein recht kleines System mit etwa 500 Nutzern betreibe fand ich die Replication als ziemlich beruhigend :-) Meine Frage in die Runde: Wie wollt Ihr die Replikation ersetzen? Irgendwas mit NFS oder gibt es da eine empfohlene Herangehensweise? Fragende Grüße Hanns -- Sig for hire.

4 4

Re: 2.4.0 on MacOS
by John Muccigrosso 01 Feb '25

01 Feb '25

(Looks like I forgot to reply to the list last time.) With `sudo dovecot -F`, I'm now getting: Fatal: setrlimit(RLIMIT_DATA, 2147483648): Invalid argument The old(?) solution doesn't work for me: default_vsz_limit = 0 It reports: doveconf: Fatal: Error in configuration file /opt/homebrew/etc/dovecot/dovecot.conf: service(log): vsz_limit is too low > On Feb 1, 2025, at 08:55, John Muccigrosso <muccigrosso(a)icloud.com> wrote: > > On Feb 1, 2025, at 07:31, Aki Tuomi via dovecot <dovecot(a)dovecot.org> wrote: > >> You sure the default internal group isnt _dovecot as well? > > I'm not sure what you're asking. Do I not have the internal group set via this line?: > > default_internal_group = mail > > I think I may have to dig into group memberships, if this is the issue. I'm getting conflicting result when checking. That is, when I check the group "mail" for members, _dovecot isn't in it, even though checking which groups _dovecot belongs to includes "mail". The opposite happens for the "dovecot" group. And dovecot itself can't find the _dovecot or dovecot group. Seems to be a macOS thing with group membership. > >> You are missing >> >> mail_home = /Users/%{user} > > Thanks. As I said, I was trying to follow the documentation where they give the two lines under Maildir, but not this third one (even though it's given in the first part of this section).

3 4