April 2022 - dovecot - dovecot.org

What is the state of SMTPUTF8 support?
by Sylvhem 02 Jun '22

02 Jun '22

Hi! It’s been some time since I last saw a mention of SMTPUTF8 here. Last time I heard about it, it was worked on. Does anybody have news about that? -- Sylvhem

5 8

Message attachments, relocated with Tbird in Dovecot maildir store, not openable; reversible by moving BACK to inbox?
by PGNet Dev 25 May '22

25 May '22

I run dovecot --version 2.3.17.1 (476cd46418) on lsb_release -rd Description: Fedora release 35 (Thirty Five) Release: 35 mail store is maildir mail_location = maildir:/data/vmail/%d/%n/Maildir:CONTROL=/data/vmail/%d/%n/_control:INDEX=/var/vmail-index/%d/%n:LAYOUT=fs:UTF-8 front-end client is thunderbird --version Thunderbird 91.7.0 also on lsb_release -rd Description: Fedora release 35 (Thirty Five) Release: 35 mail is delivered to the store via lmtp transport from a Postfix agent mail, with attachments (e.g., .PDF) is received OK it's readable, attachments can be opened and/or saved to disk\ if, in TBird client, i move the message to any other folder in the IMAP hierarchy, the message relocates, as expected. the message itself is still readable, and _appears_ to have the attachment. but, attempting to OPEN the *attachment*, with e.g. Okular, from within the relocated message pops up a dialog, "Could not open file:///tmp/pid-28435/manual_13542AE.pdf" and saving it to Desktop, ls -al manual_13542AE.pdf -rw-r--r-- 1 pgnd pgnd 27 Apr 1 06:01 manual_13542AE.pdf file manual_13542AE.pdf manual_13542AE.pdf: data then opening it similarly returns "Could not open file:///home/pgnd/Desktop/manual_13542AE.pdf." Moving the message, via TBird, **BACK** to the Inbox, cures the problem -- -- I can immediately, again, open & save the attachments. This 'feels' like cache management misconfiguration to me. Not sure whether in Tbird &/or Dovecot, or what to look for specificially. Hints as to cause/fix of this attachment issue? Or event what/where, specifically, to log?

4 12

Dovecot 2.3.18 and OpenMetrics-Prometheus problem
by Christian Rößner 23 May '22

23 May '22

Hi, I have defined several metrics in Dovecot and activated the metrics service. After the I tried to include this in Prometheues, but I get strange errors. I gave up now after several hours in the hope that this might be a bug. So I ask for help here: Distribution: Rocky 8 Official Dovecot Repo Version: 2.3.18 Config for metrics, service and events: ----------------------------------------------------------------- metric auth_success { exporter = log filter = event=auth_request_finished AND success=yes } metric auth_failures { exporter = log filter = event=auth_request_finished AND NOT success=yes } metric auth_client_userdb { filter = event=auth_client_userdb_lookup_finished } metric auth_master { exporter = log filter = event=auth_master_client_login_finished } metric client_connected { filter = event=client_connection_connected } metric client_disconnected { filter = event=client_connection_disconnected group_by = reason } metric imap_command { filter = event=imap_command_finished group_by = cmd_name tagged_reply_state } metric lmtp_command { filter = event=smtp_server_command_finished AND protocol=lmtp group_by = cmd_name status_code duration:exponential:1:5:10 } metric imap_hibernate { filter = event=imap_client_hibernated group_by = error } metric imap_unhibernate { filter = event=imap_client_hibernated group_by = reason error } metric mail_delivery { filter = event=mail_delivery_finished group_by = duration:exponential:1:5:10 } metric sql_query { filter = event=sql_query_finished group_by = error_code } metric sieve_actions { filter = event=sieve_action_finished group_by = action_name error } metric managesieve { filter = event=managesieve_command_finished group_by = script_name } metric dict_lookups { filter = event=dict_lookup_finished group_by = driver error } service stats { inet_listener http { port = 9900 } } event_exporter log { format = json format_args = time-rfc3339 transport = log } ----------------------------------------------------------------- Shortened curl output: ----------------------------------------------------------------- # HELP process_start_time_seconds Timestamp of service start # TYPE process_start_time_seconds gauge process_start_time_seconds 1645743604 # HELP dovecot_build Dovecot build information # TYPE dovecot_build info dovecot_build_info{version="2.3.18",revision="9dd8408c18"} 1 # HELP dovecot_auth_success Total number of all events of this kind # TYPE dovecot_auth_success counter dovecot_auth_success_total 1669 # HELP dovecot_auth_success_duration_seconds Total duration of all events of this kind # TYPE dovecot_auth_success_duration_seconds counter dovecot_auth_success_duration_seconds_total 1.317570 # HELP dovecot_auth_failures Total number of all events of this kind # TYPE dovecot_auth_failures counter dovecot_auth_failures_total 1 # HELP dovecot_auth_failures_duration_seconds Total duration of all events of this kind # TYPE dovecot_auth_failures_duration_seconds counter ... # EOF ----------------------------------------------------------------- Prometheus yml config snippet: ----------------------------------------------------------------- global: scrape_interval: 15s evaluation_interval: 15s scrape_configs: - job_name: dovecot_exporter static_configs: - targets: - 172.30.10.1:9900 ----------------------------------------------------------------- Prometheus (Docker stable latest) with debug level show these lines: ----------------------------------------------------------------- prometheus_1 | ts=2022-02-25T10:22:00.840Z caller=scrape.go:1307 level=debug component="scrape manager" scrape_pool=dovecot_exporter target=http://172.30.10.1:9900/metrics msg="Append failed" err="expected label name or left brace, got \"INVALID\"" ----------------------------------------------------------------- Therefor the service is considered as being down. I really have no ide, what I can do here. Thanks for any help in advance Christian Rößner -- Rößner-Network-Solutions Zertifizierter ITSiBe / CISO Karl-Bröger-Str. 10, 36304 Alsfeld Fax: +49 6631 78823409, Mobil: +49 171 9905345 USt-IdNr.: DE225643613, https://roessner.website PGP fingerprint: 658D 1342 B762 F484 2DDF 1E88 38A5 4346 D727 94E5

4 9

Shared mailboxes, users with dots and a bug in subscriptions
by Tobias Stein 17 May '22

17 May '22

Hi, i'm running Dovecot 2.3.14.alpha0 with shared namespaces and stumbled across some errors messages logged, when the list of subscribed mailboxes is queried by a client. For every distinct account of in the list of subscriptions two corresponding lines are logged: Jan 28 11:42:34 mx1 dovecot: auth: missing passwd file: /etc/dovecot/private/example/users Jan 28 11:42:34 mx1 dovecot: auth: missing passwd file: /etc/dovecot/private/example/users Jan 28 11:42:34 mx1 dovecot: auth: missing passwd file: /etc/dovecot/private/example/users Jan 28 11:42:36 mx1 dovecot: imap(example_user(a)example.com)<3638><xl6li/O5VsVfWxyn>: Error: mkdir(/var/run/dovecot/user-not-found/noc@example) failed: Permission denied (euid=109(vmail) egid=118(vmail) missing +w perm: /var/run/dovecot, dir owned by 0:0 mode=0755) Jan 28 11:42:36 mx1 dovecot: imap(example_user(a)example.com)<3638><xl6li/O5VsVfWxyn>: Error: mkdir(/var/run/dovecot/user-not-found/info@example) failed: Permission denied (euid=109(vmail) egid=118(vmail) missing +w perm: /var/run/dovecot, dir owned by 0:0 mode=0755) Jan 28 11:42:36 mx1 dovecot: imap(example_user(a)example.com)<3638><xl6li/O5VsVfWxyn>: Error: mkdir(/var/run/dovecot/user-not-found/root@example) failed: Permission denied (euid=109(vmail) egid=118(vmail) missing +w perm: /var/run/dovecot, dir owned by 0:0 mode=0755) Similar messages are logged, when invalid entries are listed in '/var/lib/dovecot/db/shared-mailboxes', which i already pruned and haven't received them anymore since. I think these errors are caused by an unintended behaviour when writing "~/Maildir/subscriptions", which looks (shortened) like this. V 2 INBOX/INBOX shared/noc@example com/INBOX shared/info@example com/INBOX shared/root@example com/test shared/root@example com/test test_sub The subscription-file.c explodes the name on every hierarchy separator ('.','\0') and inserts a TAB character. Unfortunately it also explodes on the DNS label delimiter „.‟. This should probably be fixed by passing a structure containing the required information to the formatter to distinguish mailboxes from domain-names. Subscription in combination with multiple domains and shared mailboxes seems broken to me. Actually i can't even explain to me, why it is working in face of the errors. :-) Unfortunately in Maildir++ the separator dot is hard-coded. There is a very old thread on this mailing list, that suggests using „auth_username_translation‟ to replace dots with a different character, but this idea is getting worse the longer i think about it. I absolutely dislike the idea to set LAYOUT=fs, namespace/separator = § to change the separators to split on, because this would mean to restructure the physical layout of all mailboxes (hierarchically) and mess around with lots of files. I attached a dovecot-sysreport to reproduce the behaviour. /etc/dovecot/private/example.com/users looks like this: ### user:password:uid:gid:(gecos):home:(shell):extra_fields noc:{SSHA512}_hash_:::::: info:{SSHA512}_hash_:::::: root:{SSHA512}_hash_:::::: Please correct me if i'm wrong or point me to a workaround, but i think the layout code needs some love. :-) Best regards Tobias

2 4

Recovering deleted mailbox
by Sean McBride 14 May '22

14 May '22

Hi all, I have a user (coworker) that accidentally deleted a mailbox and all its sub-mailboxes. I use Maildir format storage. I have backups. Is it enough to put the mailbox folder back where it was? I'm talking about the folder that contains 'cur', 'new', 'tmp', 'dovecot-uidlist', etc. Or would this desynchronize or otherwise confuse dovecot? Or is it preferable to use some doveadm command? Or...? Thanks, Sean

6 13

FTS solr or xapian and Public Folders
by David Williams 13 May '22

13 May '22

Hi all I hope someone can help with this. I'm looking at setting up FTS on a Dovecot server which has some quite big pubic folders (100,000+ emails each) and would appreciate some info about how fts both Xapian and Solr works with public folders. As background of the folders can receive emails and I use a dummy user and an alias file to deliver them for example testfolder: dummy+Public/MYTESTFOLDER(a)example.com delivers email addressed to testfolder(a)example.com to MYTESTFOLDER in the namespace Public. Has anyone successfully setup FTS with public folders using either Xapian or Solr and what happens when one user changes a folder, does that change show up for other users? E.G.: When an email is delivered by dummy and indexed will that email be indexed for other users as well? Similarly what happens that when an email is moved between, deleted from or copied to public folders by one user will that change then show up for the other users or not? I guess the basic question is: Are public folders indexed once and then the indexes are made available to each user with access or are they indexed for each user? Any info or experience that anyone has will be appreciated. Many thanks David

2 1

no full syncs after upgrading to dovecot 2.3.18
by Arnaud Abélard 12 May '22

12 May '22

Hello, I am working on replicating a server (and adding compression on the other side) and since I had "Error: dsync I/O has stalled, no activity for 600 seconds (version not received)" errors I upgraded both source and destination server with the latest 2.3 version (2.3.18). While before the upgrade all the 15 replication connections were busy after upgrading dovecot replicator dsync-status shows that most of the time nothing is being replicated at all. I can see some brief replications that last, but 99,9% of the time nothing is happening at all. I have a replication_full_sync_interval of 12 hours but I have thousands of users with their last full sync over 90 hours ago. "doveadm replicator status" also shows that i have over 35,000 queued full resync requests, but no sync, high or low queued requests so why aren't the full requests occuring? There are no errors in the logs. Thanks, Arnaud -- Arnaud Abélard Responsable pôle Système et Stockage Service Infrastructures DSIN Université de Nantes -

8 19

Authentication type for lan and haproxy (internet)
by Kees van Vloten 29 Apr '22

29 Apr '22

Hi Team, Would it be possible to setup a different authentication method depending on the connection source? I would like to use oauth2 (with mfa) for connections from internet via haproxy, whereas on the lan I run samba-dc and hence AD (krb5) is the method to use. The latter is already setup and works fine, now I want to add oauth2 for haproxy connections (or outside lan ip-range). Here's the relevant bit of "dovecot -n": auth_default_realm = EXAMPLE.COM auth_gssapi_hostname = mailserver.example.com auth_krb5_keytab = /etc/keytab/dovecot.keytab auth_master_user_separator = * auth_mechanisms = gssapi gss-spnego plain auth_realms = EXAMPLE.COM passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap pass = yes } passdb { driver = pam } userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } - Kees.

1 0

director service tries to start without any configuration
by Eirik Rye 29 Apr '22

29 Apr '22

Hello, - Dovecot 2.3.18 (originally Dovecot 2.3.16, but upgraded in an attempt to fix issue) - Ubuntu 20.04 using Dovecot repo In my organization have run a cluster of dovecot proxy machines, with the director service enabled in a transition period as we have migrated all our users to do direct proxying to backend (e.g. `proxy=y, host=192.168.0.2`), bypassing the director service. Now that all users have been migrated, it is time to disable the director service. However, even after removing all director configuration ("doveconf -n | grep director" returns nothing) and restarting the entire dovecot service (not just "doveadm reload"), the error log still spits out these errors: 2022-04-29T11:27:25+0200 director: Error: Empty server list 2022-04-29T11:27:25+0200 director: Fatal: Invalid value for director_mail_servers setting 2022-04-29T11:27:25+0200 master: Error: service(director): command startup failed, throttling for 60.000 secs Why is this happening? It's not critical, and logins work fine without the director service running, but I'm trying to figure out why and stop the log noise. doveconf stuff: root@director03:~# doveconf -n | grep director # Hostname: director03 root@director03:~# doveconf -n # 2.3.18 (9dd8408c18): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.18 (0bc28b32) # OS: Linux 5.4.0-88-generic x86_64 Ubuntu 20.04.4 LTS # Hostname: director03 debug_log_path = /var/log/dovecot.log default_vsz_limit = 768 M disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it doveadm_port = 24245 event_exporter log { format = json format_args = time-rfc3339 transport = log } imap_id_retain = yes info_log_path = /var/log/dovecot.log lmtp_add_received_header = no lmtp_proxy = yes log_path = /var/log/dovecot.err log_timestamp = "%FT%T%z " login_greeting = %{hostname} ready. login_log_format_elements = user=<%u> orig=<%{orig_user}> method=%m proto=%s rip=%r lip=%l session=<%{session}> %c %k login_proxy_max_disconnect_delay = 10 secs login_trusted_networks = <redacted> mail_plugins = quota managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext metric auth_passdb_request_finished { filter = event=auth_passdb_request_finished AND category=service:auth group_by = service passdb_name result } passdb { args = /etc/dovecot/dovecot-redis.conf.ext auth_verbose = yes driver = dict name = redis override_fields = proxy=y } protocols = imap pop3 lmtp sieve service doveadm { inet_listener http { port = 8080 } } service imap-login { client_limit = 2048 executable = imap-login inet_listener imap { address = port = 143 } inet_listener imaps { address = port = 993 ssl = yes } process_limit = 32 process_min_avail = 16 service_count = 0 vsz_limit = 1 G } service lmtp { inet_listener lmtp { port = 24 } } service managesieve-login { client_limit = 1024 executable = managesieve-login inet_listener sieve { address = port = 4190 } process_limit = 2 process_min_avail = 1 service_count = 0 vsz_limit = 1 G } service pop3-login { client_limit = 2048 executable = pop3-login inet_listener pop3 { address = port = 110 } inet_listener pop3s { address = port = 995 ssl = yes } process_limit = 4 process_min_avail = 2 service_count = 0 vsz_limit = 1 G } service stats { client_limit = 65536 inet_listener http { port = 9900 } } ssl_cert = </etc/dovecot/wildcard.crt ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1 verbose_proctitle = yes protocol imap { mail_max_userip_connections = 25 } protocol pop { mail_max_userip_connections = 25 } local 127.0.0.0/8 { doveadm_password = # hidden, use -P to show it }

2 3

Informational messages written to ~/.dovecot.sieve.log
by Gedalya 29 Apr '22

29 Apr '22

Hi, After adding "vnd.dovecot.debug" to the require list at the top of a sieve script, informational messages started appearing in ~/.dovecot.sieve.log, even when debug_log was not used. The script looks like this: require ["fileinto","imap4flags","reject","duplicate","vnd.dovecot.debug"]; # rule:[Duplicate] if duplicate :header "X-dd-hash" :last { debug_log "Duplicate message, discarding"; discard; } [ .. more rules .. ] Log lines: sieve: info: started log at 2022-04-28 23:49:43 -0400. info: msgid=<xxxxxxxxx>: stored mail into mailbox 'INBOX'. (no rule matched) sieve: info: started log at 2022-04-29 00:51:40 -0400. info: msgid=<xxxxxxxxx>: fileinto action: stored mail into mailbox 'Alerts'. I would expect / prefer that the log would be used only when debug_log was used. With too many log messages, the interesting ones would get rotated out sooner than desired.

1 0