March 2022 - dovecot - dovecot.org

Unclear how to get both user and domain quotas to work
by Hans van Zijst 10 Sep '22

10 Sep '22

Hi, First of all: my apologies for this long message. I'm trying to post all the information anybody could need to help me on my way. What I'm trying to make is an installation where both users and domains can have quotas. Ideally I would configure a quota for a domain and set a quota of 0 for all users within that domain, so that mail will be accepted up to the point where the domain quota is reached. Setting up user quota was fairly easy, there's lots of documentation around and examples to take inspiration from. Not so much for domain quota however, or I'm really bad at searching the Net. I'm running Debian Bullseye here: MariaDB 10.5.12 Dovecot 2.3.13 Postfixadmin 3.3.10 I want to use dictionaries in MySQL, because eventually this will be a platform with several Dovecot and Postfix servers. At this time I'm only running one Dovecot installation and a few Postfix servers (two for communication with the outside world and one to do the virus and spam scanning). My current setup for user quota is how Postfixadmin wants it: MariaDB [postfixadmin]> desc quota2; +----------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +----------+--------------+------+-----+---------+-------+ | username | varchar(100) | NO | PRI | NULL | | | bytes | bigint(20) | NO | | 0 | | | messages | int(11) | NO | | 0 | | +----------+--------------+------+-----+---------+-------+ 3 rows in set (0.001 sec) In /etc/dovecot/conf.d/90-quota.conf I have this: plugin { quota = dict:User quota::proxy::sqlquota quota_rule = *:bytes=1024M quota_rule2 = Trash:bytes=+100M quota_vsizes = yes quota_status_success = DUNNO quota_status_nouser = DUNNO quota_status_overquota = "552 5.2.2 Mailbox is full" } dict { sqlquota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } The SQL-configuration /etc/dovecot/dovecot-dict-sql.conf.ext is this: connect = host=... dbname=... user=... password=... map { pattern = priv/quota/storage table = quota2 username_field = username value_field = bytes } map { pattern = priv/quota/messages table = quota2 username_field = username value_field = messages } map { pattern = shared/expire/$user/$mailbox table = expires value_field = expire_stamp fields { username = $user mailbox = $mailbox } } I use LMTP to accept mail from Postfix; the quota-plugin is activated for LMTP, so every incoming mail should be counted. This works like a charm: if I don't configure a quota for a user, he gets the default 1GB, if I override it in Postfixadmin, then that's what he gets. I'm running quota-status on a TCP-port, so the gateway SMTP-servers can connect to Dovecot for a quotacheck to decide whether to accept or reject an incoming message. Again, this works flawlessly. I can't find any serious documentation about this mapping thing. In particular, what's with that "pattern"? All I can find in the documentation is: "The dict key must match exactly priv/quota/storage. The dict keys are hardcoded in the Dovecot code, so depending on what functionality you’re configuring you need to know the available dict keys used it." -- https://doc.dovecot.org/configuration_manual/dict/#dict-sql If anyone can point me at better documentation than the actual source code: please! I browsed through the quota-plugin source code, but I'm not a programmer and couldn't find what I thought I was looking for. That said, I'm not sure where I got the third "map" from, with the shared/expire pattern. I seem to have forgotten to document that bit of my journey. Now, domain quotas... I've tried all kinds of stuff, but at the moment I'm stuck with something that seems to almost work. I added a second quota root: plugin { quota2 = dict:Domain quota:%d:proxy::sqldomainquota quota2_rule = *:bytes=1024M quota2_status_success = DUNNO quota2_status_nouser = DUNNO quota2_status_overquota = "552 5.2.2 Domain quota exceeded" quota2_vsizes = yes } dict { sqldomainquota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } This dict refers to the same mapping as the one for user quotas, and I think that's the way it should be. Please correct me if I'm wrong. This creates a new user in the quota2 table, by the name of just the domain. Unfortunately, it simply fills it with the numbers from the first user it can find in that domain, not all users in that domain combined. What I think I should do, is change something in the dict so that it sums up all users in that domain, something like this: SELECT SUM(bytes) FROM quota2 WHERE username LIKE '%@domain'; Problem is that I can't find anything useful online with regards to this, I'm not even sure if I'm thinking in the right direction here. Any hints, suggestions or corrections? Or pointers to articles or documentation I might have missed? Kind regards, Hans

2 1

running alternate dovecot instances on the same server
by Chris Hoogendyk 21 Jun '22

21 Jun '22

I'm posting to the list, but not on the list. I presume that means a reply-all to get to me as well as the list? We have two servers (dovecot --version: 2.2.22 (fe789d2)) that handle email for two different departments. We are transitioning mail service to the University central IT. They need to move accounts in an automated fashion and therefore need a master password to our dovecot servers. However, we are running with LDAP authentication, and I understand that a master password is not possible in that configuration. Would it be possible to run an alternate dovecot process that would use local account authentication, have a master password, and use an alternate port for connecting? Ideally it would only read accounts without changing anything, and would not interfere with the operation of the other dovecot process. I'm hoping that I could copy the configuration files, make these changes, and then launch it manually without any startup scripts in /etc/inetd.conf. Oh, by the way, we are running Ubuntu 16.04 LTS and have contracts with Ubuntu Advantage for ongoing patch support. The dovecot version is from the distribution, installed with aptitude. -- --------------- Chris Hoogendyk - O__ ---- Systems Administrator, Retired c/ /'_ --- Biology & Geosciences Departments (*) \(*) -- 315 Morrill Science Center III ~~~~~~~~~~ - University of Massachusetts, Amherst <hoogendyk(a)bio.umass.edu> --------------- Erdös 4

3 7

What is the state of SMTPUTF8 support?
by Sylvhem 02 Jun '22

02 Jun '22

Hi! It’s been some time since I last saw a mention of SMTPUTF8 here. Last time I heard about it, it was worked on. Does anybody have news about that? -- Sylvhem

5 8

Dovecot 2.3.18 and OpenMetrics-Prometheus problem
by Christian Rößner 23 May '22

23 May '22

Hi, I have defined several metrics in Dovecot and activated the metrics service. After the I tried to include this in Prometheues, but I get strange errors. I gave up now after several hours in the hope that this might be a bug. So I ask for help here: Distribution: Rocky 8 Official Dovecot Repo Version: 2.3.18 Config for metrics, service and events: ----------------------------------------------------------------- metric auth_success { exporter = log filter = event=auth_request_finished AND success=yes } metric auth_failures { exporter = log filter = event=auth_request_finished AND NOT success=yes } metric auth_client_userdb { filter = event=auth_client_userdb_lookup_finished } metric auth_master { exporter = log filter = event=auth_master_client_login_finished } metric client_connected { filter = event=client_connection_connected } metric client_disconnected { filter = event=client_connection_disconnected group_by = reason } metric imap_command { filter = event=imap_command_finished group_by = cmd_name tagged_reply_state } metric lmtp_command { filter = event=smtp_server_command_finished AND protocol=lmtp group_by = cmd_name status_code duration:exponential:1:5:10 } metric imap_hibernate { filter = event=imap_client_hibernated group_by = error } metric imap_unhibernate { filter = event=imap_client_hibernated group_by = reason error } metric mail_delivery { filter = event=mail_delivery_finished group_by = duration:exponential:1:5:10 } metric sql_query { filter = event=sql_query_finished group_by = error_code } metric sieve_actions { filter = event=sieve_action_finished group_by = action_name error } metric managesieve { filter = event=managesieve_command_finished group_by = script_name } metric dict_lookups { filter = event=dict_lookup_finished group_by = driver error } service stats { inet_listener http { port = 9900 } } event_exporter log { format = json format_args = time-rfc3339 transport = log } ----------------------------------------------------------------- Shortened curl output: ----------------------------------------------------------------- # HELP process_start_time_seconds Timestamp of service start # TYPE process_start_time_seconds gauge process_start_time_seconds 1645743604 # HELP dovecot_build Dovecot build information # TYPE dovecot_build info dovecot_build_info{version="2.3.18",revision="9dd8408c18"} 1 # HELP dovecot_auth_success Total number of all events of this kind # TYPE dovecot_auth_success counter dovecot_auth_success_total 1669 # HELP dovecot_auth_success_duration_seconds Total duration of all events of this kind # TYPE dovecot_auth_success_duration_seconds counter dovecot_auth_success_duration_seconds_total 1.317570 # HELP dovecot_auth_failures Total number of all events of this kind # TYPE dovecot_auth_failures counter dovecot_auth_failures_total 1 # HELP dovecot_auth_failures_duration_seconds Total duration of all events of this kind # TYPE dovecot_auth_failures_duration_seconds counter ... # EOF ----------------------------------------------------------------- Prometheus yml config snippet: ----------------------------------------------------------------- global: scrape_interval: 15s evaluation_interval: 15s scrape_configs: - job_name: dovecot_exporter static_configs: - targets: - 172.30.10.1:9900 ----------------------------------------------------------------- Prometheus (Docker stable latest) with debug level show these lines: ----------------------------------------------------------------- prometheus_1 | ts=2022-02-25T10:22:00.840Z caller=scrape.go:1307 level=debug component="scrape manager" scrape_pool=dovecot_exporter target=http://172.30.10.1:9900/metrics msg="Append failed" err="expected label name or left brace, got \"INVALID\"" ----------------------------------------------------------------- Therefor the service is considered as being down. I really have no ide, what I can do here. Thanks for any help in advance Christian Rößner -- Rößner-Network-Solutions Zertifizierter ITSiBe / CISO Karl-Bröger-Str. 10, 36304 Alsfeld Fax: +49 6631 78823409, Mobil: +49 171 9905345 USt-IdNr.: DE225643613, https://roessner.website PGP fingerprint: 658D 1342 B762 F484 2DDF 1E88 38A5 4346 D727 94E5

4 9

Shared mailboxes, users with dots and a bug in subscriptions
by Tobias Stein 17 May '22

17 May '22

Hi, i'm running Dovecot 2.3.14.alpha0 with shared namespaces and stumbled across some errors messages logged, when the list of subscribed mailboxes is queried by a client. For every distinct account of in the list of subscriptions two corresponding lines are logged: Jan 28 11:42:34 mx1 dovecot: auth: missing passwd file: /etc/dovecot/private/example/users Jan 28 11:42:34 mx1 dovecot: auth: missing passwd file: /etc/dovecot/private/example/users Jan 28 11:42:34 mx1 dovecot: auth: missing passwd file: /etc/dovecot/private/example/users Jan 28 11:42:36 mx1 dovecot: imap(example_user(a)example.com)<3638><xl6li/O5VsVfWxyn>: Error: mkdir(/var/run/dovecot/user-not-found/noc@example) failed: Permission denied (euid=109(vmail) egid=118(vmail) missing +w perm: /var/run/dovecot, dir owned by 0:0 mode=0755) Jan 28 11:42:36 mx1 dovecot: imap(example_user(a)example.com)<3638><xl6li/O5VsVfWxyn>: Error: mkdir(/var/run/dovecot/user-not-found/info@example) failed: Permission denied (euid=109(vmail) egid=118(vmail) missing +w perm: /var/run/dovecot, dir owned by 0:0 mode=0755) Jan 28 11:42:36 mx1 dovecot: imap(example_user(a)example.com)<3638><xl6li/O5VsVfWxyn>: Error: mkdir(/var/run/dovecot/user-not-found/root@example) failed: Permission denied (euid=109(vmail) egid=118(vmail) missing +w perm: /var/run/dovecot, dir owned by 0:0 mode=0755) Similar messages are logged, when invalid entries are listed in '/var/lib/dovecot/db/shared-mailboxes', which i already pruned and haven't received them anymore since. I think these errors are caused by an unintended behaviour when writing "~/Maildir/subscriptions", which looks (shortened) like this. V 2 INBOX/INBOX shared/noc@example com/INBOX shared/info@example com/INBOX shared/root@example com/test shared/root@example com/test test_sub The subscription-file.c explodes the name on every hierarchy separator ('.','\0') and inserts a TAB character. Unfortunately it also explodes on the DNS label delimiter „.‟. This should probably be fixed by passing a structure containing the required information to the formatter to distinguish mailboxes from domain-names. Subscription in combination with multiple domains and shared mailboxes seems broken to me. Actually i can't even explain to me, why it is working in face of the errors. :-) Unfortunately in Maildir++ the separator dot is hard-coded. There is a very old thread on this mailing list, that suggests using „auth_username_translation‟ to replace dots with a different character, but this idea is getting worse the longer i think about it. I absolutely dislike the idea to set LAYOUT=fs, namespace/separator = § to change the separators to split on, because this would mean to restructure the physical layout of all mailboxes (hierarchically) and mess around with lots of files. I attached a dovecot-sysreport to reproduce the behaviour. /etc/dovecot/private/example.com/users looks like this: ### user:password:uid:gid:(gecos):home:(shell):extra_fields noc:{SSHA512}_hash_:::::: info:{SSHA512}_hash_:::::: root:{SSHA512}_hash_:::::: Please correct me if i'm wrong or point me to a workaround, but i think the layout code needs some love. :-) Best regards Tobias

2 4

FTS solr or xapian and Public Folders
by David Williams 13 May '22

13 May '22

Hi all I hope someone can help with this. I'm looking at setting up FTS on a Dovecot server which has some quite big pubic folders (100,000+ emails each) and would appreciate some info about how fts both Xapian and Solr works with public folders. As background of the folders can receive emails and I use a dummy user and an alias file to deliver them for example testfolder: dummy+Public/MYTESTFOLDER(a)example.com delivers email addressed to testfolder(a)example.com to MYTESTFOLDER in the namespace Public. Has anyone successfully setup FTS with public folders using either Xapian or Solr and what happens when one user changes a folder, does that change show up for other users? E.G.: When an email is delivered by dummy and indexed will that email be indexed for other users as well? Similarly what happens that when an email is moved between, deleted from or copied to public folders by one user will that change then show up for the other users or not? I guess the basic question is: Are public folders indexed once and then the indexes are made available to each user with access or are they indexed for each user? Any info or experience that anyone has will be appreciated. Many thanks David

2 1

Permissions and ownership on /dev/shm/dovecot
by doug 12 Apr '22

12 Apr '22

Hi, Environment: Dovecot 2.3.18 running on CentOS 7, mdbox, LDAP users I'm in the process of moving my mailboxes to NFS and moving with lock and index files in temp storage following instructions from https://doc.dovecot.org/configuration_manual/nfs. I set mail_location as: mail_location = mdbox:/mailstore/%u/mail:VOLATILEDIR=/dev/shm/dovecot/%u:LISTINDEX=/dev/shm/dovecot/%u/dovecot.list.index What I discovered is /dev/shm/dovecot is created by the initial user who accesses their mail from a client, and with permissions 700. This prevents subsequent users from creating their own index and lock files. # ls -l /dev/shm/dovecot total 0 drwx------ 2 mary users 60 Mar 25 10:00 mary Sample error message from maillog during mail delivery and from a dsync script. Mar 25 10:37:15 mailsrv1 dovecot: imap(doug)<19284><WKcX5gvbRe7AqFhA>: Error: mkdir(/dev/shm/dovecot/doug) failed: Permission denied (euid=1002(doug) egid=100(users) missing +x perm: /dev/shm/dovecot, dir owned by 97:100 mode=0700) dsync(test): Error: mkdir(/dev/shm/dovecot/test) failed: Permission denied (euid=2003(test) egid=100(users) missing +x perm: /dev/shm/dovecot, dir owned by 97:100 mode=0700) I couldn't locate documentation or discussions on how to set the ownership or permissions for /dev/shm/dovecot in the Dovecot configuration files. As a hack, I added this to /usr/libexec/dovecot/prestartscript. ! [[ -d /dev/shm/dovecot ]] && mkdir /dev/shm/dovecot chown dovecot:users /dev/shm/dovecot chmod 770 /dev/shm/dovecot This solved the problem, but left me wondering if I missed something obvious or if I am setting myself up for a problem later on, like with a Dovecot version upgrade. I could run these commands at bootup out of rc.local or a systemd script rather than customizing a Dovecot provided script. Is there a appropriate way of doing this that I missed? TIA, Doug

3 6

macOS ManageSieve client?
by Gerben Wierda 07 Apr '22

07 Apr '22

I have a dovecot & dovecot-sieve running under MacPorts on my macOS ’server’. I do a very low tech maintenance on sieve, by using the cli on the server and edit sieve by hand for myself. I’d like to open this up to other users. How do people use this from their macOS clients? For this, the ManageSieve protocol exists and this is implemented by dovecot-sieve, but other than installing roundcube and offering a web-based mail client that also supports ManageSieve, is there another way? A ManageSieve client that directly runs on the macOS client and interfaces with dovecot-sieve on the server? After all, routing mail to different mailboxes better happens server-side so that all clients profit. Gerben Wierda (LinkedIn <https://www.linkedin.com/in/gerbenwierda>) R&A IT Strategy <https://ea.rna.nl/> (main site) Book: Chess and the Art of Enterprise Architecture <https://ea.rna.nl/the-book/> Book: Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/>

6 5

Dovecot Submission - rate limiting
by itanguy＠univ-brest.fr 31 Mar '22

31 Mar '22

Hello, is there any way to rate limit submission mail (counting per user) when using Dovecot/Submission (no Postfix)? Using a milter seems impossible. Using Dovecot Events seems "too" global (not per users discrimination). Maybe rate-limiting has to be external and then include in the pass_db query ? Thank your for any hints. Ismaël Tanguy

3 3

Simplifying Support of Virtual and System Users
by Mark Olbert 29 Mar '22

29 Mar '22

Apologies if this has already been raised here (which I suspect it has 😊). I tried to raise it as an issue over on github but issues are not enabled for the repository. The support for mixing virtual users, with fully-qualified email addresses, and system users could be simpler. Assuming it doesn’t mess up other stuff in the code base, of course 😊. The problem appears to be that the PAM passwd module requires just user names without a domain (which makes sense given that they’re system users) but does not, so far as I can see, support the username_format argument. In my setup, the default structure of 10-auth.conf demonstrates this: auth_username_format = %n That means all username arguments lack the domain part…which complicates using fully-qualified ones for virtual users. I realize I could assign arbitrary unique names to the virtual accounts in the lookup file. But that complicates administering the system, so I want to be able to include the domain for virtual users. After about five hair-pulling hours of wrestling with the configuration I stumbled across an answer utilizing conditionals<https://serverfault.com/questions/260488/dovecot-user-lookup-fails-when-usi…> on ServerFault. It works fine. But being able to pass a username_format parameter to the PAM module (which I tried, but it was rejected) would be a lot simpler, and a lot more intuitive. - Mark

5 4