September 2021 - dovecot

SQL help please (Mysql/MariaDB).
by C. Petro 12 Sep '21

12 Sep '21

I'm trying to build a mail server for a friend using Postfix, Dovecot and Postfixadmin, using Mariadb as the backend for user information and authentication. I've got it *almost* working. (dovecot -n at the bottom because I don't think the problem is in the SQL) I have things up to the point where mail comes in, and where you can get Thunderbird to connect to Dovecot, but delivery is looking at a different directory than IMAP: root@mailserver:/var/mail/vmail# find ./ ./ ./petro(a)republicofmean.com ./petro(a)republicofmean.com/dovecot.index.log ./petro(a)republicofmean.com/dovecot-uidvalidity.6136d995 ./petro(a)republicofmean.com/tmp <...> ./petro(a)republicofmean.com/new ./petro(a)republicofmean.com/subscriptions ./republicofmean.com ./republicofmean.com/petro ./republicofmean.com/petro/tmp ./republicofmean.com/petro/cur ./republicofmean.com/petro/new ./republicofmean.com/petro/new/1630897562.Vfd00Ib40009M307139.mailserver <....> ./republicofmean.com/petro/new/1630899062.Vfd00Ib40011M264001.mailserver ./republicofmean.com/cer ./republicofmean.com/cer/tmp ./republicofmean.com/cer/cur ./republicofmean.com/cer/new ./republicofmean.com/cer/new/1630900562.Vfd00Ib40013M218855.mailserver ./republicofmean.com/cer/new/1630899062.Vfd00Ib40010M259724.mailserver The SQL I think Dovecot is using: root@mailserver:/etc/dovecot# grep -i select dovecot-sql.conf.ext | grep -v "^#" password_query = SELECT username as user, domain, password FROM mailbox WHERE username = '%u' user_query = select CONCAT("/var/mail/vmail/",maildir) as home, 500 as uid, 500 as gid from mailbox where username="%u" Here's what Postfix is using: mysql_virtual_alias_domain_catchall_maps.cf: query = SELECT goto FROM alias,alias_domain WHERE alias_domain.alias_domain = '%d' and alias.address = CONCAT('@', alias_domain.target_domain) AND alias.active = 1 AND alias_domain.active='1' mysql_virtual_alias_domain_mailbox_maps.cf: query = SELECT maildir FROM mailbox,alias_domain WHERE alias_domain.alias_domain = '%d' and mailbox.username = CONCAT('%u', '@', alias_domain.target_domain) AND mailbox.active = 1 AND alias_domain.active='1' mysql_virtual_alias_domain_maps.cf: query = SELECT goto FROM alias,alias_domain WHERE alias_domain.alias_domain = '%d' and alias.address = CONCAT('%u', '@', alias_domain.target_domain) AND alias.active = 1 AND alias_domain.active='1' mysql_virtual_alias_maps.cf: query = SELECT goto FROM alias WHERE address='%s' AND active = '1' mysql_virtual_domains_maps.cf: query = SELECT domain FROM domain WHERE domain='%s' AND active = '1' mysql_virtual_mailbox_limit_maps.cf: query = SELECT quota FROM mailbox WHERE username='%s' AND active = '1' mysql_virtual_mailbox_maps.cf: query = SELECT maildir FROM mailbox WHERE username='%s' AND active = '1' # dovecot -n # 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7.2 () # OS: Linux 5.4.0-81-generic x86_64 Ubuntu 20.04.3 LTS ext4 # Hostname: <redacted> auth_mechanisms = plain login log_path = /var/log/dovecot.log login_greeting = Not your mail server mail_location = maildir:/var/mail/vmail/%u/ mail_max_userip_connections = 50 mail_privileged_group = vmail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocols = " imap lmtp pop3 submission" service auth-worker { user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0600 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } user = dovecot } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { unix_listener lmtp { group = postfix mode = 0600 user = postfix } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } ssl_cert = </etc/letsencrypt/live/<redacted>/fullchain.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver =

3 5

Re: Problem with the dovecot-fts-xapian package.
by Bob Marcan 12 Sep '21

12 Sep '21

On Sun, 12 Sep 2021 11:36:46 +0100 Joan Moreau <jom(a)grosjo.net> wrote: > This is where I am for now : > > https://koji.fedoraproject.org/koji/packageinfo?packageID=34417 > > Probably, I should wait for Fedora batch programs to push that into main rep > > On 2021-09-12 11:18, Joan Moreau wrote: > > > Hi Bob, > > > > I am trying to achieve that. > > > > But do you know the process of pushing an update as a maintainer, in > fedore repositories ? > > > > Thank you > > > > On 2021-09-12 11:02, Bob Marcan wrote: > > On Sun, 12 Sep 2021 09:45:35 +0100 > > Joan Moreau <jom(a)grosjo.net> wrote: > > > > Thank you for notice. > > > > What is the process to rebuild the package with recent dovecot, as > (instead of existing 1.4.12-1) ? > > There are no (yet) 1.4.12-2 in updates-testing or > updates-testing-modular repository. > > Should i'll wait for update? > > BR, Bob Got the new version and there is no more API mismatch. It's not so important for me, since i'm retired and i'm running this on my home computer. But i think it needs more support from te dovecot group. There a lot file protection issues nad lack of documentation on dovecot side. BR, Bob

3 2

suse.de dovecot issues
by Bernhard M. Wiedemann 12 Sep '21

12 Sep '21

Hi, I am one of the people taking over our new suse.de email setup (consisting of dovecot+rspamd+postfix) and wanted to report some issues we experience: 1. we use dovecot-director to distribute users between 2 backend servers that share an NFS mount. We found that it proxies lmtp to a different backend than imap of the same user and that caused NFS stale-filehandle errors on the dovecot-uidlist. It then proceeds to re-generate the dovecot-uidlist with new UIDs that creates trouble for users. a) shouldn't dovecot use locks (fcntl or flock) to protect such files from concurrent updates? b) could it generate uidlist in a way that re-generating it, assigns the same UIDs again? E.g. via hash over file content c) how to get dovecot-director to send all traffic for a user to one backend? 2. We have 2 backends so that we can do maintenance on one of them while users can still access their emails through the other backend. However, we found that stopping dovecot on one backend left users unable to access their mails. Maybe this is related to how user auth works? How to get this HA setup right, so that we don't have a single point of failure? grep PRETTY /etc/os-release PRETTY_NAME="SUSE Linux Enterprise Server 15 SP3" rpm -q dovecot23 dovecot23-2.3.15 https://www.zq1.de/~bernhard/temp/dovecot/ has some sysreports. Ciao Bernhard M.

5 12

Problem with the dovecot-fts-xapian package.
by Bob Marcan 12 Sep '21

12 Sep '21

Problem with the dovecot-fts-xapian package. Fedora 34 with latest updates. dovecot-2.3.16-1.fc34.x86_64 dovecot-fts-xapian-1.4.12-1.fc34.x86_64 [root@smicro conf.d]# systemctl restart dovecot [root@smicro conf.d]# doveadm index -A \* Fatal: Couldn't load required plugin /usr/lib64/dovecot/lib21_fts_xapian_plugin.so: Module is for different ABI version 2.3.ABIv15(2.3.15) (we have 2.3.ABIv16(2.3.16)) BR, Bob

2 1

Looking for SNI feature
by Sherin A 10 Sep '21

10 Sep '21

Hello, I am looking for an alternative feature for SNI certificates in dovecout. Current dovecot need to integreate the sni codes into the configuration as follows, local_name imap.example.org { ssl_cert = </etc/ssl/certs/imap.example.org.crt ssl_key = </etc/ssl/private/imap.example.org.key } I have thousands of domain in the server. So during the ssl certificate creation of each domain , I need to add the codes to the configuration and reload the service. That looks not a suitable feature. So my question is Do dovecot have a feature to read the certificate of every domain from another service , let us say I have service that will send the above sni setting to a socket and dovecot read it from that socket. Kindly let me know if dovecot have any such feature exists. This feature may be like the authentication using dictionary proxy service ? -- Regards Sherin A

1 0

Failed to do incremental sync
by Andrea Gabellini 09 Sep '21

09 Sep '21

Hello, Two dovecot server with replication and version 2.3.10.1 Today i have many warnings like those: Sep 9 12:13:43 dovecot: doveadm: Warning: Failed to do incremental sync for mailbox INBOX, retry with a full sync (uidnext 4465 < 4517) Sep 9 12:13:56 dovecot: doveadm: Warning: Failed to do incremental sync for mailbox INBOX, retry with a full sync (Modseq 4016 no longer in transaction log (highest=4106, last_common_uid=3451, nextuid=3452)) Do you have some suggestions on how to debug this? Or something I can check? Thanks, Andrea -- __________________________ There were computers in Biblical times. Eve had an Apple. __________________________ TIM San Marino S.p.A. Andrea Gabellini Engineering R&D TIM San Marino S.p.A. - https://www.telecomitalia.sm Via Ventotto Luglio, 212 - Piano -2 47893 - Borgo Maggiore - Republic of San Marino Tel: (+378) 0549 886237 Fax: (+378) 0549 886188 -- Informativa Privacy Questa email ha per destinatari dei contatti presenti negli archivi di TIM San Marino S.p.A.. Tutte le informazioni vengono trattate e tutelate nel rispetto della normativa vigente sulla protezione dei dati personali (Reg. EU 2016/679). Per richiedere informazioni e/o variazioni e/o la cancellazione dei vostri dati presenti nei nostri archivi potete inviare una email a privacy(a)telecomitalia.sm. Avviso di Riservatezza Il contenuto di questa e-mail e degli eventuali allegati e' strettamente confidenziale e destinato alla/e persona/e a cui e' indirizzato. Se avete ricevuto per errore questa e-mail, vi preghiamo di segnalarcelo immediatamente e di cancellarla dal vostro computer. E' fatto divieto di copiare e divulgare il contenuto di questa e-mail. Ogni utilizzo abusivo delle informazioni qui contenute da parte di persone terze o comunque non indicate nella presente e-mail potra' essere perseguito ai sensi di legge.

2 2

Using %{ssl_security} in the policy server
by Andrea Gabellini 09 Sep '21

09 Sep '21

Hello, I am looking for a way to verify, at the moment of the connection, who does not use encryption or uses weak encryption. My choice was the use of the policy server because I would like to block only a certain number of users per day, so as not to overload the support service. In the call to the policy server, only tls = true / false is passed as information. I would also be interested in %{ssl_security} but it is not present among the usable variables. Is there any way, perhaps using userdb/passdb, to be able to pass this variable to the policy server? Thanks, Andrea -- __________________________ Seen it all, done it all, can't remember most of it __________________________ TIM San Marino S.p.A. Andrea Gabellini Engineering R&D TIM San Marino S.p.A. - https://www.telecomitalia.sm Via Ventotto Luglio, 212 - Piano -2 47893 - Borgo Maggiore - Republic of San Marino Tel: (+378) 0549 886237 Fax: (+378) 0549 886188 -- Informativa Privacy Questa email ha per destinatari dei contatti presenti negli archivi di TIM San Marino S.p.A.. Tutte le informazioni vengono trattate e tutelate nel rispetto della normativa vigente sulla protezione dei dati personali (Reg. EU 2016/679). Per richiedere informazioni e/o variazioni e/o la cancellazione dei vostri dati presenti nei nostri archivi potete inviare una email a privacy(a)telecomitalia.sm. Avviso di Riservatezza Il contenuto di questa e-mail e degli eventuali allegati e' strettamente confidenziale e destinato alla/e persona/e a cui e' indirizzato. Se avete ricevuto per errore questa e-mail, vi preghiamo di segnalarcelo immediatamente e di cancellarla dal vostro computer. E' fatto divieto di copiare e divulgare il contenuto di questa e-mail. Ogni utilizzo abusivo delle informazioni qui contenute da parte di persone terze o comunque non indicate nella presente e-mail potra' essere perseguito ai sensi di legge.

2 1

SSL errors after certificate renewal
by Amol Kulkarni 08 Sep '21

08 Sep '21

Hello, After I replaced my certificate with a new one yesterday, I'm seeing some ssl related errors. There are successful pop/imap logins using SSL also. So I think the certificate in itself is fine. No user has complained as yet, so I don't know for sure. However the count of errors has surely increased after installing the new certificate. There are 2 errors seen : dovecot: imap-login: Disconnected (no auth attempts in 1 secs): user=<>, rip=, lip =, TLS handshaking: SSL_accept() failed: error:14094416:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate unknown: SSL alert number 46, session=<9m0AnVnL 2pHf4hso> dovecot: imap-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=, lip =, TLS: SSL_read() failed: error:14094412:SSL routines:SSL3_READ_BYTES:sslv3 alert bad certificate: SSL alert number 42, session=<ww/b6VfLmeR7yTog> Kindly help with some pointers. Thanks and Regards, Amol

8 8

How can I always send a vacation response with sieve?
by Steve Dondley 08 Sep '21

08 Sep '21

I don't want dovecot to wait X days before sending out another vacation response. However, setting the :days to "0" doesn't work. RFC https://www.rfc-editor.org/rfc/rfc5230.html#section-4.1 says: 4.1. Days Parameter The ":days" argument is used to specify the period in which addresses are kept and are not responded to, and is always specified in days. The minimum value used for this parameter is normally 1. Sites MAY define a different minimum value as long as the minimum is greater than 0. Sites MAY also define a maximum days value, which MUST be greater than 7, and SHOULD be greater than 30. If ":days" is omitted, the default value is either 7 or the minimum value (as defined above), whichever is greater. If the parameter given to ":days" is less than the minimum value, then the minimum value is used instead. If ":days" exceeds the site-defined maximum, the site-defined maximum is used instead. Sorry if this is more of a sieve question and is slightly off topic.

5 7

Dovecot - FTS Solr: disk usage & position information?
by Vincent Brillault 07 Sep '21

07 Sep '21

Dear all, On a local dovecot cluster currently hosting roughly 2.1TB of data, using Solr as its FTS backend, we now have 256GB of data in Solr, split in 12 shard (to which replication adds 256GB of data through 12 additional cores). I'm now trying to see if we can optimize that data. Looking at one core at random (22G), I see that the data is split mostly between - .pos files: 12G - .tim files: 4.2G - .doc files: 3.8G - .cfs files: 1.8G Looking around a bit, I found https://lucene.apache.org/core/6_2_0/core/org/apache/lucene/codecs/lucene50… (which is unfortunately a bit outdated I think) that explains each file content: - .tim: Term Dictionary - .tip: Term Index - .doc: Frequencies and Skip Data - .pos: Positions - .pay: Payloads and Offsets So clearly the file naming convention have changed, but still if .pos is really position information ("lists of positions that each term occurs at within documents."), this sounds rather useless for the dovecot integration. Looking at Solr documentation on search (https://solr.apache.org/guide/8_6/the-standard-query-parser.html) it seems that position aware query are written as `"term1 term2"~[0-9]+`. Looking at the dovecot code (https://github.com/dovecot/core/blob/master/src/plugins/fts-solr/fts-backen…) I don't see this kind of query being made, `~` only being used for fuzzy search. Has anyone ever tried to set omitTermFreqAndPositions or omitPositions to true for the text fields in the Solr Schema? It sounds that this could improve a lot the disk space used by Solr without losing any feature. The only thing I'm not too clear about is the "autoGeneratePhraseQueries" which is enabled in https://github.com/dovecot/core/blob/master/doc/solr-schema-7.7.0.xml. Thanks in advance, Vincent Brillault PS: I have attached the schema we are using for completeness. It's based on the one in the dovecot repo, with a bit of simplification for headers that don't really require as much massaging.

3 7