May 2022 - dovecot - dovecot.org

solr-jetty package in Debian 11 bullseye
by John Gateley 25 May '22

25 May '22

Hello, I am in the process of upgrading the OS on my mailserver, currently Debian 9, to Debian 11. The solr-jetty package does not appear to exist in Debian 11. Does anyone know where it is? Thanks John

2 1

Re: Duplicate messages if message is moved when using dsync
by Thom Pol 25 May '22

25 May '22

Hi, Thanks for your reply! The dsync is triggered on IMAP/POP3/LDA/LMTP activity. See my config dump from the first email. I'm wondering if my config is not OK so it causes dsync to sync twice? > you need / preferably use a database for your user auth I already do this. > run the command doveadm user '*' - it needs to return a list of users > that will trigger replication (after doing config above) This part works fine. Thanks, Thom Pol On 18/05/2022 16:51, Paul Kudla (SCOM.CA Internet Services Inc.) wrote: > ok - question > > when you say dsync are you running that manually or as a cronjob ?? > > or using replication via dsync > > there is a difference > > replication is designed to handle mailbox changes by quing the changes > on the fly and then update in the background. > > replication tracks all changes in a mailbox and then sends them one at a > time to the target server. > > i run replication with tcp (not tcps or dsync) > > I find (other then the 300+folder issues) that it runs pretty good in > real time. > > if you are running dsync manually then yes timing on its own will be a > hit and miss and it will never sync properly if something appears during > the dsync process. > > if you have the sive running as discussed that is the first step > > you should setup replication so things are handled in real time. > > pls advise if you have already done replication or just manually dsyncing. > > Below is basic replication setup but you need to make sure various > pre-requirements are in place a head of time. > > 1. you need / preferably use a database for your user auth > > you need to setup an iterate sql statement that returns the user > list of users to be replicated > > Example (i use pgsql) : > > # cat dovecot-pgsql.conf > driver = pgsql > connect = host=localhost port=5433 dbname=scom_billing user=pgsql password= > default_pass_scheme = PLAIN > > password_query = SELECT username as user, password FROM email_users > WHERE username = '%u' and password <> 'alias' and status = True and > destination = '%u' > > user_query = SELECT home, uid, gid FROM email_users WHERE username = > '%u' and password <> 'alias' and status = True and destination = '%u' > > iterate_query = SELECT "username" as user, domain FROM email_users WHERE > status = True and alias_flag = False > > #iterate_query above is what returns the user lists > > 2. run the command doveadm user '*' - it needs to return a list of users > that will trigger replication (after doing config above) > > 3. in general here is my relative parts for tcp replication > > In dovecot.conf > > --------------------------------------------------------------------------- > > mail_plugins = " virtual notify replication fts fts_lucene " > > replication_dsync_parameters = -d -N -l 300 -U > > dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u > > plugin { > > ..... > mail_replica = tcp:10.221.0.19:12345 > > #(note this is the ip address of the other server) > #it is the only config change ie server one above > #server2 would be the other server for communicating. > > > > ..... > > } > > protocol imap { > mail_max_userip_connections = 50 > mail_plugins = $mail_plugins notify replication > } > > protocol pop3 { > mail_max_userip_connections = 50 > mail_plugins = $mail_plugins notify replication > } > > protocol imaps { > mail_max_userip_connections = 25 > mail_plugins = $mail_plugins notify replication > } > > protocol pop3s { > mail_max_userip_connections = 25 > mail_plugins = $mail_plugins notify replication > } > > replication_max_conns = 100 (optional) > > replication_full_sync_interval = 1d > > service replicator { > client_limit = 0 > drop_priv_before_exec = no > idle_kill = 4294967295s > process_limit = 1 > process_min_avail = 0 > service_count = 0 > vsz_limit = 8g > unix_listener replicator-doveadm { > mode = 0600 > user = vmail > } > vsz_limit = 8192M > } > > service aggregator { > process_limit = 1000 > #vsz_limit = 1g > fifo_listener replication-notify-fifo { > user = vmail > group = vmail > mode = 0666 > } > > } > > > > ---------------------------------------------------------------------------- > > that should do it? > > > > > > > > > > > > > Happy Wednesday !!! > Thanks - paul > > Paul Kudla > > > Scom.ca Internet Services<http://www.scom.ca> > 004-1009 Byron Street South > Whitby, Ontario - Canada > L1N 4S3 > > Toronto 416.642.7266 > Main?1.866.411.7266 > Fax?1.888.892.7266

1 0

Panic: file userdb-blocking with Dovecot 2.3.19
by Niklas Meyer 25 May '22

25 May '22

Hello all, since we´ve tested around with the new dovecot release in the mailcow project we´ve came across a curious and new error with Dovecot: /auth: Panic: file userdb-blocking.c: line 124 (userdb_blocking_iter_next): assertion failed: (ctx->conn != NULL)/ *System Information:* *Dovecot Version: 2.3.19 (b3ad6004dc)** * *OS: Debian 11 (dovecot is running in a docker container)* *CPU: x86* *Filesystem: ext4* The error occurs only with the newest Dovecot release (2.3.19), the config hasn´t been changed. In the Attachment you can find all (hopefully) helpful informations. Maybe it is a simple fix. Kind regards and thanks for your help. -- The mailcow Team - Niklas --- The Infrastructure Company GmbH Parkstr. 42 47877 Willich Handelsregister: Amtsgericht Krefeld, HRB 15904 USt-IdNr.: DE308854956 Geschäftsführer: Martin Vogt /*Hinweis: *Der Inhalt dieser E-Mail ist vertraulich und nur für den in der Nachricht angegebenen Empfänger bestimmt. Es ist strengstens untersagt, irgendeinen Teil dieser Nachricht ohne schriftliche Zustimmung des Absenders an Dritte weiterzugeben. Wenn Sie diese Nachricht irrtümlich erhalten haben, antworten Sie bitte auf diese Nachricht und folgen Sie mit ihrer Löschung, so dass wir sicherstellen können, dass ein solcher Fehler in Zukunft nicht mehr vorkommt./

3 2

Message attachments, relocated with Tbird in Dovecot maildir store, not openable; reversible by moving BACK to inbox?
by PGNet Dev 25 May '22

25 May '22

I run dovecot --version 2.3.17.1 (476cd46418) on lsb_release -rd Description: Fedora release 35 (Thirty Five) Release: 35 mail store is maildir mail_location = maildir:/data/vmail/%d/%n/Maildir:CONTROL=/data/vmail/%d/%n/_control:INDEX=/var/vmail-index/%d/%n:LAYOUT=fs:UTF-8 front-end client is thunderbird --version Thunderbird 91.7.0 also on lsb_release -rd Description: Fedora release 35 (Thirty Five) Release: 35 mail is delivered to the store via lmtp transport from a Postfix agent mail, with attachments (e.g., .PDF) is received OK it's readable, attachments can be opened and/or saved to disk\ if, in TBird client, i move the message to any other folder in the IMAP hierarchy, the message relocates, as expected. the message itself is still readable, and _appears_ to have the attachment. but, attempting to OPEN the *attachment*, with e.g. Okular, from within the relocated message pops up a dialog, "Could not open file:///tmp/pid-28435/manual_13542AE.pdf" and saving it to Desktop, ls -al manual_13542AE.pdf -rw-r--r-- 1 pgnd pgnd 27 Apr 1 06:01 manual_13542AE.pdf file manual_13542AE.pdf manual_13542AE.pdf: data then opening it similarly returns "Could not open file:///home/pgnd/Desktop/manual_13542AE.pdf." Moving the message, via TBird, **BACK** to the Inbox, cures the problem -- -- I can immediately, again, open & save the attachments. This 'feels' like cache management misconfiguration to me. Not sure whether in Tbird &/or Dovecot, or what to look for specificially. Hints as to cause/fix of this attachment issue? Or event what/where, specifically, to log?

4 12

doveadm mailbox status -u paul@scom.ca -t all (syntax)
by Paul Kudla (SCOM.CA Internet Services Inc.) 24 May '22

24 May '22

Ok I know this is a general question and at my level I should be able to figure this out all i want is the complete status of an inbox ? # doveadm mailbox status -u paul(a)scom.ca -t all keeps returning .... doveadm mailbox status [-u <user>|-A] [-S <socket_path>] <fields> <mailbox> [...] ie no specific error. and no status answer just looking for default info ? -- Happy Monday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email paul(a)scom.ca

3 5

enable/control fts-tika debug logging in Dovecot 2.3.18 + Tika Server 2.4.0?
by PGNet Dev 24 May '22

24 May '22

i run dovecot-2.3.18-1.fc36.x86_64 i've installed Apache Tika, v 2.4.0 ls -al tika-server-standard-2.4.0.jar -rw-r--r-- 1 root root 59M May 2 09:53 tika-server-standard-2.4.0.jar tika's listening telnet 127.0.0.1 9998 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. telnet> and responds to a test curl \ -T /tmp/test.pdf \ http://127.0.0.1:9998/meta pdf:unmappedUnicodeCharsPerPage,0,0,0,0,0,0,0,0,0,0,0,0,0,0 pdf:PDFVersion,1.4 xmp:CreatorTool,Adobe InDesign 15.1 (Macintosh) pdf:hasXFA,false access_permission:modify_annotations,true access_permission:can_print_degraded,true X-TIKA:Parsed-By-Full-Set,org.apache.tika.parser.DefaultParser,org.apache.tika.parser.pdf.PDFParser dcterms:created,2020-08-13T14:55:46Z language,en dcterms:modified,2020-09-24T23:38:28Z dc:format,application/pdf; version=1.4 xmpMM:DocumentID,xmp.id:8a612346-9d03-4caf-8ebf-da6f3716ed0a pdf:docinfo:creator_tool,Adobe InDesign 15.1 (Macintosh) access_permission:fill_in_form,true pdf:docinfo:modified,2020-09-24T23:38:28Z pdf:hasCollection,false pdf:encrypted,false pdf:hasMarkedContent,true Content-Type,application/pdf dc:language,en-US pdf:producer,Adobe PDF Library 15.0 access_permission:extract_for_accessibility,true access_permission:assemble_document,true xmpTPg:NPages,14 pdf:hasXMP,true pdf:charsPerPage,84,676,1653,1914,814,1022,645,1221,1087,732,887,1295,1263,149 access_permission:extract_content,true xmpMM:DerivedFrom:DocumentID,xmp.did:b98726d4-04c4-48f5-88be-0a48a0074356 access_permission:can_print,true pdf:docinfo:trapped,false X-TIKA:Parsed-By,org.apache.tika.parser.DefaultParser,org.apache.tika.parser.pdf.PDFParser xmpMM:DerivedFrom:InstanceID,xmp.iid:3dd6a91f-a114-4d63-804e-e2b749c15075 pdf:annotationTypes,null access_permission:can_modify,true pdf:docinfo:producer,Adobe PDF Library 15.0 pdf:docinfo:created,2020-08-13T14:55:46Z pdf:annotationSubtypes,Link in dovecot config, i've added plugin { fts_tika = http://127.0.0.1:9998/tika/ } and log_debug = (category=fts-flatcurve OR category=fts-tika) on message receipt, I see verbose logs for fts-flatcurve, as expected, but not a trace of output from fts-tika, in dovecot logs how to correctly turn on debug/verbose logging for fts-tika use in/by dovecot?

2 2

Many Thanks to the Programmers
by Paul Kudla (SCOM.CA Internet Services Inc.) 23 May '22

23 May '22

I am still in testing mode with 2.3.19 but the replication with the folders over 300+ seems to be fixed everything synced ok on its own within 30 minutes Again MUSH APPRECIATED ! Moving to 2.3.19 was woth it! -- Happy Monday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email paul(a)scom.ca

1 0

mail_replica in userdb?
by Coy Hile 23 May '22

23 May '22

I seem to have got replication working from an older instance to a newer one (insofar as the data are present on the new box) that I’m looking to stand up to replace a newer box. In this case, nothing is configured to talk to the new machine yet, so the replication is unidirectional (old -> new). However, I’m seeing an error like like this repeatedly in syslog: 2022-05-21T23:23:15.295252+00:00 basement-imap01 dovecot: doveadm(hile(a)coyhile.com): Error: sync: User has no mail_replica in userdb What setting did I miss? kdoveconf -n # 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7.2 () # OS: Linux 5.4.0-110-generic x86_64 Ubuntu 20.04.4 LTS # Hostname: basement-imap01.coyhile.com doveadm_password = # hidden, use -P to show it mail_location = maildir:~/Maildir mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocols = " imap lmtp" service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = vmail } unix_listener replication-notify { mode = 0666 user = vmail } } service doveadm { inet_listener { port = 12345 } user = vmail } service replicator { unix_listener replicator-doveadm { mode = 0666 } } ssl_cert = </etc/dovecot/private/dovecot.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap override_fields = uid=vmail gid=vmail home=/data/mail/vmail/%d/%n } Is that error something that I can (for the moment) ignore since I’m explicitly NOT trying to have this host replicate data anywhere, or do I need to tweak something? -- Coy Hile coy.hile(a)coyhile.com

3 7

SQL user_query quota_rule for *:messages=
by James 23 May '22

23 May '22

Hello, dovecot.conf sets the system default quota for storage and message count: plugin { ... quota_rule = *:storage=1G quota_rule2 = *:messages=10000 ... } This quota_rule for the storage is overridden by an SQL user_query in dovecot-sql.conf: user_query = "SELECT ... '*:storage=' || maxstorage || 'M' AS userdb_quota_rule, ...;" Is it possible to override the message count limit with an SQL user_query (and password_query)? I tried returning userdb_quota_rule2 in the SQL but it did not work. "SELECT ... '*:messages=' || maxcount AS userdb_quota_rule2 ...;" Ref: https://doc.dovecot.org/configuration_manual/authentication/user_database_e…

2 2

Dovecot 2.3.18 and OpenMetrics-Prometheus problem
by Christian Rößner 23 May '22

23 May '22

Hi, I have defined several metrics in Dovecot and activated the metrics service. After the I tried to include this in Prometheues, but I get strange errors. I gave up now after several hours in the hope that this might be a bug. So I ask for help here: Distribution: Rocky 8 Official Dovecot Repo Version: 2.3.18 Config for metrics, service and events: ----------------------------------------------------------------- metric auth_success { exporter = log filter = event=auth_request_finished AND success=yes } metric auth_failures { exporter = log filter = event=auth_request_finished AND NOT success=yes } metric auth_client_userdb { filter = event=auth_client_userdb_lookup_finished } metric auth_master { exporter = log filter = event=auth_master_client_login_finished } metric client_connected { filter = event=client_connection_connected } metric client_disconnected { filter = event=client_connection_disconnected group_by = reason } metric imap_command { filter = event=imap_command_finished group_by = cmd_name tagged_reply_state } metric lmtp_command { filter = event=smtp_server_command_finished AND protocol=lmtp group_by = cmd_name status_code duration:exponential:1:5:10 } metric imap_hibernate { filter = event=imap_client_hibernated group_by = error } metric imap_unhibernate { filter = event=imap_client_hibernated group_by = reason error } metric mail_delivery { filter = event=mail_delivery_finished group_by = duration:exponential:1:5:10 } metric sql_query { filter = event=sql_query_finished group_by = error_code } metric sieve_actions { filter = event=sieve_action_finished group_by = action_name error } metric managesieve { filter = event=managesieve_command_finished group_by = script_name } metric dict_lookups { filter = event=dict_lookup_finished group_by = driver error } service stats { inet_listener http { port = 9900 } } event_exporter log { format = json format_args = time-rfc3339 transport = log } ----------------------------------------------------------------- Shortened curl output: ----------------------------------------------------------------- # HELP process_start_time_seconds Timestamp of service start # TYPE process_start_time_seconds gauge process_start_time_seconds 1645743604 # HELP dovecot_build Dovecot build information # TYPE dovecot_build info dovecot_build_info{version="2.3.18",revision="9dd8408c18"} 1 # HELP dovecot_auth_success Total number of all events of this kind # TYPE dovecot_auth_success counter dovecot_auth_success_total 1669 # HELP dovecot_auth_success_duration_seconds Total duration of all events of this kind # TYPE dovecot_auth_success_duration_seconds counter dovecot_auth_success_duration_seconds_total 1.317570 # HELP dovecot_auth_failures Total number of all events of this kind # TYPE dovecot_auth_failures counter dovecot_auth_failures_total 1 # HELP dovecot_auth_failures_duration_seconds Total duration of all events of this kind # TYPE dovecot_auth_failures_duration_seconds counter ... # EOF ----------------------------------------------------------------- Prometheus yml config snippet: ----------------------------------------------------------------- global: scrape_interval: 15s evaluation_interval: 15s scrape_configs: - job_name: dovecot_exporter static_configs: - targets: - 172.30.10.1:9900 ----------------------------------------------------------------- Prometheus (Docker stable latest) with debug level show these lines: ----------------------------------------------------------------- prometheus_1 | ts=2022-02-25T10:22:00.840Z caller=scrape.go:1307 level=debug component="scrape manager" scrape_pool=dovecot_exporter target=http://172.30.10.1:9900/metrics msg="Append failed" err="expected label name or left brace, got \"INVALID\"" ----------------------------------------------------------------- Therefor the service is considered as being down. I really have no ide, what I can do here. Thanks for any help in advance Christian Rößner -- Rößner-Network-Solutions Zertifizierter ITSiBe / CISO Karl-Bröger-Str. 10, 36304 Alsfeld Fax: +49 6631 78823409, Mobil: +49 171 9905345 USt-IdNr.: DE225643613, https://roessner.website PGP fingerprint: 658D 1342 B762 F484 2DDF 1E88 38A5 4346 D727 94E5

4 9