- dovecot - dovecot.org

Message attachments, relocated with Tbird in Dovecot maildir store, not openable; reversible by moving BACK to inbox?
by PGNet Dev 25 May '22

25 May '22

I run dovecot --version 2.3.17.1 (476cd46418) on lsb_release -rd Description: Fedora release 35 (Thirty Five) Release: 35 mail store is maildir mail_location = maildir:/data/vmail/%d/%n/Maildir:CONTROL=/data/vmail/%d/%n/_control:INDEX=/var/vmail-index/%d/%n:LAYOUT=fs:UTF-8 front-end client is thunderbird --version Thunderbird 91.7.0 also on lsb_release -rd Description: Fedora release 35 (Thirty Five) Release: 35 mail is delivered to the store via lmtp transport from a Postfix agent mail, with attachments (e.g., .PDF) is received OK it's readable, attachments can be opened and/or saved to disk\ if, in TBird client, i move the message to any other folder in the IMAP hierarchy, the message relocates, as expected. the message itself is still readable, and _appears_ to have the attachment. but, attempting to OPEN the *attachment*, with e.g. Okular, from within the relocated message pops up a dialog, "Could not open file:///tmp/pid-28435/manual_13542AE.pdf" and saving it to Desktop, ls -al manual_13542AE.pdf -rw-r--r-- 1 pgnd pgnd 27 Apr 1 06:01 manual_13542AE.pdf file manual_13542AE.pdf manual_13542AE.pdf: data then opening it similarly returns "Could not open file:///home/pgnd/Desktop/manual_13542AE.pdf." Moving the message, via TBird, **BACK** to the Inbox, cures the problem -- -- I can immediately, again, open & save the attachments. This 'feels' like cache management misconfiguration to me. Not sure whether in Tbird &/or Dovecot, or what to look for specificially. Hints as to cause/fix of this attachment issue? Or event what/where, specifically, to log?

4 12

doveadm mailbox status -u paul@scom.ca -t all (syntax)
by Paul Kudla (SCOM.CA Internet Services Inc.) 24 May '22

24 May '22

Ok I know this is a general question and at my level I should be able to figure this out all i want is the complete status of an inbox ? # doveadm mailbox status -u paul(a)scom.ca -t all keeps returning .... doveadm mailbox status [-u <user>|-A] [-S <socket_path>] <fields> <mailbox> [...] ie no specific error. and no status answer just looking for default info ? -- Happy Monday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email paul(a)scom.ca

3 5

enable/control fts-tika debug logging in Dovecot 2.3.18 + Tika Server 2.4.0?
by PGNet Dev 24 May '22

24 May '22

i run dovecot-2.3.18-1.fc36.x86_64 i've installed Apache Tika, v 2.4.0 ls -al tika-server-standard-2.4.0.jar -rw-r--r-- 1 root root 59M May 2 09:53 tika-server-standard-2.4.0.jar tika's listening telnet 127.0.0.1 9998 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. telnet> and responds to a test curl \ -T /tmp/test.pdf \ http://127.0.0.1:9998/meta pdf:unmappedUnicodeCharsPerPage,0,0,0,0,0,0,0,0,0,0,0,0,0,0 pdf:PDFVersion,1.4 xmp:CreatorTool,Adobe InDesign 15.1 (Macintosh) pdf:hasXFA,false access_permission:modify_annotations,true access_permission:can_print_degraded,true X-TIKA:Parsed-By-Full-Set,org.apache.tika.parser.DefaultParser,org.apache.tika.parser.pdf.PDFParser dcterms:created,2020-08-13T14:55:46Z language,en dcterms:modified,2020-09-24T23:38:28Z dc:format,application/pdf; version=1.4 xmpMM:DocumentID,xmp.id:8a612346-9d03-4caf-8ebf-da6f3716ed0a pdf:docinfo:creator_tool,Adobe InDesign 15.1 (Macintosh) access_permission:fill_in_form,true pdf:docinfo:modified,2020-09-24T23:38:28Z pdf:hasCollection,false pdf:encrypted,false pdf:hasMarkedContent,true Content-Type,application/pdf dc:language,en-US pdf:producer,Adobe PDF Library 15.0 access_permission:extract_for_accessibility,true access_permission:assemble_document,true xmpTPg:NPages,14 pdf:hasXMP,true pdf:charsPerPage,84,676,1653,1914,814,1022,645,1221,1087,732,887,1295,1263,149 access_permission:extract_content,true xmpMM:DerivedFrom:DocumentID,xmp.did:b98726d4-04c4-48f5-88be-0a48a0074356 access_permission:can_print,true pdf:docinfo:trapped,false X-TIKA:Parsed-By,org.apache.tika.parser.DefaultParser,org.apache.tika.parser.pdf.PDFParser xmpMM:DerivedFrom:InstanceID,xmp.iid:3dd6a91f-a114-4d63-804e-e2b749c15075 pdf:annotationTypes,null access_permission:can_modify,true pdf:docinfo:producer,Adobe PDF Library 15.0 pdf:docinfo:created,2020-08-13T14:55:46Z pdf:annotationSubtypes,Link in dovecot config, i've added plugin { fts_tika = http://127.0.0.1:9998/tika/ } and log_debug = (category=fts-flatcurve OR category=fts-tika) on message receipt, I see verbose logs for fts-flatcurve, as expected, but not a trace of output from fts-tika, in dovecot logs how to correctly turn on debug/verbose logging for fts-tika use in/by dovecot?

2 2

Many Thanks to the Programmers
by Paul Kudla (SCOM.CA Internet Services Inc.) 23 May '22

23 May '22

I am still in testing mode with 2.3.19 but the replication with the folders over 300+ seems to be fixed everything synced ok on its own within 30 minutes Again MUSH APPRECIATED ! Moving to 2.3.19 was woth it! -- Happy Monday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email paul(a)scom.ca

1 0

mail_replica in userdb?
by Coy Hile 23 May '22

23 May '22

I seem to have got replication working from an older instance to a newer one (insofar as the data are present on the new box) that I’m looking to stand up to replace a newer box. In this case, nothing is configured to talk to the new machine yet, so the replication is unidirectional (old -> new). However, I’m seeing an error like like this repeatedly in syslog: 2022-05-21T23:23:15.295252+00:00 basement-imap01 dovecot: doveadm(hile(a)coyhile.com): Error: sync: User has no mail_replica in userdb What setting did I miss? kdoveconf -n # 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7.2 () # OS: Linux 5.4.0-110-generic x86_64 Ubuntu 20.04.4 LTS # Hostname: basement-imap01.coyhile.com doveadm_password = # hidden, use -P to show it mail_location = maildir:~/Maildir mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocols = " imap lmtp" service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = vmail } unix_listener replication-notify { mode = 0666 user = vmail } } service doveadm { inet_listener { port = 12345 } user = vmail } service replicator { unix_listener replicator-doveadm { mode = 0666 } } ssl_cert = </etc/dovecot/private/dovecot.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap override_fields = uid=vmail gid=vmail home=/data/mail/vmail/%d/%n } Is that error something that I can (for the moment) ignore since I’m explicitly NOT trying to have this host replicate data anywhere, or do I need to tweak something? -- Coy Hile coy.hile(a)coyhile.com

3 7

SQL user_query quota_rule for *:messages=
by James 23 May '22

23 May '22

Hello, dovecot.conf sets the system default quota for storage and message count: plugin { ... quota_rule = *:storage=1G quota_rule2 = *:messages=10000 ... } This quota_rule for the storage is overridden by an SQL user_query in dovecot-sql.conf: user_query = "SELECT ... '*:storage=' || maxstorage || 'M' AS userdb_quota_rule, ...;" Is it possible to override the message count limit with an SQL user_query (and password_query)? I tried returning userdb_quota_rule2 in the SQL but it did not work. "SELECT ... '*:messages=' || maxcount AS userdb_quota_rule2 ...;" Ref: https://doc.dovecot.org/configuration_manual/authentication/user_database_e…

2 2

Dovecot 2.3.18 and OpenMetrics-Prometheus problem
by Christian Rößner 23 May '22

23 May '22

Hi, I have defined several metrics in Dovecot and activated the metrics service. After the I tried to include this in Prometheues, but I get strange errors. I gave up now after several hours in the hope that this might be a bug. So I ask for help here: Distribution: Rocky 8 Official Dovecot Repo Version: 2.3.18 Config for metrics, service and events: ----------------------------------------------------------------- metric auth_success { exporter = log filter = event=auth_request_finished AND success=yes } metric auth_failures { exporter = log filter = event=auth_request_finished AND NOT success=yes } metric auth_client_userdb { filter = event=auth_client_userdb_lookup_finished } metric auth_master { exporter = log filter = event=auth_master_client_login_finished } metric client_connected { filter = event=client_connection_connected } metric client_disconnected { filter = event=client_connection_disconnected group_by = reason } metric imap_command { filter = event=imap_command_finished group_by = cmd_name tagged_reply_state } metric lmtp_command { filter = event=smtp_server_command_finished AND protocol=lmtp group_by = cmd_name status_code duration:exponential:1:5:10 } metric imap_hibernate { filter = event=imap_client_hibernated group_by = error } metric imap_unhibernate { filter = event=imap_client_hibernated group_by = reason error } metric mail_delivery { filter = event=mail_delivery_finished group_by = duration:exponential:1:5:10 } metric sql_query { filter = event=sql_query_finished group_by = error_code } metric sieve_actions { filter = event=sieve_action_finished group_by = action_name error } metric managesieve { filter = event=managesieve_command_finished group_by = script_name } metric dict_lookups { filter = event=dict_lookup_finished group_by = driver error } service stats { inet_listener http { port = 9900 } } event_exporter log { format = json format_args = time-rfc3339 transport = log } ----------------------------------------------------------------- Shortened curl output: ----------------------------------------------------------------- # HELP process_start_time_seconds Timestamp of service start # TYPE process_start_time_seconds gauge process_start_time_seconds 1645743604 # HELP dovecot_build Dovecot build information # TYPE dovecot_build info dovecot_build_info{version="2.3.18",revision="9dd8408c18"} 1 # HELP dovecot_auth_success Total number of all events of this kind # TYPE dovecot_auth_success counter dovecot_auth_success_total 1669 # HELP dovecot_auth_success_duration_seconds Total duration of all events of this kind # TYPE dovecot_auth_success_duration_seconds counter dovecot_auth_success_duration_seconds_total 1.317570 # HELP dovecot_auth_failures Total number of all events of this kind # TYPE dovecot_auth_failures counter dovecot_auth_failures_total 1 # HELP dovecot_auth_failures_duration_seconds Total duration of all events of this kind # TYPE dovecot_auth_failures_duration_seconds counter ... # EOF ----------------------------------------------------------------- Prometheus yml config snippet: ----------------------------------------------------------------- global: scrape_interval: 15s evaluation_interval: 15s scrape_configs: - job_name: dovecot_exporter static_configs: - targets: - 172.30.10.1:9900 ----------------------------------------------------------------- Prometheus (Docker stable latest) with debug level show these lines: ----------------------------------------------------------------- prometheus_1 | ts=2022-02-25T10:22:00.840Z caller=scrape.go:1307 level=debug component="scrape manager" scrape_pool=dovecot_exporter target=http://172.30.10.1:9900/metrics msg="Append failed" err="expected label name or left brace, got \"INVALID\"" ----------------------------------------------------------------- Therefor the service is considered as being down. I really have no ide, what I can do here. Thanks for any help in advance Christian Rößner -- Rößner-Network-Solutions Zertifizierter ITSiBe / CISO Karl-Bröger-Str. 10, 36304 Alsfeld Fax: +49 6631 78823409, Mobil: +49 171 9905345 USt-IdNr.: DE225643613, https://roessner.website PGP fingerprint: 658D 1342 B762 F484 2DDF 1E88 38A5 4346 D727 94E5

4 9

Fix lmtp_save_to_detail_mailbox to be case insensitive?
by dovecot＠ptld.com 22 May '22

22 May '22

To bring this up again because i have not found a work around solution. Can dovecot make lmtp_save_to_detail_mailbox case-insensitive? If you create a folder "foobar" then an email with recipient user+foobar(a)example.com will be placed in the folder. However with the folder as "foobar" an email with recipient user+FOOBAR(a)example.com will report folder "FOOBAR" not found and the email will be placed in the inbox folder.

4 10

LDAP virtual user username being rewritten:
by Coy Hile 21 May '22

21 May '22

Hi all, I’m working on migrating a dovecot instance that formerly had its only user (me) stored in a SQL database to a new instance that uses LDAP so I don’t have to maintain a handful of different passwords. However, I’m ending up getting sent to an incorrect mailbox because the %d part of the username I enter is getting stripped somewhere along the lines. Dovecot debug logs show this happening here: May 21 18:36:53 auth: Debug: client in: AUTH 1 PLAIN service=imap secured session=itEO5InfPJusEgKD lip=172.18.2.131 rip=172.18.2.131 lport=143 rport=39740 resp=<hidden> May 21 18:36:53 auth: Debug: ldap(hile(a)coyhile.com,172.18.2.131,<itEO5InfPJusEgKD>): Performing passdb lookup May 21 18:36:53 auth: Debug: ldap(hile(a)coyhile.com,172.18.2.131,<itEO5InfPJusEgKD>): pass search: base=ou=People,dc=coyhile,dc=com scope=subtree filter=(&(objectClass=posixAccount)(uid=hile)) fields=uid,userPassword May 21 18:36:53 auth: Debug: ldap(hile(a)coyhile.com,172.18.2.131,<itEO5InfPJusEgKD>): result: uid=hile userPassword=<hidden>; uid,userPassword unused May 21 18:36:53 auth: Debug: ldap(hile(a)coyhile.com,172.18.2.131,<itEO5InfPJusEgKD>): username changed hile(a)coyhile.com -> hile May 21 18:36:53 auth: Debug: ldap(hile,172.18.2.131,<itEO5InfPJusEgKD>): Finished passdb lookup May 21 18:36:53 auth: Debug: auth(hile,172.18.2.131,<itEO5InfPJusEgKD>): Auth request finished May 21 18:36:53 auth: Debug: client passdb out: OK 1 user=hile original_user=hile(a)coyhile.com May 21 18:36:53 auth: Debug: master in: REQUEST 473956353 9409 1 2c1f94ded4de2c343425c90eeee8d094 session_pid=9412 request_auth_token May 21 18:36:53 auth: Debug: ldap(hile,172.18.2.131,<itEO5InfPJusEgKD>): Performing userdb lookup May 21 18:36:53 auth: Debug: ldap(hile,172.18.2.131,<itEO5InfPJusEgKD>): user search: base=ou=People,dc=coyhile,dc=com scope=subtree filter=(&(objectClass=posixAccount)(uid=hile)) fields=homeDirectory,uidNumber,gidNumber May 21 18:36:53 auth: Debug: ldap(hile,172.18.2.131,<itEO5InfPJusEgKD>): result: homeDirectory=/home/hile uidNumber=10000 gidNumber=10000; homeDirectory,uidNumber,gidNumber unused May 21 18:36:53 auth: Debug: ldap(hile,172.18.2.131,<itEO5InfPJusEgKD>): Finished userdb lookup May 21 18:36:53 auth: Debug: master userdb out: USER 473956353 hile home=/data/mail/vmail//hile uid=998 gid=998 auth_token=3a262da408d33ce2c51ecb1ddd943203fdbb17a3 auth_user=hile(a)coyhile.com Note the “username changed” line above. Clearly I’ve misconfigured something, butI am unsure what. My configuration is as follows: root@basement-imap01:/var/log# doveconf -n # 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7.2 () # OS: Linux 5.4.0-110-generic x86_64 Ubuntu 20.04.4 LTS # Hostname: basement-imap01.coyhile.com auth_debug = yes debug_log_path = /var/log/dovecot-debug.log doveadm_password = # hidden, use -P to show it mail_location = maildir:~/Maildir mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocols = " imap lmtp" service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = vmail } unix_listener replication-notify { mode = 0666 user = vmail } } service doveadm { inet_listener { port = 12345 } user = vmail } service replicator { unix_listener replicator-doveadm { mode = 0666 } } ssl_cert = </etc/dovecot/private/dovecot.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap override_fields = uid=vmail gid=vmail home=/data/mail/vmail/%d/%n } root@basement-imap01:/var/log# with the dovecot-ldap.conf.ext as follows: root@basement-imap01:/etc/dovecot# cat dovecot-ldap.conf.ext # This file is commonly accessed via passdb {} or userdb {} section in # conf.d/auth-ldap.conf.ext # This file is opened as root, so it should be owned by root and mode 0600. # # http://wiki2.dovecot.org/AuthDatabase/LDAP # # NOTE: If you're not using authentication binds, you'll need to give # dovecot-auth read access to userPassword field in the LDAP server. # With OpenLDAP this is done by modifying /etc/ldap/slapd.conf. There should # already be something like this: # access to attribute=userPassword # by dn="<dovecot's dn>" read # add this # by anonymous auth # by self write # by * none # Space separated list of LDAP hosts to use. host:port is allowed too. #hosts = hosts = ldap.coyhile.com # LDAP URIs to use. You can use this instead of hosts list. Note that this # setting isn't supported by all LDAP libraries. #uris = # Distinguished Name - the username used to login to the LDAP server. # Leave it commented out to bind anonymously (useful with auth_bind=yes). #dn = dn = uid=dovecotquery,ou=people,dc=coyhile,dc=com # Password for LDAP server, if dn is specified. #dnpass = dnpass = [REDACTED] # Use SASL binding instead of the simple binding. Note that this changes # ldap_version automatically to be 3 if it's lower. #sasl_bind = no # SASL mechanism name to use. #sasl_mech = # SASL realm to use. #sasl_realm = # SASL authorization ID, ie. the dnpass is for this "master user", but the # dn is still the logged in user. Normally you want to keep this empty. #sasl_authz_id = # Use TLS to connect to the LDAP server. #tls = no # TLS options, currently supported only with OpenLDAP: #tls_ca_cert_file = #tls_ca_cert_dir = #tls_cipher_suite = # TLS cert/key is used only if LDAP server requires a client certificate. #tls_cert_file = #tls_key_file = # Valid values: never, hard, demand, allow, try #tls_require_cert = # Use the given ldaprc path. #ldaprc_path = # LDAP library debug level as specified by LDAP_DEBUG_* in ldap_log.h. # -1 = everything. You may need to recompile OpenLDAP with debugging enabled # to get enough output. #debug_level = 0 # Use authentication binding for verifying password's validity. This works by # logging into LDAP server using the username and password given by client. # The pass_filter is used to find the DN for the user. Note that the pass_attrs # is still used, only the password field is ignored in it. Before doing any # search, the binding is switched back to the default DN. #auth_bind = no # If authentication binding is used, you can save one LDAP request per login # if users' DN can be specified with a common template. The template can use # the standard %variables (see user_filter). Note that you can't # use any pass_attrs if you use this setting. # # If you use this setting, it's a good idea to use a different # dovecot-ldap.conf.ext for userdb (it can even be a symlink, just as long as # the filename is different in userdb's args). That way one connection is used # only for LDAP binds and another connection is used for user lookups. # Otherwise the binding is changed to the default DN before each user lookup. # # For example: # auth_bind_userdn = cn=%u,ou=people,o=org # #auth_bind_userdn = # LDAP protocol version to use. Likely 2 or 3. #ldap_version = 3 # LDAP base. %variables can be used here. # For example: dc=mail, dc=example, dc=org base = ou=People,dc=coyhile,dc=com # Dereference: never, searching, finding, always #deref = never # Search scope: base, onelevel, subtree #scope = subtree scope = subtree # User attributes are given in LDAP-name=dovecot-internal-name list. The # internal names are: # uid - System UID # gid - System GID # home - Home directory # mail - Mail location # # There are also other special fields which can be returned, see # http://wiki2.dovecot.org/UserDatabase/ExtraFields #user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid # Filter for user lookup. Some variables can be used (see # http://wiki2.dovecot.org/Variables for full list): # %u - username # %n - user part in user@domain, same as %u if there's no domain # %d - domain part in user@domain, empty if user there's no domain #user_filter = (&(objectClass=posixAccount)(uid=%u)) user_filter = (&(objectClass=posixAccount)(uid=%u)) # Password checking attributes: # user: Virtual user name (user@domain), if you wish to change the # user-given username to something else # password: Password, may optionally start with {type}, eg. {crypt} # There are also other special fields which can be returned, see # http://wiki2.dovecot.org/PasswordDatabase/ExtraFields #pass_attrs = uid=user,userPassword=password # If you wish to avoid two LDAP lookups (passdb + userdb), you can use # userdb prefetch instead of userdb ldap in dovecot.conf. In that case you'll # also have to include user_attrs in pass_attrs field prefixed with "userdb_" # string. For example: #pass_attrs = uid=user,userPassword=password,\ # homeDirectory=userdb_home,uidNumber=userdb_uid,gidNumber=userdb_gid # Filter for password lookups #pass_filter = (&(objectClass=posixAccount)(uid=%u)) pass_filter = (&(objectClass=posixAccount)(uid=%n)) # Attributes and filter to get a list of all users #iterate_attrs = uid=user iterate_attrs = maildrop=user iterate_filter = (objectClass=posixAccount) # Default password scheme. "{scheme}" before password overrides this. # List of supported schemes is in: http://wiki2.dovecot.org/Authentication #default_pass_scheme = CRYPT # By default all LDAP lookups are performed by the auth master process. # If blocking=yes, auth worker processes are used to perform the lookups. # Each auth worker process creates its own LDAP connection so this can # increase parallelism. With blocking=no the auth master process can # keep 8 requests pipelined for the LDAP connection, while with blocking=yes # each connection has a maximum of 1 request running. For small systems the # blocking=no is sufficient and uses less resources. #blocking = no root@basement-imap01:/etc/dovecot# -- Coy Hile coy.hile(a)coyhile.com

1 1

Possible to increase connection limit for one user only?
by Kurt Fitzner 20 May '22

20 May '22

I run a medium-sized virtual domain email server for about 15 domains. I also use it myself and have 3 or 4 devices using imap. I am often using a VPN to connect to my home network, which makes all my connections appear to come from one IP. I want to increase mail_max_userip_connections just for myself only so I'm not opening up my server to abuse. Is this possible?

3 3