- dovecot - dovecot.org

doveadm backup -R -m -s not picking up a new message
by Daniel Shahaf 16 Feb '20

16 Feb '20

Good morning, I use «doveadm backup -R -m foo -s $state» to create a local mirror of an imapc remote folder. When I run that command, wait for it to finish, deliver a new message to the remote folder, and run that command again, the new message does not get downloaded from the remote. I'll attach the rawlog, doveconf, etc at the end. Allow me first to summarize what I've already checked: - I used «openssl s_client» to issue EXAMINE on the remote mailbox before and after delivering a mail to the remote mailbox, and HIGHESTMODSEQ does increase. - I looked at rawlog for both runs. doveadm issues a «STATUS foo (UIDNEXT UIDVALIDITY)» and receives a correct response (UIDNEXT in the second run is one larger than in the first run). - Other than that STATUS command, doveadm only issues LOGIN, LOGOUT, LIST, and LSUB commands. The remote server's responses do not report the new HIGHESTMODSEQ value for the folder. - The message does get mirrored locally when I run without the -s flag. I'm running dovecot 2.3.9.2 from FreeBSD ports. I've reviewed the FreeBSD port¹; it does have some patches, but none of them seem relevant to this issue. The remote server runs Cyrus. Debug output, etc follow: [[[ % doveconf -n # 2.3.9.2 (cf2918cac): /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 12.1-RELEASE amd64 # Hostname: imapmirror.v.shahaf.local2 disable_plaintext_auth = no first_valid_uid = 1000 imapc_features = " rfc822.size fetch-headers modseq" imapc_host = imap.fastmail.com imapc_password = # hidden, use -P to show it imapc_port = 993 imapc_ssl = imaps imapc_user = ... last_valid_uid = 65500 mail_location = maildir:~/Maildir:UTF-8 mail_prefetch_count = 99 namespace { inbox = yes location = prefix = separator = / } passdb { driver = pam } protocols = imap lmtp service imap { drop_priv_before_exec = yes } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl_cert = </usr/local/etc/dovecot/cert.pem ssl_key = # hidden, use -P to show it userdb { driver = passwd } protocol lmtp { auth_username_format = %n } ]]] [[[ # Baseline % rm -rf /home/fmsync/Maildir % rm -rf /home/fmsync/imapcdir % rm -rf /mnt/rawlog % mkdir /mnt/rawlog % mdmfs -s 1024m auto /mnt/rawlog # mount a tmpfs % chown fmsync /mnt/rawlog # doveadm -o mail_fsync=never backup -u fmsync -s "" -l 15 -R -m lists/zsh imapc:/home/fmsync/imapcdir | sponge state.out dsync(fmsync): Info: imapc(imap.fastmail.com:993): Connected to 66.111.4.135:993 (local 192.168.122.253:56226) # doveadm -o mail_fsync=never backup -u fmsync -s "$(cat state.out)" -l 15 -R -m lists/zsh imapc:/home/fmsync/imapcdir | sponge state.out dsync(fmsync): Info: imapc(imap.fastmail.com:993): Connected to 66.111.4.136:993 (local 192.168.122.253:39150) % openssl s_client -verify_return_error -connect imap.fastmail.com:imaps ⋮ b EXAMINE lists/zsh * 24117 EXISTS * 1 RECENT * FLAGS (\Answered \Flagged \Draft \Deleted \Seen $X-ME-Annot-2 $IsMailingList $IsNotification $HasAttachment $HasTD hasnoatt hasnotd hasatt $X-ME-Annot-1 Old $NotJunk $Junk $Forwarded) * OK [PERMANENTFLAGS ()] Ok * OK [UNSEEN 1759] Ok * OK [UIDVALIDITY 1529153257] Ok * OK [UIDNEXT 24289] Ok * OK [HIGHESTMODSEQ 4910711] Ok * OK [URLMECH INTERNAL] Ok * OK [ANNOTATIONS 65536] Ok b OK [READ-ONLY] Completed c CLOSE c OK Completed ]]] [[[ # In another terminal % swaks -t lists.zsh@… ⋮ <- 250 2.0.0 Ok: queued as 48CjYx6xvmzWp ⋮ ]]] [[[ # Back in s_client d EXAMINE lists/zsh * 24118 EXISTS * 2 RECENT * FLAGS (\Answered \Flagged \Draft \Deleted \Seen $X-ME-Annot-2 $IsMailingList $IsNotification $HasAttachment $HasTD hasnoatt hasnotd hasatt $X-ME-Annot-1 Old $NotJunk $Junk $Forwarded) * OK [PERMANENTFLAGS ()] Ok * OK [UNSEEN 1759] Ok * OK [UIDVALIDITY 1529153257] Ok * OK [UIDNEXT 24290] Ok * OK [HIGHESTMODSEQ 4910779] Ok * OK [URLMECH INTERNAL] Ok * OK [ANNOTATIONS 65536] Ok d OK [READ-ONLY] Completed e CLOSE e OK Completed ]]] [[[ # doveadm -o imapc_rawlog_dir=/mnt/rawlog -Dv backup -u fmsync -s "$(cat state.out)" -l 15 -R -m lists/zsh imapc:/home/fmsync/imapcdir | sponge state.out Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: Undefined symbol "acl_user_module" (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so: Undefined symbol "expire_set_init" (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so: Undefined symbol "quota_user_module" (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: Undefined symbol "fts_list_backend" (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: Undefined symbol "mail_crypt_user_get_public_key" (this is usually intentional, so just ignore this message) doveadm(fmsync)<966><>: Debug: auth-master: userdb lookup(fmsync): Started userdb lookup doveadm(fmsync)<966><>: Debug: auth-master: conn unix:/var/run/dovecot/auth-userdb: Connecting doveadm(fmsync)<966><>: Debug: auth-master: conn unix:/var/run/dovecot/auth-userdb (uid=0): Client connected (fd=10) doveadm(fmsync)<966><>: Debug: auth-master: userdb lookup(fmsync): auth USER input: fmsync system_groups_user=fmsync uid=1002 gid=1002 home=/home/fmsync doveadm(fmsync)<966><>: Debug: auth-master: userdb lookup(fmsync): Finished userdb lookup (username=fmsync system_groups_user=fmsync uid=1002 gid=1002 home=/home/fmsync) doveadm(fmsync): Debug: Effective uid=1002, gid=1002, home=/home/fmsync doveadm(fmsync): Debug: Namespace : type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir:UTF-8 doveadm(fmsync): Debug: maildir++: root=/home/fmsync/Maildir, index=, indexpvt=, control=, inbox=/home/fmsync/Maildir, alt= doveadm(fmsync): Debug: brain M: Imported mailbox states: doveadm(fmsync): Debug: brain M: Mailbox $guid1 state: uidvalidity=1529153257 uid=24288 modseq=4910711 pvt_modseq=0 messages=24117 changes_during_sync=0 doveadm(fmsync): Debug: brain M: Namespace has location maildir:~/Maildir:UTF-8 doveadm(fmsync): Debug: Namespace : Using permissions from /home/fmsync/Maildir: mode=0700 gid=default dsync(fmsync): Debug: Effective uid=1002, gid=1002, home=/home/fmsync dsync(fmsync): Debug: Namespace : type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=imapc:/home/fmsync/imapcdir dsync(fmsync): Debug: imapc(imap.fastmail.com:993): Created new connection dsync(fmsync): Debug: imapc(imap.fastmail.com:993): Looking up IP address (reconnect_ok=true, last_connect=1580957435) dsync(fmsync): Debug: imapc(imap.fastmail.com:993): Connecting to 66.111.4.135:993 dsync(fmsync): Info: imapc(imap.fastmail.com:993): Connected to 66.111.4.135:993 (local 192.168.122.253:59074) dsync(fmsync): Debug: imapc(imap.fastmail.com:993): Starting SSL handshake dsync(fmsync): Debug: imapc(imap.fastmail.com:993): SSL handshake successful dsync(fmsync): Debug: imapc(imap.fastmail.com:993): Server capabilities: IMAP4 IMAP4rev1 LITERAL+ ENABLE UIDPLUS SASL-IR NAMESPACE CONDSTORE SORT LIST-EXTENDED QRESYNC MOVE SPECIAL-USE CREATE-SPECIAL-USE IDLE AUTH=PLAIN dsync(fmsync): Debug: imapc(imap.fastmail.com:993): Authenticating as ... dsync(fmsync): Debug: imapc(imap.fastmail.com:993): Server capabilities: IMAP4rev1 LITERAL+ ID ENABLE ACL RIGHTS=kxten QUOTA MAILBOX-REFERRALS NAMESPACE UIDPLUS NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY CATENATE CONDSTORE ESEARCH SEARCH=FUZZY SORT SORT=MODSEQ SORT=DISPLAY SORT=UID THREAD=ORDEREDSUBJECT THREAD=REFERENCES THREAD=REFS ANNOTATE-EXPERIMENT-1 METADATA LIST-EXTENDED LIST-STATUS LIST-MYRIGHTS LIST-METADATA WITHIN QRESYNC SCAN XLIST XMOVE MOVE SPECIAL-USE CREATE-SPECIAL-USE DIGEST=SHA1 X-REPLICATION STATUS=SIZE OBJECTID SAVEDATE X-CREATEDMODSEQ PREVIEW=FUZZY XAPPLEPUSHSERVICE LOGINDISABLED XCONVERSATIONS COMPRESS=DEFLATE X-QUOTA=STORAGE X-QUOTA=MESSAGE X-QUOTA=X-ANNOTATION-STORAGE X-QUOTA=X-NUM-FOLDERS IDLE dsync(fmsync): Debug: imapc(imap.fastmail.com:993): Authenticated successfully dsync(fmsync): Debug: imapc: root=/home/fmsync/imapcdir, index=, indexpvt=, control=, inbox=, alt= dsync(fmsync): Debug: maildir++: root=/home/fmsync/imapcdir, index=, indexpvt=, control=, inbox=, alt= dsync(fmsync): Debug: brain M: Locking done locally in /home/fmsync/.dovecot-sync.lock (local hostname=imapmirror.v.shahaf.local2, remote hostname=imapmirror.v.shahaf.local2) dsync(fmsync): Debug: brain S: Namespace has location imapc:/home/fmsync/imapcdir dsync(fmsync): Debug: Namespace : Using permissions from /home/fmsync/imapcdir: mode=0700 gid=default dsync(fmsync): Debug: brain M: Local mailbox tree: lists guid=00000000000000000000000000000000 uid_validity=0 uid_next=0 subs=no last_change=0 last_subs=0 dsync(fmsync): Debug: brain S: Local mailbox tree: lists guid=00000000000000000000000000000000 uid_validity=0 uid_next=0 subs=no last_change=0 last_subs=0 dsync(fmsync): Debug: brain M: Local mailbox tree: lists/zsh guid=$guid1 uid_validity=1529153257 uid_next=24289 subs=yes last_change=0 last_subs=1580956711 dsync(fmsync): Debug: brain S: Local mailbox tree: lists/zsh guid=$guid1 uid_validity=1529153257 uid_next=24290 subs=yes last_change=0 last_subs=0 dsync(fmsync): Debug: brain M: Remote mailbox tree: lists guid=00000000000000000000000000000000 uid_validity=0 uid_next=0 subs=no last_change=0 last_subs=0 dsync(fmsync): Debug: brain M: Remote mailbox tree: lists/zsh guid=$guid1 uid_validity=1529153257 uid_next=24290 subs=yes last_change=0 last_subs=0 dsync(fmsync): Debug: brain S: Remote mailbox tree: lists guid=00000000000000000000000000000000 uid_validity=0 uid_next=0 subs=no last_change=0 last_subs=0 dsync(fmsync): Debug: brain S: Remote mailbox tree: lists/zsh guid=$guid1 uid_validity=1529153257 uid_next=24289 subs=yes last_change=0 last_subs=1580956711 dsync(fmsync): Debug: brain M: Mailbox lists: local=00000000000000000000000000000000/0/0, remote=00000000000000000000000000000000/0/0: Directory name paths are equal dsync(fmsync): Debug: brain M: Mailbox lists/zsh: local=$guid1/0/1, remote=$guid1/0/1: Mailboxes are equal dsync(fmsync): Debug: brain S: Mailbox lists: local=00000000000000000000000000000000/0/0, remote=00000000000000000000000000000000/0/0: Directory name paths are equal dsync(fmsync): Debug: brain S: Mailbox lists/zsh: local=$guid1/0/1, remote=$guid1/0/1: Mailboxes are equal dsync(fmsync): Debug: brain M: Skipping mailbox $guid1 with unchanged state uidvalidity=1529153257 uidnext=24289 highestmodseq=4910711 highestpvtmodseq=0 messages=24117 dsync(fmsync): Debug: imapc(imap.fastmail.com:993): Disconnected dsync(fmsync): Debug: brain M: Exported mailbox states: dsync(fmsync): Debug: brain M: Mailbox $guid1 state: uidvalidity=1529153257 uid=24288 modseq=4910711 pvt_modseq=0 messages=24117 changes_during_sync=0 dsync(fmsync): Debug: brain M: Unlocked /home/fmsync/.dovecot-sync.lock dsync(fmsync): Debug: auth-master: conn unix:/var/run/dovecot/auth-userdb (uid=0): Disconnected: Connection closed (fd=10) % grep -v LOGIN < /mnt/rawlog/20200206-025036.966.1.out 1580957437.093905 1 CAPABILITY 1580957437.403048 3 LIST "" "" 1580957437.550179 4 LIST "" "*" 1580957437.725211 5 STATUS "lists/zsh" (UIDNEXT UIDVALIDITY) 1580957437.874174 6 LSUB "" "*" 1580957438.184244 7 LOGOUT % grep -w -B 2 5 < /mnt/rawlog/20200206-025036.966.1.in 1580957437.705302 4 OK Completed (...) 1580957437.869540 * STATUS lists/zsh (UIDNEXT 24290 UIDVALIDITY 1529153257) 1580957437.869540 5 OK Completed % doveadm search -u fmsync mailbox lists/zsh HEADER Received 48CjYx6xvmzWp | wc -l 0 % ]]] [[[ # Back in s_client: f EXAMINE lists/zsh ⋮ g UID SEARCH HEADER Received 48CjYx6xvmzWp * SEARCH 24289 g OK Completed (1 msgs in 5.268 secs) ]]] So, my question is: Why doesn't «doveadm backup» sync the new message? (Message number 24118, UID 24289, ESMTP ID 48CjYx6xvmzWp) Cheers, Daniel ¹ https://svn.freebsd.org/ports/head/mail/dovecot

1 2

.IMAP
by Jos Chrispijn 15 Feb '20

15 Feb '20

Dear team, I have my users in /usr/home/user1 for user1, user2 for user2 etc. As these users also have a website, these folders have a /usr/home/user1/www folder, which is the docroot of their website. What happens now is that Dovecot indexes also this /usr/home/user1/www folder and creates in every folder a .imap folder which content is an exact image of the folder it is created in. In practice this causes 2 times of webspace (which also in on the account of that user). Can you pls tell me what I do wrong and how to solve this? I might be missing an 'exclude' folder setting for each dovecot user. Thanks for you expertise, Jos

3 4

Dovecot 2.3 repo for CentOS 8.
by Reio Remma 15 Feb '20

15 Feb '20

Hello! Are there any plans for an official Dovecot repo for CentOS 8? Thanks, Reio

7 10

Very slow mail download/notification with dovecot 2.2.33 and Thunderbird
by ml_dovecot＠thorsten-reichelt.de 15 Feb '20

15 Feb '20

Hi! I know that there are many results if I search for "dovecot thunderbird very slow" on Google but none of them helped me with my problem. :( I am running Ubuntu 18.04 (4GB RAM, 40GB HDD with EXT4) with the following software: - Dovecot 2.2.33 ---- All accounts are using Maildir ---- Sieve filters - Postfix 3.3.0 ---- with opendkim, spamassassin, razor, pyzor, .... And as IMAP client I am using Thunderbird 68.4 on Windows10. Sieve is used to sort incoming mails into my 224 folders (56 INBOX. folders and its subfolders (output of "find .INBOX.* -maxdepth 4 -type d -print |wc -l")). My problem is that it takes very very long to synchonize the INBOX and its subfolders. Even if there are only a few new messages to be downloaded. Sometimes Thunderbird outputs something like "downloading message 1 of 19" and it takes at least 60 seconds to download all of them. None of these 19 mails is larger than 5kB. And sometimes it seems that the transfer is suspended for a long time and it takes >5 Minutes to synchronize only a few mails. :( Every helpful advice is welcome. I just don't know where to look for the cause right now, but I can provide configurations (like output of |doveadm config) |and some logfiles. Thank you very much! Thorsten

2 1

Dovecot Proxy - Oauth2 mech add custom fields
by Domenico Pastore 14 Feb '20

14 Feb '20

Hi, I have a problem with configuring dovecot passdb for Oauth2 with keyclock. A user can access more mailbox, mailboxes are associated with the user. When a user login with this method: OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ AUTH=PLAIN] Dovecot ready. a login mailbox*user password Dovecot when requiring the grant_url send to Keyclock, for example, this post (I have already enabled raw_log for analysis): grant_type=password&username=domenico&password=test&client_id=imap-client&client_secret=99e26b26-0f2a-4b64-8f57-c0ca2147d3a0&scope=emailPOST /auth/realms/example/protocol/openid-connect/token/introspect The call pass to Keyclock only master_user and miss mailbox info. In fact, the JSON response after login return the only username without mailbox: [...] "scope": "profile email", "email_verified": false, "preferred_username": "dome.nico" [...] When Dovecot proxy connects to the backend, email attribute and user have the same value, master-user. This behavior is a problem because when backend tries login access, login with the user and not with the mailbox. This is backend logging: 2020-02-13 19:34:13 auth: Debug: client passdb out: OK 1 user=domenico token=eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJYVy1fSmNnVkF3aW9GUXh1NUhwdjVlbk5uNU8zaW42Y1VpaGJsM2dWX0V3In0.eyJqdGkiOiJhYTMwZ Dk0Yy0xNjE0LTQzN2QtOTA5Zi01ZTAwNGQ2YjNmZTIiLCJleHAiOjE1ODE2MTE5NTQsIm5iZiI6MCwiaWF0IjoxNTgxNjExNjU0LCJpc3MiOiJodHRwczovL2tleWNsb2FrLXBlYy1pYW0ucGVjLWFwcHMucGFyLXRlYy5pdC9hdXRoL3JlYWxtcy9wZWMiLCJhdWQiOiJhY2NvdW50Iiwi c3ViIjoiZjphNTA1NWUzMi1lYzhkLTRmZjgtOWZjNS00ODM4MmQ1MzRhODc6ZG9tZS5uaWNvIiwidHlwIjoiQmVhcmVyIiwiYXpwIjoiaW1hcC1jbGllbnQiLCJhdXRoX3RpbWUiOjAsInNlc3Npb25fc3RhdGUiOiIyN2M0ZDMzYy01YjdlLTQzMWMtYjZmMi0yYmI4NjIzYzMyMjkiLCJ hY3IiOiIxIiwicmVhbG1fYWNjZXNzIjp7InJvbGVzIjpbIm9mZmxpbmVfYWNjZXNzIiwidW1hX2F1dGhvcml6YXRpb24iXX0sInJlc291cmNlX2FjY2VzcyI6eyJhY2NvdW50Ijp7InJvbGVzIjpbIm1hbmFnZS1hY2NvdW50IiwibWFuYWdlLWFjY291bnQtbGlua3MiLCJ2aWV3LXByb2 ZpbGUiXX19LCJzY29wZSI6InByb2ZpbGUgZW1haWwiLCJlbWFpbF92ZXJpZmllZCI6ZmFsc2UsInByZWZlcnJlZF91c2VybmFtZSI6ImRvbWUubmljbyJ9.LlIx-QeRQPr3lK4Cs1vU0qMvHF3uq3h15BGi1atNCBASkM6oPoYWLV-sYdf8hzpRFyOaTcbxN53SN6LfD0hHvUZ2sKHxh7UJ idmxS4hf1SsZq8wJTASpebcPLtBIX5JBvXmpxa-cVnZDE1JVw5np5-LLNs0j4sgHwgg85mJEoE2VmYJzbGZjUsSTvaAAoCbvTA0MfsNoKyq0E5JrLVdkI-twX7HjAESFqFD4yHe7BS4FG_UjddrSr3uXmXreB44VLZ8B4xBgVRjK9K-sjjkXT8Bkv8WbxUdEEHaarWU_qanI5DlhA0CZXlJ CyDsNcRwQfwVHOESxXE7ehgIDPm-NjA I have a mechanism for adding other attributes with Dovecot when calling Keyclock? This for insert email or other fields into the token. Thanks all, Domenico ——— Dovecot Frontend # 2.3.9.2 (cf2918cac): /config/dovecot/dovecot-proxy/dovecot.conf # OS: Linux 3.10.0-693.17.1.el7.x86_64 x86_64 CentOS Linux release 7.4.1708 (Core) # Hostname: fe-new.example.it auth_debug = yes auth_debug_passwords = yes auth_master_user_separator = * auth_verbose = yes auth_verbose_passwords = yes base_dir = /data/dovecot/var/run/dovecot-proxy default_vsz_limit = 768 M disable_plaintext_auth = no first_valid_gid = 101 first_valid_uid = 102 imap_id_send = import_environment = TZ MASTERPWD info_log_path = /LOGS/imap/dovecot-proxy.log instance_name = dovecot-proxy listen = fe-new_imap log_path = /LOGS/imap/dovecot-proxy.log log_timestamp = "%Y-%m-%d %H:%M:%S " mail_gid = 101 mail_location = maildir:%h/Maildir mail_max_userip_connections = 50 mail_plugins = quota expire mail_log notify mail_uid = 102 maildir_broken_filename_sizes = yes mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX. separator = . subscriptions = yes type = private } passdb { args = /config/dovecot/dovecot-proxy/dovecot-oauth2.conf driver = oauth2 master = yes mechanisms = plain login } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size } postmaster_address = posta(a)foo.it protocols = imap pop3 service anvil { client_limit = 3000 } service auth { client_limit = 4096 unix_listener auth-userdb { mode = 0600 } } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { port = 993 ssl = yes } process_limit = 2500 process_min_avail = 5 } service imap { drop_priv_before_exec = yes process_limit = 2500 process_min_avail = 5 } service lmtp { inet_listener lmtp { port = 24 } } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } service managesieve { drop_priv_before_exec = yes process_limit = 1024 } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { port = 995 ssl = yes } process_limit = 300 process_min_avail = 5 } service pop3 { drop_priv_before_exec = yes process_limit = 300 process_min_avail = 5 } ssl_cert = </certs/cert-selfsigned.crt ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL:!RC4::!3DES:!IDEA ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it protocol lmtp { mail_plugins = quota expire mail_log notify } protocol lda { mail_plugins = quota expire mail_log notify } protocol imap { mail_plugins = quota imap_quota mail_log notify } protocol pop3 { mail_plugins = quota mail_log notify pop3_uidl_format = UID%u-%v } -> /config/dovecot/dovecot-proxy/dovecot-oauth2.conf grant_url = https://keycloak-iam.apps.example.com/auth/realms/example/protocol/openid-c… use_grant_password = yes introspection_mode = post introspection_url = https://keycloak-iam.apps.example.com/auth/realms/example/protocol/openid-c… username_attribute = username username_format = %Lu tls_ca_cert_file = /etc/ssl/certs/ca-certificates.crt active_attribute = active active_value = true scope = email send_auth_headers = yes debug = yes rawlog_dir = /LOGS/imap/oauth2/ client_id = imap-client client_secret = 99e26b26-0f2a-4b64-8f57-c0ca2147d3a0 pass_attrs = host=192.160.10.4 proxy=y proxy_mech=xoauth2 pass=%{oauth2:access_token}

1 0

Dovecot process died with assertion failed
by TACHIBANA Masashi 13 Feb '20

13 Feb '20

Hi, I'm testing Dovecot v2.3.9.2. So, I found a problem that a Dovecot process termed with Panic, like below: Feb 10 08:50:09 imap(user1(a)example.com)<38440><p9ec0zSeSIQKEAIK>: Panic: file message-snippet.c: line 71 (snippet_add_content): assertion failed: (*count_r <= size) Feb 10 08:50:09 imap(user1(a)example.com)<38440><p9ec0zSeSIQKEAIK>: Error: Raw backtrace: /usr/local/dovecot-2.3.9.2/lib/dovecot/libdovecot.so.0(backtrace_append+0x3b) [0x7fee4c46331b] -> /usr/local/dovecot-2.3.9.2/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7fee4c46346e] -> /usr/local/dovecot-2.3.9.2/lib/dovecot/libdovecot.so.0(+0xfc90b) [0x7fee4c46f90b] -> /usr/local/dovecot-2.3.9.2/lib/dovecot/libdovecot.so.0(+0xfc9c1) [0x7fee4c46f9c1] -> /usr/local/dovecot-2.3.9.2/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fee4c3b470a] -> /usr/local/dovecot-2.3.9.2/lib/dovecot/libdovecot.so.0(+0xdea53) [0x7fee4c451a53] -> /usr/local/dovecot-2.3.9.2/lib/dovecot/libdovecot-storage.so.0(+0xdcd98) [0x7fee4c80bd98] -> /usr/local/dovecot-2.3.9.2/lib/dovecot/libdovecot-storage.so.0(+0xdd1a5) [0x7fee4c80c1a5] -> /usr/local/dovecot-2.3.9.2/lib/dovecot/libdovecot-storage.so.0(index_mail_get_special+0x2c1) [0x7fee4c80c591] -> /usr/local/dovecot-2.3.9.2/lib/dovecot/libdovecot-storage.so.0(dbox_mail_get_special+0x48) [0x7fee4c7adeb8] -> /usr/local/dovecot-2.3.9.2/lib/dovecot/libdovecot-storage.so.0(+0x78f2b) [0x7fee4c7a7f2b] -> /usr/local/dovecot-2.3.9.2/lib/dovecot/libdovecot-storage.so.0(mail_get_special+0x10) [0x7fee4c76f3e0] -> dovecot-backend-server/imap [user1(a)example.com 10.16.2.10 UID FETCH](+0x2645b) [0x55ffc6c5845b] -> dovecot-backend-server/imap [user1(a)example.com 10.16.2.10 UID FETCH](+0x239df) [0x55ffc6c559df] -> dovecot-backend-server/imap [user1(a)example.com 10.16.2.10 UID FETCH](imap_fetch_more+0x35) [0x55ffc6c578b5] -> dovecot-backend-server/imap [user1(a)example.com 10.16.2.10 UID FETCH](cmd_fetch+0x35f) [0x55ffc6c44aaf] -> dovecot-backend-server/imap [user1(a)example.com 10.16.2.10 UID FETCH](command_exec+0xb9) [0x55ffc6c535b9] -> dovecot-backend-server/imap [user1(a)example.com 10.16.2.10 UID FETCH](+0x1eacf) [0x55ffc6c50acf] -> dovecot-backend-server/imap [user1(a)example.com 10.16.2.10 UID FETCH](+0x1eb87) [0x55ffc6c50b87] -> dovecot-backend-server/imap [user1(a)example.com 10.16.2.10 UID FETCH](client_handle_input+0x215) [0x55ffc6c51885] -> dovecot-backend-server/imap [user1(a)example.com 10.16.2.10 UID FETCH](client_input+0x7e) [0x55ffc6c5206e] -> /usr/local/dovecot-2.3.9.2/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x7b) [0x7fee4c48bcdb] -> /usr/local/dovecot-2.3.9.2/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x105) [0x7fee4c48dd55] -> /usr/local/dovecot-2.3.9.2/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x59) [0x7fee4c48bde9] -> /usr/local/dovecot-2.3.9.2/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fee4c48c098] -> /usr/local/dovecot-2.3.9.2/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fee4c3e9623] -> dovecot-backend-server/imap [user1(a)example.com 10.16.2.10 UID FETCH](main+0x367) [0x55ffc6c40d37] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7fee4bfc7545] -> dovecot-backend-server/imap [user1(a)example.com 10.16.2.10 UID FETCH](+0xefb4) [0x55ffc6c40fb4] Feb 10 08:50:09 imap(user1(a)example.com)<38440><p9ec0zSeSIQKEAIK>: Fatal: master: service(imap): child 38440 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) It was occurred this by a mail had ISO-2022-JP content (Single part mail). I saw that the mail was not strange. When I appended the mail with IMAP APPEND command, that no problem happened. I think the problem comes by LMTP only. Any idea? Regards, -- Tachibana, Masashi QUALITIA CO., LTD. mailto:tachibana@qualitia.co.jp https://www.qualitia.co.jp/

1 1

Shared Mailboxes with Multiple Domains
by Daniel Miller 13 Feb '20

13 Feb '20

Trying to track down a problem I've been dealing with for a while. Everything else works fine - the problem is with shared mailboxes. My present, and desired, prefix for the shared namespace is: prefix = INBOX/shared/%%d/%%n/ Some mail clients, particularly Thunderbird and Android's AquaMail, have no problem with this. But other (presumably broken) clients don't show the shared mailboxes. This includes EM Client and Webmail Lite. Actually, Webmail Lite lists the mailboxes in the subscription window, but then the "live" folder list shows "shared" and "shared/domain" but none of the shared mailboxes below the domain. Changing to: prefix = INBOX/shared/%%u/ Works across all clients - but I'd rather have the domain separation. Testing with telnet ". LIST '' '*'" yields the full list with either config. The files /var/mail/%d/shared-mailboxes contain entries like: shared/shared-boxes/group/allshared/user(a)domain.com 1 Below is "doveconf -n" output. # 2.3.9.3 (9f41b88fa): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.5.9 (db4e9a2f) # OS: Linux 5.3.0-28-generic x86_64 Ubuntu 18.04.4 LTS # Hostname: bubba.amfes.lan auth_cache_size = 4 k auth_master_user_separator = * auth_mechanisms = plain login auth_policy_hash_nonce = # hidden, use -P to show it auth_policy_hash_truncate = 8 auth_policy_server_api_header = Authorization: Basic d2ZvcmNlOnVsdHJhLXNlY3JldC1zZWN1cmUtc2FmZQ default_login_user = nobody default_vsz_limit = 2 G disable_plaintext_auth = no imap_client_workarounds = tb-extra-mailbox-sep imap_idle_notify_interval = 29 mins listen = * login_trusted_networks = 192.168.0.0/24 mail_attachment_hash = %{sha512} mail_plugins = fts fts_solr acl zlib virtual mail_prefetch_count = 10 mail_shared_explicit_inbox = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mdbox_rotate_size = 20 M namespace archives { list = children location = mdbox:/var/mail/%d/%n/Archives/mdbox mailbox Unsorted { auto = no special_use = \Archive } prefix = INBOX/Archives/ separator = / subscriptions = no type = private } namespace inbox { hidden = no inbox = yes list = yes location = mailbox "Deleted Messages" { auto = no autoexpunge = 30 days special_use = \Trash } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Items" { auto = no special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Trash { auto = subscribe autoexpunge = 30 days special_use = \Trash } prefix = INBOX/ separator = / subscriptions = no type = private } namespace lists { list = children location = mdbox:/var/mail/%d/%n/Lists/mdbox prefix = INBOX/Lists/ separator = / subscriptions = no type = private } namespace subscriptions { hidden = yes list = no location = prefix = subscriptions = yes } namespace usershares { list = children location = sdbox:/var/mail/%%d/%%n/sdbox:NO-NOSELECT prefix = INBOX/shared/%%d/%%n/ separator = / subscriptions = no type = shared } namespace virtual { list = children location = virtual:/var/mail/%d/%n/virtual mailbox Flagged { comment = All my flagged messages special_use = \Flagged } prefix = INBOX/virtual/ separator = / subscriptions = no } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile acl_shared_dict = file:/var/mail/%d/shared-mailboxes fts = solr fts_autoindex = yes fts_autoindex_exclude = \Trash fts_autoindex_exclude2 = \Junk fts_autoindex_exclude3 = \Spam fts_enforced = no fts_index_timeout = 20s fts_solr = url=http://127.0.0.1:8983/solr/dovecot/ mailbox_alias_new = Sent Messages mailbox_alias_new2 = Sent Items mailbox_alias_new3 = Deleted Messages mailbox_alias_old = Sent mailbox_alias_old2 = Sent mailbox_alias_old3 = Trash sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = mail mode = 0600 user = vmail } } service dict { unix_listener dict { group = mail mode = 0660 user = vmail } } service imap-login { process_min_avail = 10 service_count = 1 } service imap-postlogin { executable = script-login /usr/local/etc/dovecot/post-login.sh user = $default_internal_user } service imap { executable = imap imap-postlogin vsz_limit = 4 G } service indexer-worker { process_limit = 3 } service lmtp { process_min_avail = 5 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = mail mode = 0666 user = vmail } } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } process_min_avail = 0 service_count = 1 } ssl_cert = </etc/ssl/letsencrypt/live/mail.amfes.com/fullchain.pem ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { driver = prefetch } userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol lmtp { mail_plugins = fts fts_solr acl zlib virtual sieve postmaster_address = postmaster(a)amfes.com } protocol lda { mail_plugins = fts fts_solr acl zlib virtual sieve } protocol imap { mail_max_userip_connections = 50 mail_plugins = fts fts_solr acl zlib virtual imap_acl imap_zlib mailbox_alias } -- Daniel

1 0

[FOR THE RECORD] CLIENTID Extension for IMAP/SMTP AUTH in dovecot
by Michael Peddemors 13 Feb '20

13 Feb '20

More and more email clients are now starting to support the CLIENTID extension for IMAP, including 'libetpan', SaneBox, BlueMail, and now as of Thunderbird 68.5.0, available there as well. This allows for a form of native two factor support, permission token, and/or access control to prevent the unauthorized use of email address and password. (For more information, please see the RFC Drafts, or reach out to us off list for details) While the version of dovecot we use has supported it for some time now, we would like to make it available for more email providers who use dovecot. For the record, for those interested in having your dovecot support transparent multi-factor, in your implementation, you will have to compile it with the patch listed in: https://github.com/dovecot/core/pull/86 Once that is done, please feel free to reach out to us for the dovecot plugin, that enables CLIENTID to be supported in the IMAP protocol. -- "Catch the Magic of Linux..." ------------------------------------------------------------------------ Michael Peddemors, President/CEO LinuxMagic Inc. Visit us at http://www.linuxmagic.com @linuxmagic A Wizard IT Company - For More Info http://www.wizard.ca "LinuxMagic" a Registered TradeMark of Wizard Tower TechnoServices Ltd. ------------------------------------------------------------------------ 604-682-0300 Beautiful British Columbia, Canada This email and any electronic data contained are confidential and intended solely for the use of the individual or entity to which they are addressed. Please note that any views or opinions presented in this email are solely those of the author and are not intended to represent those of the company.

1 0

Doveadm Sync problem with symbolic folders
by Gianluca Scaglia 12 Feb '20

12 Feb '20

Hi, i have two servers (debian 10) that synchronize the emails bidirectionally with Doveadm Sync (dovecot 2.3.4.1). Before updating to debian 10, then with dovecot 2.2, I could also synchronize symbolic folders (made with ln-s) without any problems. Now, however, it continues to create temporary folders (e.g. Sent-temp-1 etc.) and I am forced to stop synchronization. Is there any way to make Doveadm Sync ignore symbolic folders? Thanks a lot, Luca Scaglia

1 0

feature request for setting alternative pidfile
by Bjoern Jacke 12 Feb '20

12 Feb '20

Hi, because of an unsupported combination of configuration parameters for different dovecot services I looked into setting up two dovecot instances with different configurations on the same host. It looks like running two different dovecot instances on the same host is not easily possible because the pidfile seems to be hard-coded and there is no way to tell dovecot to use a different one, right? It would be great if this could be made customizable. Björn

2 2