- dovecot - dovecot.org

"panic ... assertion failed" with virtual and flatcurve fts together
by James Cook 17 Jul '24

17 Jul '24

Hi, I'm getting the following error: $ doveadm fetch 'mailbox storageid' guid '1721115116.M995635P1988.exoco.falsifian.org,S=28763,W=29300' Error: net_connect_unix(/var/dovecot/stats-writer) failed: Permission denied doveadm(falsifian): Panic: file index-search-result.c: line 174 (index_search_result_update_appends): assertion failed: (result->search_args->args == &search_arg) When I remove "virtual" from my "plugins" configuration line and remove the virtual namespace, the search works: $ doveadm fetch 'mailbox storageid' guid '1721115116.M995635P1988.exoco.falsifian.org,S=28763,W=29300' Error: net_connect_unix(/var/dovecot/stats-writer) failed: Permission denied mailbox: Spam storageid: 1721115116.M995635P1988.exoco.falsifian.org,S=28763,W=29300 It also works if I remove fts and fts_flatcurve from my "plugins" configuration line, leaving "virtual" alone. I first saw this when I enabled fts (with fts_flatcurve): I saw a lot of such errors in the logs when I attempted full text searches via neomutt. The errors seemed to stop happening after the indexing jobs finally finished, but just today I saw it again with the above query unrelated to fts. I have been using virtual mailboxes for a long time and hadn't noticed the error before trying fts. $ dovecot --version 2.3.21 (47349e2482) Output of doveconf -n (full config that triggers the error) follows signature. I saw something similar was reported in January [0] (that gave me the idea to try turning off virtual). Please let me know what I can do to help debug. It's just my personal mail server so I might be able to do things like run a debug build if necessary (assuming I find the time). [0] https://dovecot.org/mailman3/archives/list/dovecot@dovecot.org/thread/Z2HTM… -- James # 2.3.21 (47349e2482): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.21 (f6cd4b8e) # OS: OpenBSD 7.5 amd64 ffs # Hostname: exoco.falsifian.org first_valid_uid = 1000 mail_location = maildir:/home/%u/Maildir mail_plugins = " fts fts_flatcurve virtual" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mbox_write_locks = fcntl mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } namespace virtual { location = virtual:/mail/%u/virtual prefix = virtual/ separator = / } passdb { args = /etc/dovecot/passdb driver = passwd-file } plugin { fts = flatcurve fts_filters = lowercase fts_flatcurve_substring_search = yes fts_languages = en fts_tokenizers = generic email-address } protocols = imap ssl_cert = </etc/ssl/falsifian.org.fullchain.pem ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/userdb driver = passwd-file } protocol lda { mail_plugins = " fts fts_flatcurve virtual sieve sieve" }

1 0

Debian Bookworm packages, please !
by Laura Smith 17 Jul '24

17 Jul '24

Debian Bookworm (12) was released June 2023. It is therefore somewhat disappointing to see no Bookworm packages in https://repo.dovecot.org/ce-2.3-latest/debian/

16 37

dictionary with %d variable in sql stops insert and select domain part - dovecot 2.3.17+
by Mariusz Piasecki 16 Jul '24

16 Jul '24

Hi, I notice that quota domain when using args %d in dictionary isn't honored in mysql during updates and selects when using dovecot 2.3.17+, so I moved my tests to docker environment, because I started using alpine linux and thought it could be musl library problem. I tested few versions with debian/alpine and after 2.3.17 updates and selects for domain in sql not using domain part, but user@domain. Maybe I missed some changes in configuration, maybe someone experience this behavior? User in logs was changed to h.user(a)example.com. Configuration as described in documentation quota = dict:<quota root name>:<username>[:<option>[...]]:<dictionary URI> where username could be %d for supporting domain-wide quotas quota = dict:user_quota::proxy::sqluserquota quota2 = dict:domain_quota:%d:proxy::sqldomainquota dict { acl = mysql:/etc/dovecot/dovecot-share-folder.conf sqldomainquota = mysql:/etc/dovecot/dovecot-dict-sql-domain.conf sqluserquota = mysql:/etc/dovecot/dovecot-dict-sql-user.conf } dovecot-dict-sql-domain.conf map { pattern= priv/quota/storage table= used_domain_quota username_field= domain value_field= bytes } map { pattern= priv/quota/messages table= used_domain_quota username_field= domain value_field= messages } dovecot-dict-sql-user.conf map { pattern= priv/quota/storage table= used_quota username_field= username value_field= bytes } map { pattern= priv/quota/messages table= used_quota username_field= username value_field= messages } Example dump sql table, in real environment with newer dovecot, there was only h.user(a)example.com and no domain record. *In earlier versions of dovecot there was only domain in this table.* |//**domain**//|varchar(255)|Nie| |bytes|bigint(20)|Nie|0 |messages|bigint(20)|Nie|0 == Dumping data for table used_domain_quota |h.user(a)example.com|36000|36000 |example.com|35000|35000 Debug Doveadm *In both examples below debug args=example.com seems correct * Dovecot version 2.3.17+, in this example debian 10 2.3.17 (e2aa53df5b), select with only username. root@cf801690360c:/# doveadm -D -v quota get -u h.user(a)example.com Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message) Nov 27 15:46:31 Debug: Loading modules from directory: /usr/lib/dovecot/modules Nov 27 15:46:31 Debug: Module loaded: /usr/lib/dovecot/modules/lib01_acl_plugin.so Nov 27 15:46:31 Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Nov 27 15:46:31 Debug: Module loaded: /usr/lib/dovecot/modules/lib20_mailbox_alias_plugin.so Nov 27 15:46:31 Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm Nov 27 15:46:31 Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so Nov 27 15:46:31 Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so Nov 27 15:46:31 Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message) Nov 27 15:46:31 Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message) Nov 27 15:46:31 Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message) Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><>: Debug: auth-master: userdb lookup(h.user(a)example.com): Started userdb lookup Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><>: Debug: auth-master: conn unix:/var/run/dovecot/director-userdb: Connecting Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><>: Debug: auth-master: conn unix:/var/run/dovecot/director-userdb (pid=1,uid=0): Client connected (fd=9) Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><>: Debug: auth-master: userdb lookup(h.user(a)example.com): auth USER input: h.user(a)example.com home=/home/mail/virtual/data05/example.com/h.user@example.com quota2_rule=*:bytes=10485760000 quota_rule=*:bytes=1048576000 Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><>: Debug: auth-master: userdb lookup(h.user(a)example.com): Finished userdb lookup (username=h.user(a)example.com home=/home/mail/virtual/data05/example.com/h.user@example.com quota2_rule=*:bytes=10485760000 quota_rule=*:bytes=1048576000) Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><>: Debug: Added userdb setting: plugin/quota2_rule=*:bytes=10485760000 Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><>: Debug: Added userdb setting: plugin/quota_rule=*:bytes=1048576000 Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Effective uid=101, gid=101, home=/home/mail/virtual/data05/example.com/h.user@example.com Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Home dir not found: /home/mail/virtual/data05/example.com/h.user(a)example.com Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: dict(proxy): dict created (uri=proxy::acl, base_dir=/var/run/dovecot) Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Quota root: name=User Quota backend=dict args=:proxy::sqluserquota Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Quota rule: root=User Quota mailbox=* bytes=1048576000 messages=0 Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Quota warning: bytes=1048576000 (100%) messages=0 reverse=no command=quota-warning 100 h.user(a)example.com Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Quota warning: bytes=996147200 (95%) messages=0 reverse=no command=quota-warning 95 h.user(a)example.com Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Quota warning: bytes=943718400 (90%) messages=0 reverse=no command=quota-warning 90 h.user(a)example.com Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Quota warning: bytes=891289600 (85%) messages=0 reverse=no command=quota-warning 85 h.user(a)example.com Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Quota grace: root=User Quota bytes=104857600 (10%) Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Quota root: name=Domain Quota backend=dict args=example.com:proxy::sqldomainquota Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Quota rule: root=Domain Quota mailbox=* bytes=10485760000 messages=0 Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Quota grace: root=Domain Quota bytes=1048576000 (10%) Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: quota-dict: user=h.user(a)example.com, uri=proxy::sqluserquota, noenforcing=0 Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: dict(proxy): dict created (uri=proxy::sqluserquota, base_dir=/var/run/dovecot) Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: quota-dict: user=example.com, uri=proxy::sqldomainquota, noenforcing=0 Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: dict(proxy): dict created (uri=proxy::sqldomainquota, base_dir=/var/run/dovecot) Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Namespace : type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir://home/mail/virtual/data05/example.com/h.user@example.com/… Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: maildir++: root=//home/mail/virtual/data05/example.com/h.user@example.com/Maildir, index=, indexpvt=, control=, inbox=//home/mail/virtual/data05/example.com/h.user@example.com/Maildir, alt= Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: acl: initializing backend with data: vfile Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: acl: acl username= h.user(a)example.com Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: acl: owner= 1 Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: acl vfile: Global ACLs disabled Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Namespace : type=shared, prefix=Shared/%u/, sep=/, inbox=no, hidden=no, list=children, subscriptions=yes location=maildir:%Lh/Maildir/:INDEX=%Lh/Maildir/Shared/%u Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: shared: root=/var/run/dovecot, index=, indexpvt=, control=, inbox=, alt= Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: acl: initializing backend with data: vfile Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: acl: acl username= h.user(a)example.com Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: acl: owner= 0 Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: acl vfile: Global ACLs disabled Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: quota: quota_over_flag check: quota_over_script unset - skipping Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: quota: quota_over_flag check: quota2_over_script unset - skipping Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Looking up 'priv/quota/storage' Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Looking up (async) 'priv/quota/storage' Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): conn unix:/var/run/dovecot/dict: Connecting Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): conn unix:/var/run/dovecot/dict (pid=1,uid=0): Client connected (fd=10) Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Lookup finished for 'priv/quota/storage': found Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Lookup finished for 'priv/quota/storage': found Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Looking up 'priv/quota/messages' Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Looking up (async) 'priv/quota/messages' Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Lookup finished for 'priv/quota/messages': found Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Lookup finished for 'priv/quota/messages': found Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Looking up 'priv/quota/storage' Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Looking up (async) 'priv/quota/storage' Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): conn unix:/var/run/dovecot/dict: Connecting Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): conn unix:/var/run/dovecot/dict (pid=1,uid=0): Client connected (fd=12) Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Lookup finished for 'priv/quota/storage': found Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Lookup finished for 'priv/quota/storage': found Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Looking up 'priv/quota/messages' Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Looking up (async) 'priv/quota/messages' Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Lookup finished for 'priv/quota/messages': found Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Lookup finished for 'priv/quota/messages': found Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): dict destroyed Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Waiting for dict to finish pending operations Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): conn unix:/var/run/dovecot/dict (pid=1,uid=0): Disconnected: Connection closed (fd=10) Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): dict destroyed Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Waiting for dict to finish pending operations Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): conn unix:/var/run/dovecot/dict (pid=1,uid=0): Disconnected: Connection closed (fd=12) Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): dict destroyed Quota name Type Value Limit % User Quota STORAGE 25 1024000 0 User Quota MESSAGE 25000 - 0 Domain Quota STORAGE 36 10240000 0 Domain Quota MESSAGE 36000 - 0 Nov 27 15:46:31 doveadm(401): Debug: auth-master: conn unix:/var/run/dovecot/director-userdb (pid=1,uid=0): Disconnected: Connection closed (fd=9) Dovecot version 2.3.16, 2.3.15, 2.3.13. in this example 2.3.13 (89f716dc2) bullseye - *correct behavior, select* Debug: Loading modules from directory: /usr/lib/dovecot/modules Debug: Module loaded: /usr/lib/dovecot/modules/lib01_acl_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib20_mailbox_alias_plugin.so Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message) doveadm(h.gorska(a)example.com)<93><>: Debug: auth-master: userdb lookup(h.user(a)example.com): Started userdb lookup doveadm(h.user(a)example.com)<93><>: Debug: auth-master: conn unix:/run/dovecot/director-userdb: Connecting doveadm(h.user(a)example.com)<93><>: Debug: auth-master: conn unix:/run/dovecot/director-userdb (pid=1,uid=0): Client connected (fd=9) doveadm(h.user(a)example.com)<93><>: Debug: auth-master: userdb lookup(h.user(a)example.com): auth USER input: h.user(a)example.com home=/home/mail/virtual/data05/example.com/h.user@example.com quota2_rule=*:bytes=10485760000 quota_rule=*:bytes=1048576000 doveadm(h.user(a)example.com)<93><>: Debug: auth-master: userdb lookup(h.user(a)example.com): Finished userdb lookup (username=h.user(a)example.com home=/home/mail/virtual/data05/example.com/h.user@example.com quota2_rule=*:bytes=10485760000 quota_rule=*:bytes=1048576000) doveadm(h.user(a)example.com)<93><>: Debug: Added userdb setting: plugin/quota2_rule=*:bytes=10485760000 doveadm(h.user(a)example.com)<93><>: Debug: Added userdb setting: plugin/quota_rule=*:bytes=1048576000 doveadm(h.user(a)example.com): Debug: Effective uid=101, gid=102, home=/home/mail/virtual/data05/example.com/h.user@example.com doveadm(h.user(a)example.com): Debug: Home dir not found: /home/mail/virtual/data05/example.com/h.user(a)example.com doveadm(h.user(a)example.com): Debug: Quota root: name=User Quota backend=dict args=:proxy::sqluserquota doveadm(h.user(a)example.com): Debug: Quota rule: root=User Quota mailbox=* bytes=1048576000 messages=0 doveadm(h.user(a)example.com): Debug: Quota warning: bytes=1048576000 (100%) messages=0 reverse=no command=quota-warning 100 h.user(a)example.com doveadm(h.user(a)example.com): Debug: Quota warning: bytes=996147200 (95%) messages=0 reverse=no command=quota-warning 95 h.user(a)example.com doveadm(h.user(a)example.com): Debug: Quota warning: bytes=943718400 (90%) messages=0 reverse=no command=quota-warning 90 h.user(a)example.com doveadm(h.user(a)example.com): Debug: Quota warning: bytes=891289600 (85%) messages=0 reverse=no command=quota-warning 85 h.user(a)example.com doveadm(h.user(a)example.com): Debug: Quota grace: root=User Quota bytes=104857600 (10%) doveadm(h.user(a)example.com): Debug: Quota root: name=Domain Quota backend=dict args=example.com:proxy::sqldomainquota doveadm(h.user(a)example.com): Debug: Quota rule: root=Domain Quota mailbox=* bytes=10485760000 messages=0 doveadm(h.user(a)example.com): Debug: Quota grace: root=Domain Quota bytes=1048576000 (10%) doveadm(h.user(a)example.com): Debug: quota-dict: user=h.user(a)example.com, uri=proxy::sqluserquota, noenforcing=0 doveadm(h.user(a)example.com): Debug: quota-dict: user=example.com, uri=proxy::sqldomainquota, noenforcing=0 doveadm(h.user(a)example.com): Debug: Namespace : type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir://home/mail/virtual/data05/example.com/h.user@example.com/… doveadm(h.user(a)example.com): Debug: maildir++: root=//home/mail/virtual/data05/example.com/h.user@example.com/Maildir, index=, indexpvt=, control=, inbox=//home/mail/virtual/data05/example.com/h.user@example.com/Maildir, alt= doveadm(h.user(a)example.com): Debug: acl: initiali|//**domain**//|varchar(255)|Nie| |bytes|bigint(20)|Nie|0 |messages|bigint(20)|Nie|0 == Dumping data for table used_domain_quota |h.user(a)example.com|36000|36000 |example.com|35000|35000zing backend with data: vfile doveadm(h.user(a)example.com): Debug: acl: acl username= h.user(a)example.com doveadm(h.user(a)example.com): Debug: acl: owner= 1 doveadm(h.user(a)example.com): Debug: acl vfile: Global ACLs disabled doveadm(h.user(a)example.com): Debug: Namespace : type=shared, prefix=Shared/%u/, sep=/, inbox=no, hidden=no, list=children, subscriptions=yes location=maildir:%Lh/Maildir/:INDEX=%Lh/Maildir/Shared/%u doveadm(h.user(a)example.com): Debug: shared: root=/run/dovecot, index=, indexpvt=, control=, inbox=, alt= doveadm(h.user(a)example.com): Debug: acl: initializing backend with data: vfile doveadm(h.user(a)example.com): Debug: acl: acl username= h.user(a)example.com doveadm(h.user(a)example.com): Debug: acl: owner= 0 doveadm(h.user(a)example.com): Debug: acl vfile: Global ACLs disabled doveadm(h.user(a)example.com): Debug: quota: quota_over_flag check: quota_over_script unset - skipping doveadm(h.user(a)example.com): Debug: quota: quota_over_flag check: quota2_over_script unset - skipping Quota name Type Value Limit % User Quota STORAGE 25 1024000 0 User Quota MESSAGE 25000 - 0 Domain Quota STORAGE 35 10240000 0 Domain Quota MESSAGE 35000 - 0 doveadm(h.user(a)example.com): Debug: auth-master: conn unix:/run/dovecot/director-userdb (pid=1,uid=0): Disconnected: Connection closed (fd=9) MYSQL catch logs Mysql catch log during doveadm quota get -u h.user(a)example.com, version 2.3.17+ 592 Query SELECT username AS user, CONCAT('/home/mail/virtual/',mailbox.storagenode,mailbox.maildir,username) AS home, CONCAT('*:bytes=', domain.maxquota*1048576) AS quota2_rule, CONCAT('*:bytes=', mailbox.quota*1048576) AS quota_rule FROM mailbox,domain WHERE username = 'h.user(a)example.com'AND mailbox.domain=domain.domain AND domain.active=1 AND mailbox.`enabledoveadm`=1 AND mailbox.active=1 593 Connect root(a)10.15.1.40 on postfix using TCP/IP 593 Query SELECT bytes FROM used_quota WHERE username = 'h.user(a)example.com' 593 Query SELECT messages FROM used_quota WHERE username = 'h.user(a)example.com' 594 Connect root(a)10.15.1.40 on postfix using TCP/IP 594 Query SELECT bytes FROM used_domain_quota WHERE domain = 'h.user(a)example.com' 594 Query SELECT messages FROM used_domain_quota WHERE domain = 'h.user(a)example.com' example SQL UPDATE, dovecot version 2.3.20 (80a5ac675d) alpine 3.18.4 - bad behavior # User@Host: user[user @ [192.168.254.176] # Thread_id: 58405753 Schema: postfix QC_hit: No # Query_time: 0.000294 Lock_time: 0.000104 Rows_sent: 0 Rows_examined: 1 # Rows_affected: 1 Bytes_sent: 52 # # explain: id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra # explain: 1 SIMPLE used_domain_quota range PRIMARY PRIMARY 767 NULL 1 1.00 100.00 100.00 Using where # SET timestamp=1701103297; UPDATE used_domain_quota SET bytes=bytes+4359,messages=messages+1 WHERE domain = 'h.user(a)example.com'; example SQL UPDATE domain and user quota - *good behavior*, dovecot version 2.3.4.1 (f79e8e7e4), debian 10 # User@Host: postfix[postfix] @ [192.168.254.175] # Thread_id: 58406279 Schema: postfix QC_hit: No # Query_time: 0.000252 Lock_time: 0.000080 Rows_sent: 0 Rows_examined: 1 # Rows_affected: 1 Bytes_sent: 52 # # explain: id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra # explain: 1 SIMPLE used_domain_quota range PRIMARY PRIMARY 767 NULL 1 1.00 100.00 100.00 Using where # SET timestamp=1701103048; UPDATE used_domain_quota SET bytes=bytes+11597,messages=messages+1 WHERE domain = 'example.com'; # User@Host: user[user] @ [192.168.254.175] # Thread_id: 58406274 Schema: postfix QC_hit: No # Query_time: 0.000247 Lock_time: 0.000076 Rows_sent: 0 Rows_examined: 1 # Rows_affected: 1 Bytes_sent: 52 # # explain: id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra # explain: 1 SIMPLE used_quota range PRIMARY PRIMARY 767 NULL 1 1.00 100.00 100.00 Using where # SET timestamp=1701103048; UPDATE used_quota SET bytes=bytes+11597,messages=messages+1 WHERE username = 'h.user(a)example.com'; Hi, I notice that quota domain when using args %d in dictionary isn't honored in mysql during updates and selects when using dovecot 2.3.17+, so I moved my tests to docker environment, because I started using alpine linux and thought it could be musl library problem. I tested few versions with debian/alpine and after 2.3.17 updates and selects for domain in sql not using domain part, but user@domain. Maybe I missed some changes in configuration, maybe someone experience this behavior? User in logs was changed to h.user(a)example.com. ***** Configuration ***** as described in documentation quota = dict:<quota root name>:<username>[: <option>[...]]:<dictionary URI> where username could be %d for supporting domain-wide quotas quota = dict:user_quota::proxy::sqluserquota quota2 = dict:domain_quota:%d:proxy::sqldomainquota dict { acl = mysql:/etc/dovecot/dovecot-share-folder.conf sqldomainquota = mysql:/etc/dovecot/dovecot-dict-sql-domain.conf sqluserquota = mysql:/etc/dovecot/dovecot-dict-sql-user.conf } dovecot-dict-sql-domain.conf map { pattern = priv/quota/storage table = used_domain_quota username_field = domain value_field = bytes } map { pattern = priv/quota/messages table = used_domain_quota username_field = domain value_field = messages } dovecot-dict-sql-user.conf map { pattern = priv/quota/storage table = used_quota username_field = username value_field = bytes } map { pattern = priv/quota/messages table = used_quota username_field = username value_field = messages } Example dump sql table, in real environment with newer dovecot, there was only h.user(a)example.com and no domain record. In earlier versions of dovecot there was only domain in this table. |//**domain**//|varchar(255)|Nie| |bytes|bigint(20)|Nie|0 |messages|bigint(20)|Nie|0 == Dumping data for table used_domain_quota |h.user(a)example.com|36000|36000 |example.com|35000|35000 ***** Debug ***** **** Doveadm **** In both examples below debug args=example.com seems correct Dovecot version 2.3.17+, in this example debian 10 2.3.17 (e2aa53df5b), select with only username. root@cf801690360c:/# doveadm -D -v quota get -u h.user(a)example.com Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/ dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib/ dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/ lib10_doveadm_sieve_plugin.so Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: / usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib/ dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: / usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message) Nov 27 15:46:31 Debug: Loading modules from directory: /usr/lib/dovecot/modules Nov 27 15:46:31 Debug: Module loaded: /usr/lib/dovecot/modules/ lib01_acl_plugin.so Nov 27 15:46:31 Debug: Module loaded: /usr/lib/dovecot/modules/ lib10_quota_plugin.so Nov 27 15:46:31 Debug: Module loaded: /usr/lib/dovecot/modules/ lib20_mailbox_alias_plugin.so Nov 27 15:46:31 Debug: Loading modules from directory: /usr/lib/dovecot/ modules/doveadm Nov 27 15:46:31 Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/ lib10_doveadm_acl_plugin.so Nov 27 15:46:31 Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/ lib10_doveadm_quota_plugin.so Nov 27 15:46:31 Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/ lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message) Nov 27 15:46:31 Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message) Nov 27 15:46:31 Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/ libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message) Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><>: Debug: auth-master: userdb lookup(h.user(a)example.com): Started userdb lookup Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><>: Debug: auth-master: conn unix:/var/run/dovecot/director-userdb: Connecting Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><>: Debug: auth-master: conn unix:/var/run/dovecot/director-userdb (pid=1,uid=0): Client connected (fd=9) Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><>: Debug: auth-master: userdb lookup(h.user(a)example.com): auth USER input: h.user(a)example.com home=/home/ mail/virtual/data05/example.com/h.user@example.com quota2_rule=*: bytes=10485760000 quota_rule=*:bytes=1048576000 Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><>: Debug: auth-master: userdb lookup(h.user(a)example.com): Finished userdb lookup (username=h.user(a)example.com home=/home/mail/virtual/data05/example.com/h.user@example.com quota2_rule=*: bytes=10485760000 quota_rule=*:bytes=1048576000) Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><>: Debug: Added userdb setting: plugin/quota2_rule=*:bytes=10485760000 Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><>: Debug: Added userdb setting: plugin/quota_rule=*:bytes=1048576000 Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Effective uid=101, gid=101, home=/home/mail/virtual/data05/example.com/ h.user(a)example.com Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Home dir not found: /home/mail/virtual/data05/example.com/ h.user(a)example.com Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: dict(proxy): dict created (uri=proxy::acl, base_dir=/var/run/dovecot) Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Quota root: name=User Quota backend=dict args=:proxy::sqluserquota Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Quota rule: root=User Quota mailbox=* bytes=1048576000 messages=0 Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Quota warning: bytes=1048576000 (100%) messages=0 reverse=no command=quota-warning 100 h.user(a)example.com Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Quota warning: bytes=996147200 (95%) messages=0 reverse=no command=quota-warning 95 h.user(a)example.com Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Quota warning: bytes=943718400 (90%) messages=0 reverse=no command=quota-warning 90 h.user(a)example.com Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Quota warning: bytes=891289600 (85%) messages=0 reverse=no command=quota-warning 85 h.user(a)example.com Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Quota grace: root=User Quota bytes=104857600 (10%) Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Quota root: name=Domain Quota backend=dict args=example.com:proxy:: sqldomainquota Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Quota rule: root=Domain Quota mailbox=* bytes=10485760000 messages=0 Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Quota grace: root=Domain Quota bytes=1048576000 (10%) Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: quota-dict: user=h.user(a)example.com, uri=proxy::sqluserquota, noenforcing=0 Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: dict(proxy): dict created (uri=proxy::sqluserquota, base_dir=/var/run/ dovecot) Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: quota-dict: user=example.com, uri=proxy::sqldomainquota, noenforcing=0 Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: dict(proxy): dict created (uri=proxy::sqldomainquota, base_dir=/var/run/ dovecot) Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Namespace : type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir://home/mail/virtual/data05/ example.com/h.user@example.com/Maildir/:INDEX=//home/mail/virtual/data05/ example.com/h.user@example.com/Maildir/ Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: maildir++: root=//home/mail/virtual/data05/example.com/ h.user(a)example.com/Maildir, index=, indexpvt=, control=, inbox=//home/mail/ virtual/data05/example.com/h.user@example.com/Maildir, alt= Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: acl: initializing backend with data: vfile Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: acl: acl username = h.user(a)example.com Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: acl: owner = 1 Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: acl vfile: Global ACLs disabled Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: Namespace : type=shared, prefix=Shared/%u/, sep=/, inbox=no, hidden=no, list=children, subscriptions=yes location=maildir:%Lh/Maildir/:INDEX=%Lh/ Maildir/Shared/%u Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: shared: root=/var/run/dovecot, index=, indexpvt=, control=, inbox=, alt= Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: acl: initializing backend with data: vfile Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: acl: acl username = h.user(a)example.com Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: acl: owner = 0 Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: acl vfile: Global ACLs disabled Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: quota: quota_over_flag check: quota_over_script unset - skipping Nov 27 15:46:31 doveadm(h.user(a)example.com)<401><4l0XONe5ZGWRAQAAQCEzFg>: Debug: quota: quota_over_flag check: quota2_over_script unset - skipping Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Looking up 'priv/quota/storage' Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Looking up (async) 'priv/quota/storage' Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): conn unix:/ var/run/dovecot/dict: Connecting Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): conn unix:/ var/run/dovecot/dict (pid=1,uid=0): Client connected (fd=10) Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Lookup finished for 'priv/quota/storage': found Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Lookup finished for 'priv/quota/storage': found Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Looking up 'priv/quota/messages' Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Looking up (async) 'priv/quota/messages' Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Lookup finished for 'priv/quota/messages': found Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Lookup finished for 'priv/quota/messages': found Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Looking up 'priv/quota/storage' Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Looking up (async) 'priv/quota/storage' Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): conn unix:/ var/run/dovecot/dict: Connecting Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): conn unix:/ var/run/dovecot/dict (pid=1,uid=0): Client connected (fd=12) Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Lookup finished for 'priv/quota/storage': found Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Lookup finished for 'priv/quota/storage': found Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Looking up 'priv/quota/messages' Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Looking up (async) 'priv/quota/messages' Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Lookup finished for 'priv/quota/messages': found Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Lookup finished for 'priv/quota/messages': found Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): dict destroyed Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Waiting for dict to finish pending operations Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): conn unix:/ var/run/dovecot/dict (pid=1,uid=0): Disconnected: Connection closed (fd=10) Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): dict destroyed Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): Waiting for dict to finish pending operations Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): conn unix:/ var/run/dovecot/dict (pid=1,uid=0): Disconnected: Connection closed (fd=12) Nov 27 15:46:31 doveadm(h.user(a)example.com): Debug: dict(proxy): dict destroyed Quota name Type Value Limit % User Quota STORAGE 25 1024000 0 User Quota MESSAGE 25000 - 0 Domain Quota STORAGE 36 10240000 0 Domain Quota MESSAGE 36000 - 0 Nov 27 15:46:31 doveadm(401): Debug: auth-master: conn unix:/var/run/dovecot/ director-userdb (pid=1,uid=0): Disconnected: Connection closed (fd=9) Dovecot version 2.3.16, 2.3.15, 2.3.13. in this example 2.3.13 (89f716dc2) bullseye - correct behavior, select Debug: Loading modules from directory: /usr/lib/dovecot/modules Debug: Module loaded: /usr/lib/dovecot/modules/lib01_acl_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib20_mailbox_alias_plugin.so Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/ lib10_doveadm_acl_plugin.so Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/ lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/ lib10_doveadm_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/ lib10_doveadm_sieve_plugin.so Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: / usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib/ dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: / usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message) doveadm(h.gorska(a)example.com)<93><>: Debug: auth-master: userdb lookup (h.user(a)example.com): Started userdb lookup doveadm(h.user(a)example.com)<93><>: Debug: auth-master: conn unix:/run/dovecot/ director-userdb: Connecting doveadm(h.user(a)example.com)<93><>: Debug: auth-master: conn unix:/run/dovecot/ director-userdb (pid=1,uid=0): Client connected (fd=9) doveadm(h.user(a)example.com)<93><>: Debug: auth-master: userdb lookup (h.user(a)example.com): auth USER input: h.user(a)example.com home=/home/mail/ virtual/data05/example.com/h.user@example.com quota2_rule=*:bytes=10485760000 quota_rule=*:bytes=1048576000 doveadm(h.user(a)example.com)<93><>: Debug: auth-master: userdb lookup (h.user(a)example.com): Finished userdb lookup (username=h.user(a)example.com home=/home/mail/virtual/data05/example.com/h.user@example.com quota2_rule=*: bytes=10485760000 quota_rule=*:bytes=1048576000) doveadm(h.user(a)example.com)<93><>: Debug: Added userdb setting: plugin/ quota2_rule=*:bytes=10485760000 doveadm(h.user(a)example.com)<93><>: Debug: Added userdb setting: plugin/ quota_rule=*:bytes=1048576000 doveadm(h.user(a)example.com): Debug: Effective uid=101, gid=102, home=/home/ mail/virtual/data05/example.com/h.user@example.com doveadm(h.user(a)example.com): Debug: Home dir not found: /home/mail/virtual/ data05/example.com/h.user@example.com doveadm(h.user(a)example.com): Debug: Quota root: name=User Quota backend=dict args=:proxy::sqluserquota doveadm(h.user(a)example.com): Debug: Quota rule: root=User Quota mailbox=* bytes=1048576000 messages=0 doveadm(h.user(a)example.com): Debug: Quota warning: bytes=1048576000 (100%) messages=0 reverse=no command=quota-warning 100 h.user(a)example.com doveadm(h.user(a)example.com): Debug: Quota warning: bytes=996147200 (95%) messages=0 reverse=no command=quota-warning 95 h.user(a)example.com doveadm(h.user(a)example.com): Debug: Quota warning: bytes=943718400 (90%) messages=0 reverse=no command=quota-warning 90 h.user(a)example.com doveadm(h.user(a)example.com): Debug: Quota warning: bytes=891289600 (85%) messages=0 reverse=no command=quota-warning 85 h.user(a)example.com doveadm(h.user(a)example.com): Debug: Quota grace: root=User Quota bytes=104857600 (10%) doveadm(h.user(a)example.com): Debug: Quota root: name=Domain Quota backend=dict args=example.com:proxy::sqldomainquota doveadm(h.user(a)example.com): Debug: Quota rule: root=Domain Quota mailbox=* bytes=10485760000 messages=0 doveadm(h.user(a)example.com): Debug: Quota grace: root=Domain Quota bytes=1048576000 (10%) doveadm(h.user(a)example.com): Debug: quota-dict: user=h.user(a)example.com, uri=proxy::sqluserquota, noenforcing=0 doveadm(h.user(a)example.com): Debug: quota-dict: user=example.com, uri=proxy:: sqldomainquota, noenforcing=0 doveadm(h.user(a)example.com): Debug: Namespace : type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir://home/mail/ virtual/data05/example.com/h.user@example.com/Maildir/:INDEX=//home/mail/ virtual/data05/example.com/h.user@example.com/Maildir/ doveadm(h.user(a)example.com): Debug: maildir++: root=//home/mail/virtual/data05/ example.com/h.user@example.com/Maildir, index=, indexpvt=, control=, inbox=// home/mail/virtual/data05/example.com/h.user@example.com/Maildir, alt= doveadm(h.user(a)example.com): Debug: acl: initiali|//**domain**//|varchar (255)|Nie| |bytes|bigint(20)|Nie|0 |messages|bigint(20)|Nie|0 == Dumping data for table used_domain_quota |h.user(a)example.com|36000|36000 |example.com|35000|35000zing backend with data: vfile doveadm(h.user(a)example.com): Debug: acl: acl username = h.user(a)example.com doveadm(h.user(a)example.com): Debug: acl: owner = 1 doveadm(h.user(a)example.com): Debug: acl vfile: Global ACLs disabled doveadm(h.user(a)example.com): Debug: Namespace : type=shared, prefix=Shared/%u/, sep=/, inbox=no, hidden=no, list=children, subscriptions=yes location=maildir: %Lh/Maildir/:INDEX=%Lh/Maildir/Shared/%u doveadm(h.user(a)example.com): Debug: shared: root=/run/dovecot, index=, indexpvt=, control=, inbox=, alt= doveadm(h.user(a)example.com): Debug: acl: initializing backend with data: vfile doveadm(h.user(a)example.com): Debug: acl: acl username = h.user(a)example.com doveadm(h.user(a)example.com): Debug: acl: owner = 0 doveadm(h.user(a)example.com): Debug: acl vfile: Global ACLs disabled doveadm(h.user(a)example.com): Debug: quota: quota_over_flag check: quota_over_script unset - skipping doveadm(h.user(a)example.com): Debug: quota: quota_over_flag check: quota2_over_script unset - skipping Quota name Type Value Limit % User Quota STORAGE 25 1024000 0 User Quota MESSAGE 25000 - 0 Domain Quota STORAGE 35 10240000 0 Domain Quota MESSAGE 35000 - 0 doveadm(h.user(a)example.com): Debug: auth-master: conn unix:/run/dovecot/ director-userdb (pid=1,uid=0): Disconnected: Connection closed (fd=9) **** MYSQL catch logs **** Mysql catch log during doveadm quota get -u h.user(a)example.com, version 2.3.17+ 592 Query SELECT username AS user, CONCAT('/home/mail/virtual/ ',mailbox.storagenode,mailbox.maildir,username) AS home, CONCAT('*:bytes=', domain.maxquota*1048576) AS quota2_rule, CONCAT('*:bytes=', mailbox.quota*1048576) AS quota_rule FROM mailbox,domain WHERE username = 'h.user(a)example.com' AND mailbox.domain=domain.domain AND domain.active=1 AND mailbox.`enabledoveadm`=1 AND mailbox.active=1 593 Connect root(a)10.15.1.40 on postfix using TCP/IP 593 Query SELECT bytes FROM used_quota WHERE username = 'h.user(a)example.com' 593 Query SELECT messages FROM used_quota WHERE username = 'h.user(a)example.com' 594 Connect root(a)10.15.1.40 on postfix using TCP/IP 594 Query SELECT bytes FROM used_domain_quota WHERE domain = 'h.user(a)example.com' 594 Query SELECT messages FROM used_domain_quota WHERE domain = 'h.user(a)example.com' example SQL UPDATE, dovecot version 2.3.20 (80a5ac675d) alpine 3.18.4 - bad behavior # User@Host: user[user @ [192.168.254.176] # Thread_id: 58405753 Schema: postfix QC_hit: No # Query_time: 0.000294 Lock_time: 0.000104 Rows_sent: 0 Rows_examined: 1 # Rows_affected: 1 Bytes_sent: 52 # # explain: id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra # explain: 1 SIMPLE used_domain_quota range PRIMARY PRIMARY 767 NULL 1 1.00 100.00 100.00 Using where # SET timestamp=1701103297; UPDATE used_domain_quota SET bytes=bytes+4359,messages=messages+1 WHERE domain = 'h.user(a)example.com'; example SQL UPDATE domain and user quota - good behavior, dovecot version 2.3.4.1 (f79e8e7e4), debian 10 # User@Host: postfix[postfix] @ [192.168.254.175] # Thread_id: 58406279 Schema: postfix QC_hit: No # Query_time: 0.000252 Lock_time: 0.000080 Rows_sent: 0 Rows_examined: 1 # Rows_affected: 1 Bytes_sent: 52 # # explain: id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra # explain: 1 SIMPLE used_domain_quota range PRIMARY PRIMARY 767 NULL 1 1.00 100.00 100.00 Using where # SET timestamp=1701103048; UPDATE used_domain_quota SET bytes=bytes+11597,messages=messages+1 WHERE domain = 'example.com'; # User@Host: user[user] @ [192.168.254.175] # Thread_id: 58406274 Schema: postfix QC_hit: No # Query_time: 0.000247 Lock_time: 0.000076 Rows_sent: 0 Rows_examined: 1 # Rows_affected: 1 Bytes_sent: 52 # # explain: id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra # explain: 1 SIMPLE used_quota range PRIMARY PRIMARY 767 NULL 1 1.00 100.00 100.00 Using where # SET timestamp=1701103048; UPDATE used_quota SET bytes=bytes+11597,messages=messages+1 WHERE username = 'h.user(a)example.com';

3 3

doveadm auth lookup fails for system user
by Christian H. Kuhn 15 Jul '24

15 Jul '24

Hi all, next step with my auth problem with dovecot. I want to authenticate a system user. The user exists, can log in, can sudo -i etc.pp. SASL with sql passdb and userdb works fine. root@bywater /etc/dovecot/conf.d # doveadm user qno field value uid 1001 gid 1001 home /home/qno mail maildir:~/Maildir system_groups_user qno But: root@bywater /etc/dovecot/conf.d # doveadm auth lookup qno passdb lookup: user qno doesn't exist And no surprise: root@bywater /etc/dovecot/conf.d # doveadm auth test qno Password: passdb: qno auth failed extra fields: user=qno root@bywater /etc/dovecot/conf.d # doveconf -n # 2.3.16 (7e2e900c1a): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.16 (09c29328) # OS: Linux 5.15.0-113-generic x86_64 Ubuntu 22.04.4 LTS # Hostname: bywater.qno.de auth_debug = yes auth_debug_passwords = yes auth_verbose = yes auth_verbose_passwords = plain listen = 65.21.136.15, [::] mail_location = maildir:~/Maildir mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/tables.d/dovecot-sql.conf.ext driver = sql } passdb { args = blocking=no driver = passwd } passdb { driver = pam } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve } postmaster_address = postmaster(a)qno.de protocols = " imap sieve" service auth-worker { user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = dovecot } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { unix_listener lmtp { group = postfix mode = 0600 user = postfix } } ssl = required ssl_cert = </etc/letsencrypt/live/imap2.qno.de/fullchain.pem ssl_cipher_list = ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305 ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it syslog_facility = local0 userdb { args = uid=vmail gid=vmail home=/var/mail/vhosts/%d/%u driver = static } userdb { driver = passwd } verbose_proctitle = yes How can it be that a user is found by userdb passwd, but not by passdb passwd or PAM? TIA QNo

3 3

Failed to connect to dns-client: Resource temporarily unavailable
by m＠maltris.org 15 Jul '24

15 Jul '24

When using DNS names for backends in a proxied setup, the async dns_client process after a short time of high load (burst scenario) seems to get overwhelmed: auth: Debug: conn unix:dns-client: dns(backend.local): Lookup failed after 0 msecs: Failed to connect to dns-client: Resource temporarily unavailable auth: Error: DNS lookup for backend.local failed: Failed to connect to dns-client: Resource temporarily unavailable The whole thing itself runs for a few thousand requests until this happens. ulimit nofile is set to 65000. When changing the backend from DNS name to IP, there is no problem. What are the possibilities to use dns names but without these problems? Any way to either increase dns resolving capacity OR cache the results for some time (or forever?)? Best regards

2 1

dovecot replication
by Jeff Pang 13 Jul '24

13 Jul '24

Hello, Does the community version of dovecot have the replication feature? When one dovecot was down, another one could take over the tasks. Thanks. _________________________________________________________________ ________________________________________________________ Your E-Mail. Your Cloud. Your Office. eclipso Mail Europe. https://www.eclipso.de

4 7

dovecot imap_zlib
by Joan Moreau 12 Jul '24

12 Jul '24

Hi I tested teh git version of dovecot. It seems the IMAP Compress plugin (imap_zlib) has disappeared. How to get it back ? Thank you

6 7

New Member Introduction
by Selena Thomas 10 Jul '24

10 Jul '24

Hi everyone, I am new to this forum and excited to be here. I'm interested in learning more about Dovecot and its features, and I'm eager to participate in the discussions here. Could someone please guide me on how to ask questions here ? Where should I post if I have a query ? Looking forward to your advice and connecting with you all. Thanks,

2 1

dovecot-lda: timeout for external scripts called by sieve
by Meinhard Schneider 10 Jul '24

10 Jul '24

Hello, I have a small problem with dovecot-lda. I am using Debian 12 Bookworm: root@lnxmail01:~# dovecot --version 2.3.13 (89f716dc2) Historically grown (probably still from sendmail times, long time ago) I always used procmail for local e-mail delivery. This also included the call for spamc for SpamAssassin. I have now changed this and set up a route via sieve. There is a small script for this: root@lnxmail01:~# cat /usr/lib/dovecot/sieve-filter/spamc-filter.sh #!/bin/bash /usr/bin/spamc My sieve configuration says (in extracts): root@lnxmail01:~# head -12 ~meini/.dovecot.sieve ## Generated by Roundcube Webmail SieveRules Plugin ## require ["body","comparator-i;ascii-numeric","copy","envelope","fileinto","imap4flags","include","regex","relational","spamtestplus","vacation","variables","spamtest"]; include :global "spamc"; # rule:[Spam] if spamtest :value "ge" :comparator "i;ascii-numeric" "5" { fileinto "Spam"; stop; } In general, this has been working for many months without any problems. But sometimes the whole thing fails due to a timeout: Jul 7 22:00:13 lnxmail01 dovecot: lda(meini)<766014><G9l7JcPzimY+sAsAmbNf3A>: Error: program exec:/usr/lib/dovecot/sieve-filter/spamc-filter.sh (766016): Execution timed out (> 10000 msecs) Jul 7 22:00:13 lnxmail01 dovecot: lda(meini)<766014><G9l7JcPzimY+sAsAmbNf3A>: Error: program exec:/usr/lib/dovecot/sieve-filter/spamc-filter.sh (766016): Forcibly terminated with signal 15 Jul 7 22:00:13 lnxmail01 dovecot: lda(meini)<766014><G9l7JcPzimY+sAsAmbNf3A>: Panic: output stream (temp iostream in /tmp/dovecot.lda. for (program client seekable output)) is missing error handling Jul 7 22:00:13 lnxmail01 dovecot: lda(meini)<766014><G9l7JcPzimY+sAsAmbNf3A>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x42) [0x7ff00aee04e2] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7ff00aee05fe] -> /usr/lib/dovecot/libdovecot.so.0(+0xfc49b) [0x7ff00aeec49b] -> /usr/lib/dovecot/libdovecot.so.0(+0xfc501) [0x7ff00aeec501] -> /usr/lib/dovecot/libdovecot.so.0(+0x53aee) [0x7ff00ae43aee] -> /usr/lib/dovecot/libdovecot.so.0(o_stream_unref+0x5e) [0x7ff00af1003e] -> /usr/lib/dovecot/libdovecot.so.0(program_client_destroy+0x8e) [0x7ff00ae6ee6e] -> /usr/lib/dovecot/modules/sieve/lib90_sieve_extprograms_plugin.so(sieve_extprogram_destroy+0x18) [0x7ff00aacd8f8] -> /usr/lib/dovecot/modules/sieve/lib90_sieve_extprograms_plugin.so(+0x556d) [0x7ff00aacb56d] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xea) [0x7ff00ab4c9ba] -> /usr/lib/dovecot/libdovecot-sieve.so.0(ext_include_execute_include+0x2c7) [0x7ff00ab72e17] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xea) [0x7ff00ab4c9ba] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_multiscript_run+0x82) [0x7ff00ab62b32] -> /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x37a3) [0x7ff00abd47a3] -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x176) [0x7ff00b11f906] -> /usr/lib/dovecot/dovecot-lda(main+0x879) [0x55ef085e4fb9] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7ff00ac3ad0a] -> /usr/lib/dovecot/dovecot-lda(_start+0x2a) [0x55ef085e56ea] The worst thing is that the entire mail is lost (/dev/null)! Then I tried to set the timeout higher (than the current 10 seconds): root@lnxmail01:~# doveconf | grep -A 12 ^plugin plugin { sieve = file:~/sieve;active=~/.dovecot.sieve sieve_execute_exec_timeout = 60s sieve_extension_exec_timeout = 60s sieve_extensions = +vnd.dovecot.filter +spamtest +spamtestplus sieve_filter_bin_dir = /usr/lib/dovecot/sieve-filter sieve_filter_socket_dir = sieve-filter sieve_global = /usr/lib/dovecot/sieve-global sieve_plugins = sieve_extprograms sieve_spamtest_max_value = 5 sieve_spamtest_status_header = X-Spam-Level sieve_spamtest_status_type = strlen } I have set both the values for "sieve_execute_exec_timeout" and "sieve_extension_exec_timeout" to 60 sec, but according to the log, this does not seem to affect the 10000 msec mentioned therein. The SpamAssassin daemon has enough child processes to handle the requests (most of them were idle at the time of the error). How can I configure that the external script gets more time for execution? And what can I do so that - should the script terminate contrary to expectations or be killed in the timeout - the mail is not lost but simply processed further? Many thanks & best regards Meinhard

2 3

Re: dovecot imap_zlib
by Michael Slusarz 10 Jul '24

10 Jul '24

> On 07/09/2024 6:42 AM MDT Bjoern Franke via dovecot <dovecot(a)dovecot.org> wrote: > > >> I tested teh git version of dovecot. It seems the IMAP Compress plugin > >> (imap_zlib) has disappeared. > > > > https://github.com/dovecot/core/commit/5f27e25c64555dcaae6cb00c479cd05bc263… > > so the zlib plugin is also deprecated and clients should run compress > themselves? These are two distinct plugins. imap-zlib = compression of the IMAP protocol stream zlib = mailbox storage compression on the Dovecot server For 2.4, "zlib" plugin has been renamed to "mail-compress". https://doc.dovecot.org/2.4/core/plugins/mail_compress.html michael

3 2