- dovecot - dovecot.org

iPhone Mail clients generate high Solr I/O
by Alessio Cecchi 22 Mar '23

22 Mar '23

Hi, I'm running a mail server with Dovecot 2.3.20 and Apache Solr as FTS backend. It seems that iPhone/iPad Mail clients are generating some IMAP searches, especially on header Message-ID, that are increasing our Solr I/O load, especially during the night. Are these kind of queries normal? What are they? You can find some logs below. Dovecot logs: Mar 13 21:06:14 dovecot: imap-login: Login: user=<user@domain>, method=PLAIN, rip=x.x.x.x, lip=z.z.z.z, mpid=11263, secured, session=<PQSppM32gePCBJGh> Mar 13 21:06:14 dovecot: imap(user@domain) session=<PQSppM32gePCBJGh>: ID sent: name=iPhone Mail, version=20D67, os=iOS, os-version=16.3.1 (20D67) Mar 13 21:06:14 dovecot: imap(user@domain) session=<PQSppM32gePCBJGh>: Disconnected: Logged out in=207 out=1128 deleted=0 expunged=0 autoexpunged=0 trashed=0 appended=0 hdr_count=0 hdr_bytes=0 body_count=0 body_bytes=0 Solr logs: 2023-03-13 21:06:14.411 INFO (qtp1671846437-1652) [ x:dovecot] o.a.s.c.S.Request [dovecot] webapp=/solr path=/select params={q={!lucene+q.op%3DAND}hdr:<DB8P189MB08692AE65580C36C3B7D6F53C1F59(a)domain6.tld>+OR+hdr:<\!\%26\!AAAAAAAAAAAYAAAAAAAAAI\/Oo\%2BConZZMkCvmEY7yWqjCgAAAEAAAAP968Yud4n1LmhwJxWgTURQBAAAAAA%3D%3D(a)domain5.tld>+OR+hdr:<CALEb\-gfyeQ1zJhjGKS3V8Rn3gk3byNa9co9nOU1L2BtUtKO0aw(a)domain7.tld>+OR+hdr:<AS4P195MB1790AA65F5F367F3CF0888BEDEA19(a)domain5.tld>+OR+hdr:<24E36A6E5BB2410C8090482B3E1EB40F(a)domain8.tld>&fl=uid,score&sort=uid+asc&fq=%2Bbox:d0f433254ee1b161c82c00002e1056c4+%2Buser:user@domain&rows=88&wt=xml} hits=0 status=0 QTime=275 2023-03-13 21:06:14.511 INFO (qtp1671846437-1628) [ x:dovecot] o.a.s.c.S.Request [dovecot] webapp=/solr path=/select params={q={!lucene+q.op%3DAND}hdr:<af03d59c\-2cd5\-1165\-aea5\-6ce7cfe16a89(a)domain.tld>+OR+hdr:<CAJbcrLvxCOGy6fcKBuhuu9B1SALwm9opS\-\-O\-kcMd3nevKrf1A(a)domain7.tld>+OR+hdr:<00eb01d8c8dc$caa8a580$5ff9f080$(a)domain1.tld>+OR+hdr:<VE1PR08MB4846DD525845DC743D47ED52EEBA9(a)domain7.tld>+OR+hdr:<00c701d8bb83$f307bc70$d9173550$(a)domain1.tld>&fl=uid,score&sort=uid+asc&fq=%2Bbox:d0f433254ee1b161c82c00002e1056c4+%2Buser:user@domain&rows=88&wt=xml} hits=0 status=0 QTime=59 2023-03-13 21:06:14.714 INFO (qtp1671846437-1652) [ x:dovecot] o.a.s.c.S.Request [dovecot] webapp=/solr path=/select params={q={!lucene+q.op%3DAND}hdr:<AS4P195MB179002723893D3932AF83A4CDEA29(a)domain5.tld>+OR+hdr:<F3112D85FDF144319FB0ABE348A75AE5(a)domain8.tld>+OR+hdr:<AM9PR10MB510032098A44B16182C9AAF39C4D9(a)domain8.tld>+OR+hdr:<\!\%26\!AAAAAAAAAAAYAAAAAAAAAI\/Oo\%2BConZZMkCvmEY7yWqjCgAAAEAAAADl2MLfkWRxJjQW\%2B3dOhZn0BAAAAAA%3D%3D(a)domain5.tld>+OR+hdr:<CAJbcrLtAEjLO0wbK5AAp8a3p0iRaoUeKA1wsD2ppDFeMnhL\-kA(a)domain7.tld>&fl=uid,score&sort=uid+asc&fq=%2Bbox:d0f433254ee1b161c82c00002e1056c4+%2Buser:user@domain&rows=88&wt=xml} hits=0 status=0 QTime=178 2023-03-13 21:06:14.771 INFO (qtp1671846437-1628) [ x:dovecot] o.a.s.c.S.Request [dovecot] webapp=/solr path=/select params={q={!lucene+q.op%3DAND}hdr:<DU0P189MB208990AA60E3EB5CB2F9BE98DCD39@domain>+OR+hdr:<DDA00A9A19CE4864AD87EF31B0E2113F(a)domain8.tld>+OR+hdr:<GV1PR01MB882049EDFF560AF4F4444F4BB1EE9(a)domain7.tld>&fl=uid,score&sort=uid+asc&fq=%2Bbox:d0f433254ee1b161c82c00002e1056c4+%2Buser:user@domain&rows=88&wt=xml} hits=0 status=0 QTime=34 2023-03-13 21:06:14.899 INFO (qtp1671846437-1652) [ x:dovecot] o.a.s.c.S.Request [dovecot] webapp=/solr path=/select params={q={!lucene+q.op%3DAND}hdr:<AS4P195MB1790922D3154E37AEF263FD1DEDF9(a)domain5.tld>+OR+hdr:<02E5E16DF27945B6A764F49A5D9E055E(a)domain8.tld>+OR+hdr:<CAEK\-5TdF_f%3Dn\%2BL1VWj3EVP9i%3DQe\%2BWaYcW%3DFNy8GSLfE5O1U7dw(a)domain7.tld>+OR+hdr:<4edb1720076ab71575067b0fc2b5d96c(a)domain2.tld>+OR+hdr:<1657012847.68884591(a)domain6.tld>&fl=uid,score&sort=uid+asc&fq=%2Bbox:4830f5144ee1b161c82c00002e1056c4+%2Buser:user@domain&rows=1&wt=xml} hits=0 status=0 QTime=46 Thanks -- Alessio Cecchi Postmaster @http://www.qboxmail.it https://www.linkedin.com/in/alessice

2 1

Import collection of files containing one raw message each into dovecot
by Markus Schönhaber 21 Mar '23

21 Mar '23

Hi, given a bunch of files, each containing the raw source of a single email message. There's no structural or naming scheme like Maildir or something. Is there a (easy) way to import those mails into dovecot? AIUI doveadm import needs a source location in a supported mailbox format - which doesn't seem applicable here. -- Regards mks

2 2

"doveadm altmove -r" not working ?
by Benoit Branciard 21 Mar '23

21 Mar '23

Hello, I have some trouble using "doveadm altmove -r". Regular "doveadm altmove" is OK, selected mails were moved to alternate storage as expected. But I can't manage to get them back to original location, "doveadm altmove -r" has no effect. We are using Dovecot 2.3.4 (Debian Buster package). mail_location = mdbox:~/mdbox:DIRNAME=_@@_dbox-Mails_@@_:ALT=/slow%h/mdbox doveadm altmove -u myuser since 90d -> some message files are created under /slow/my/homedir/mdbox/storage doveadm altmove -r -u myuser all -> message files are still sitting under /slow/my/homedir/mdbox/storage, none were created into /my/homedir/mdbox/storage. Is there some known issues with doveadm altmove in this version ? Or am I missing something ? I can share more config details if needed. -- Benoit BRANCIARD Université Paris 1 Panthéon-Sorbonne - DSIUN-SIS B405 - Centre PMF - 90 rue de Tolbiac - 75013 Paris Tel. 01 44 07 89 68 http://dsiun.univ-paris1.fr

4 5

Read-only / archive mode for IMAP mailboxes?
by Rᴇɴᴇ́ Pꜰᴇɪꜰꜰᴇʀ 20 Mar '23

20 Mar '23

Hello! We are currently exploring email archiving solutions. Is there a way to use an IMAP mailbox in read-only / archive mode? The requirement is that delibveries of new emails should be possible (via SMTP/LMTP), but no messages should be deleted via IMAP access. Frankly I don't think this works, but maybe some of you have tried this already. Regards, René. -- )\._.,--....,'``. fL Let GNU/Linux work for you while you take a nap. /, _.. \ _\ (`._ ,. R. Pfeiffer <lynx at luchs.at> + https://web.luchs.at/ `._.-(,_..'--(,_..'`-.;.' - System administration + Consulting + Teaching - 📨 Got mail delivery problems? https://web.luchs.at/information/blockedmail.php 📯 Complete contact information: https://web.luchs.at/kontakt/ 🔏 Download latest GPG/PGP keys: https://web.luchs.at/kontakt/gpg.asc ⛔ W͇A͇R͇N͇I͇N͇G͇:͇ ͇D͇O͇ ͇_͇N͇O͇T͇_͇ ͇S͇E͇N͇D͇ ͇E͇M͇A͇I͇L͇S͇ ͇W͇I͇T͇H͇ ͇H͇T͇M͇L͇ ͇C͇O͇N͇T͇E͇N͇T͇ ͇T͇O͇ ͇M͇Y͇ ͇E͇M͇A͇I͇L͇ ͇A͇D͇D͇R͇E͇S͇S͇!͇ ⛔

3 3

Remote backup of IMAP user fails with dsync_remote epoll_ctl error
by dml＠plumeriahale.net 20 Mar '23

20 Mar '23

I’m trying backup a user’s IMAP folders from a system running dovecot 2.1.7. to a system running dovecot 2.3.19.1. After much perusing of the web and Dovecot documentation I tried this: 1) On the 2.1.7 host set up the doveadm server to listen on TCP port 2425 and defined a doveadm_password value 2) Issued the following command on the 2.13.19.1 system: doveadm -o doveadm_password=<password> backup -R -u <username> -S <host>:2425 tcp:<host> 3) The command returned the following errors: Error: doveadm server disconnected before handshake: EOF Error: Command backup failed for <username>: EOF 4) In the Dovecot log on the 2.1.7 system I found the following error message: dsync(<username>): Error: user <username>: Initialization failed: Namespace '': Unknown mail storage driver tcp dovecot: dsync(<username>): Fatal: User init failed A couple of questions: 1) I thought that setting up doveadm-server as an inet_listener (with a password) meant that all communications (including the backup) occurred over that link - is that not correct? Or does SSH also need to be enabled as well? 2) Am I specifying the arguments correctly? The definition of the destination argument is confusing in the documentation - if doveadm is trying to backup a user’s IMAP mailboxes from a remote source, and you’ve already specified the information needed to connect to the remote sources doveadm-server why would the source need to be specified again via the TCP: argument?

1 0

Disable folder creation for details username
by Robert Blayzor 20 Mar '23

20 Mar '23

This may be more of a CPanel issue than Dovecot but CPanel uses Dovecot and not sure if this is a knob we have available in Dovecot or we need to reach out to Cpanel directly to see if this is some custom code they added to their distros. Anyway, W/R/T emails sent with dtails/ + addresses ie: foo+bar(a)mydomain.com And with LDA/LMTP, in Cpanel by default the detials part, in this case "bar" will be created in it's own folder. Automatically. To us, this can be an attack vector/DOS from a malicious actor so we want to turn it off. We understand there is: lda_mailbox_autocreate Which we have yes, as we do want to create mailboxes automatically when the first message comes in, but not these folders. I don't know if this folder autocreation on detail part is a Dovecot thing, or specific to Cpanel. It appears CPanel has a patch/code to option to turn it off, but it's only on a per mailbox basis. We want to turn this behavior off globally. As far as I know, CPanel doesn't release whatever patches they made to make this happen, asking here if anyone has experience with this before we start chasing them for answers. -- inoc.net!rblayzor XMPP: rblayzor.AT.inoc.net PGP: https://pgp.inoc.net/rblayzor/

2 2

Dovecot unified event filtering
by Felix Auringer 20 Mar '23

20 Mar '23

Hello, I have an issue with debug logging when using a custom plugin for Dovecot. In my plugin, I create a child event of the session's user event: ```c struct event *plugin_event = event_create(list->ns->user->event); event_set_name(plugin_event, "oidc_shared_mailboxes_plugin"); event_set_min_log_level(plugin_event, LOG_TYPE_WARNING); event_set_append_log_prefix(plugin_event, "oidc-shared-mailboxes-plugin: "); ``` I then use passthrough events like the following: ``` e_debug(event_create_passthrough(data->event)->event(), "Modified mailbox_list_iter_init was called."); e_warning(event_create_passthrough(event)->event(), "Did not find required key 'roles' in the JWT body."); ``` `e_warning` and `e_error` work fine. If I understand the documentation for the unified event filtering (https://doc.dovecot.org/configuration_manual/event_filter/#unified-filter-l…) correctly, I should be able to enable debug logging for with `log_debug = event="oidc_shared_mailboxes_plugin"`. However, this setting has no influence on the logging behavior. The setting I have to set for the debug logging to work is `mail_debug = yes` (I do not have to set `log_debug` at all). Is this intended behavior or am I doing something wrong? Best regards, Felix Auringer Here is the output of `dovecot -n` (I removed the passdb, userdb, namespace, service, and managesieve settings as they cluttered the output): ``` # 2.3.19 (b3ad6004dc): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.19 (4eae2f79) # OS: Linux 5.10.0-15-amd64 x86_64 Debian 11.6 # Hostname: c3dd5b346d22 auth_mechanisms = plain login xoauth2 oauthbearer debug_log_path = /dev/stdout disable_plaintext_auth = no first_valid_uid = 1000 info_log_path = /dev/stdout last_valid_uid = 1000 log_debug = event="oidc_shared_mailboxes_plugin" log_path = /dev/stdout login_greeting = Hello there! .... GIZ Mail! mail_location = maildir:~/mail mail_plugins = " oidc_shared_mailboxes" plugin { oidc_shared_mailboxes_client_id = dovecot-test } protocols = imap lmtp ``` --- Gesellschaft für interkulturelles Zusammenleben gGmbH (GIZ) Felix Auringer IT Reformationsplatz 2 13597 Berlin Tel: 030/513 0100 00; Fax: 030/513 0100 09 www.giz.berlin; felix.auringer(a)giz.berlin Amtsgericht Charlottenburg HRB 200872 B Geschäftsführerin: Dr. Britta Marschke

2 1

Error: Can't load SSL certificate
by Horst Simon 20 Mar '23

20 Mar '23

I try to logon to map using dovecot with SSL required. I get following entries in the log:Mar 20 13:49:30 auth: Debug: Loading modules from directory: /opt/lib/dovecot/modules/auth Mar 20 13:49:30 auth: Debug: Module loaded: /opt/lib/dovecot/modules/auth/lib20_auth_var_expand_crypt.so Mar 20 13:49:30 auth: Debug: Read auth token secret from /opt/var/run/dovecot/auth-token-secret.dat Mar 20 13:49:30 auth: Debug: passwd-file /opt/etc/dovecot/userdb:Read 1 users in 0 secs Mar 20 13:49:30 auth: Debug: auth client connected (pid=26120) Mar 20 13:49:30 imap-login: Error: Failed to initialize SSL server context: Can't load SSL certificate (ssl_cert setting): error:12800067:DSO support routines::could not load the shared library: filename(libz.so): libz.so: cannot open shared object file: No such file or directory, error:12800067:DSO support routines::could not load the shared library: user=<>, rip=192.168.117.5, lip=192.168.117.2, session=<eX3e+Uv3k8DAqHUF> Mar 20 13:49:30 imap-login: Info: Disconnected: TLS initialization failed. (no auth attempts in 0 secs): user=<>, rip=192.168.117.5, lip=192.168.117.2, session=<eX3e+Uv3k8DAqHUF The library files exist in the include directory [/opt] # find . -name libz* -print ./lib/opkg/info/libzstd.control ./lib/opkg/info/libzstd.list ./lib/libz.so ./lib/libz.so.1 ./lib/libz.so.1.2.13 ./lib/libzstd.so ./lib/libzstd.so.1 ./lib/libzstd.so.1.5.2 ./lib/libz.a Have the dovecot settings the same on other servers and it works, Dovecot is v2.3.18, installed from Entware for QNAP. Any help appreciated. Horst

2 1

Can't seem to setup remote access to doveadmI'm using
by dml＠plumeriahale.net 18 Mar '23

18 Mar '23

I’m running version 2.1.7 under Raspbian Wheezy (and have been for a number of years). I want to allow one of my other computers to remotely issue doveadm commands to my server but can’t seem to find the right way to add an inet listener to permit this. Using ‘doveconf -a’ I’ve found the default service definition for doveadm-server. So I copied that and added it to the dovecot configuration with an inet listener section added to it. But that was rejected as a duplicate service definition when I restarted dovecot. I then tried specifying a “host:port” value for the doveadm_socket_path value but that didn’t work as no listening socket at that port appeared when I restarted dovecot. I can’t seem to find in any of the dovecot documentation a way to do this and there seems to be nothing like a “Here’s how to setup doveadm remote access” section in the documentation (which would be most helpful). How can I get doveadm-server to listen not only locally (as it’s already doing) but also open an inet port for remote access?

3 2

Re: [EXT] Re: Corrupted sizes using zlib plugin
by Tim Evers 16 Mar '23

16 Mar '23

Am 06.03.23 um 16:45 schrieb Aki Tuomi: >> On 06/03/2023 16:52 EET Tim Evers <te-ml-ext(a)artfiles.de> wrote: >> >> >> Am 06.03.23 um 15:29 schrieb Aki Tuomi: >>>> On 06/03/2023 15:56 EET Tim Evers <te-ml-ext(a)artfiles.de> wrote: >>>> >>>> >>>> Am 06.03.23 um 14:49 schrieb Aki Tuomi: >>>>>> On 06/03/2023 15:45 EET Tim Evers <te-ml-ext(a)artfiles.de> wrote: >>>>>> >>>>>> >>>>>> Am 06.03.23 um 14:00 schrieb Aki Tuomi: >>>>>>>> On 06/03/2023 13:06 EET Tim Evers <te-ml-ext(a)artfiles.de> wrote: >>>>>>>> >>>>>>>> >>>>>>>> Am 06.03.23 um 11:59 schrieb Aki Tuomi: >>>>>>>> >>>>>>>>>> On 06/03/2023 12:44 EET Tim Evers <te-ml-ext(a)artfiles.de> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Am 06.03.23 um 11:42 schrieb Aki Tuomi: >>>>>>>>>> >>>>>>>>>>>> On 06/03/2023 12:32 EET Tim Evers <te-ml-ext(a)artfiles.de> wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> since I did not get any feedback on my bug report post regarding >>>>>>>>>>>> corrupted sizes while using zlib >>>>>>>>>>>> (https://dovecot.org/pipermail/dovecot/2023-February/126105.html) I >>>>>>>>>>>> would like to confirm that this bug report reached someone in charge. >>>>>>>>>>>> >>>>>>>>>>>> Or if not - I would kindly ask for directions on how to post it in a way >>>>>>>>>>>> that spawns some action. >>>>>>>>>>>> >>>>>>>>>>>> Thx >>>>>>>>>>>> >>>>>>>>>>>> Tim >>>>>>>>>>> Can you try erasing dovecot.index.cache and see if the problem fixes itself? It's possible your cache contains invalid values from old errors. >>>>>>>>>>> >>>>>>>>>>> Aki >>>>>>>>>> Did that already several times (other dovecot.index.* files too). The >>>>>>>>>> only thing that helps is actually deleting the dovecot-uidlist file but >>>>>>>>>> the problem returns. >>>>>>>>>> >>>>>>>>>> Tim >>>>>>>>> I think it's because the value on the filename is wrong. The filename contains the virtual and physical size. >>>>>>>>> >>>>>>>>> Can you try if https://github.com/dovecot/tools/blob/main/maildir-size-fix.pl fixes your issue? >>>>>>>>> >>>>>>>>> Aki >>>>>>>> Quoting >>>>>>>> >>>>>>>> https://doc.dovecot.org/configuration_manual/zlib_plugin/ : >>>>>>>> >>>>>>>> "All mails must have ,|S=<size>|in their filename where <size> contains >>>>>>>> the original uncompressed mail size, otherwise there will be problems >>>>>>>> with quota calculation as well as other potential random failures." >>>>>>>> >>>>>>>> And the script you send: >>>>>>>> >>>>>>>> "# Check if the files are compressed. Use the uncompressed size for S=size." >>>>>>>> >>>>>>>> Which it does in my example. S= contains the uncompressed size, not the >>>>>>>> size on disk. Is this not correct? All Files are written exclusively by >>>>>>>> dovecot, no other software is involved. >>>>>>>> >>>>>>>> Tim >>>>>>> Can you send output from `doveconf -n`? I am having suspicion that zlib plugin isn't always involved in your config, which causes the compressed file to be cached wrongly. Especially it feels like your IMAP protocol does not have compression on while LMTP does. >>>>>>> >>>>>>> Aki >>>>>> Actually it is the other way round (I removed zlib from lmtp to reduce >>>>>> the impact of the problem): >>>>>> >>>>>> # 2.3.20 (80a5ac675d): /etc/dovecot/dovecot.conf >>>>>> # Pigeonhole version 0.5.20 (149edcf2) >>>>>> # OS: Linux 4.15.0-204-generic x86_64 Ubuntu 18.04.6 LTS >>>>>> # Hostname: node2.mst27.artfiles.de >>>>>> auth_verbose = yes >>>>>> auth_worker_max_count = 1000 >>>>>> default_client_limit = 5000 >>>>>> dict { >>>>>> sqlquota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext >>>>>> } >>>>>> disable_plaintext_auth = no >>>>>> import_environment = TZ CLUSTERNAME >>>>>> lda_mailbox_autocreate = yes >>>>>> lda_mailbox_autosubscribe = yes >>>>>> mail_location = maildir:~/Maildir >>>>>> mail_plugins = " mail_log notify old_stats" >>>>>> mailbox_idle_check_interval = 10 mins >>>>>> maildir_broken_filename_sizes = yes >>>>>> managesieve_notify_capability = mailto >>>>>> managesieve_sieve_capability = fileinto reject envelope >>>>>> encoded-character vacation subaddress comparator-i;ascii-numeric >>>>>> relational regex imap4flags copy include variables body enotify >>>>>> environment mailbox date index ihave duplicate mime foreverypart >>>>>> extracttext editheader imapsieve vnd.dovecot.imapsieve >>>>>> namespace inbox { >>>>>> inbox = yes >>>>>> location = >>>>>> mailbox Drafts { >>>>>> auto = subscribe >>>>>> special_use = \Drafts >>>>>> } >>>>>> mailbox Junk { >>>>>> auto = subscribe >>>>>> special_use = \Junk >>>>>> } >>>>>> mailbox Sent { >>>>>> auto = subscribe >>>>>> special_use = \Sent >>>>>> } >>>>>> mailbox "Sent Messages" { >>>>>> special_use = \Sent >>>>>> } >>>>>> mailbox Trash { >>>>>> auto = subscribe >>>>>> special_use = \Trash >>>>>> } >>>>>> prefix = INBOX. >>>>>> separator = . >>>>>> } >>>>>> passdb { >>>>>> args = /etc/dovecot/dovecot-sql.conf.ext >>>>>> driver = sql >>>>>> } >>>>>> plugin { >>>>>> imapsieve_mailbox1_before = file:~/af_report-spam.sieve >>>>>> imapsieve_mailbox1_causes = COPY >>>>>> imapsieve_mailbox1_name = INBOX.Junk >>>>>> imapsieve_mailbox2_before = file:~/af_report-ham.sieve >>>>>> imapsieve_mailbox2_causes = COPY >>>>>> imapsieve_mailbox2_from = INBOX.Junk >>>>>> imapsieve_mailbox2_name = * >>>>>> quota = dict:User quota::proxy::sqlquota >>>>>> quota_rule = *:storage=0:messages=0 >>>>>> sieve = file:~/sieve;active=~/.dovecot.sieve >>>>>> sieve_before = file:~/af_master.sieve >>>>>> sieve_before2 = file:~/af_konto.sieve >>>>>> sieve_before3 = file:~/af_domain.sieve >>>>>> sieve_before4 = file:~/af_spamfilter.sieve >>>>>> sieve_extensions = +editheader >>>>>> sieve_filter_bin_dir = /usr/local/bin/sieve >>>>>> sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.filter >>>>>> +vnd.dovecot.environment >>>>>> sieve_max_redirects = 100 >>>>>> sieve_pipe_bin_dir = /usr/local/bin/sieve >>>>>> sieve_plugins = sieve_imapsieve sieve_extprograms >>>>>> stats_refresh = 30 secs >>>>>> zlib_save = gz >>>>>> zlib_save_level = 6 >>>>>> } >>>>>> pop3_no_flag_updates = yes >>>>>> protocols = " imap lmtp sieve pop3" >>>>>> service anvil { >>>>>> client_limit = 6203 >>>>>> } >>>>>> service auth-worker { >>>>>> process_limit = 1000 >>>>>> } >>>>>> service auth { >>>>>> client_limit = 6300 >>>>>> } >>>>>> service dict { >>>>>> process_limit = 500 >>>>>> unix_listener dict { >>>>>> group = vmail >>>>>> mode = 0600 >>>>>> user = vmail >>>>>> } >>>>>> } >>>>>> service imap-login { >>>>>> process_limit = 4000 >>>>>> process_min_avail = 0 >>>>>> service_count = 0 >>>>>> vsz_limit = 512 M >>>>>> } >>>>>> service imap { >>>>>> client_limit = 1 >>>>>> process_limit = 4000 >>>>>> vsz_limit = 1 G >>>>>> } >>>>>> service lmtp { >>>>>> inet_listener lmtp { >>>>>> address = 127.0.0.1 >>>>>> port = 10024 >>>>>> } >>>>>> process_limit = 80 >>>>>> process_min_avail = 20 >>>>>> vsz_limit = 2 G >>>>>> } >>>>>> service pop3-login { >>>>>> process_limit = 2000 >>>>>> service_count = 0 >>>>>> vsz_limit = 512 M >>>>>> } >>>>>> service pop3 { >>>>>> client_limit = 1 >>>>>> process_limit = 2000 >>>>>> vsz_limit = 1 G >>>>>> } >>>>>> service stats { >>>>>> fifo_listener stats-mail { >>>>>> mode = 0600 >>>>>> user = vmail >>>>>> } >>>>>> } >>>>>> ssl_cert = </etc/dovecot/private/dovecot.pem >>>>>> ssl_client_ca_dir = /etc/ssl/certs >>>>>> ssl_key = # hidden, use -P to show it >>>>>> submission_host = smtp4.artfiles.de >>>>>> syslog_facility = local5 >>>>>> userdb { >>>>>> args = /etc/dovecot/dovecot-sql.conf.ext >>>>>> driver = sql >>>>>> } >>>>>> protocol lmtp { >>>>>> mail_plugins = " mail_log notify old_stats sieve quota" >>>>>> } >>>>>> protocol imap { >>>>>> mail_max_userip_connections = 50 >>>>>> mail_plugins = " mail_log notify old_stats quota imap_quota >>>>>> imap_sieve zlib imap_old_stats" >>>>>> rawlog_dir = /var/log/rawlog/%u >>>>>> } >>>>>> protocol pop3 { >>>>>> mail_max_userip_connections = 50 >>>>>> mail_plugins = " mail_log notify old_stats zlib" >>>>>> } >>>>> It will probably compound on the issue though, especially when rebuilding caches and indexes, as it will record the sizes wrong. I'm not sure if this is *really* the issue, but it's most likely culprit. The 8192 is very odd number to have there, which either indicates that the zlib plugin is not consistently used, or you have some issues with the gz library itself. >>>>> >>>>> Aki >>>> Not sure if I understand this - shouldn't dovecot always note the >>>> decompressed size in cache and filename regardless if it's compressed or >>>> not? >>>> >>>> I just removed the zlib plugin from lmtp to reduce the footprint of the >>>> problem. The problem itself showed up first while lmtp zlib was on. >>>> >>>> Tim >>> It is complaining about the **physical** size though, that is, the amount stored on disk. If you have a dovecot process which is unaware of compression, it can record the wrong size for the file. >>> >>> Aki >> OK, I see. Another thing I noticed: The uidlist entry for a suspicious >> number of these problematic files contains what looks like a GUID: >> >> G1674674677.M641373P33277.node2 >> >> If I remove the uidlist and let dovecot recreate it, it is usually gone. >> Might that be an indicator of some other process writing to the uidlist >> file? the GUID is not mentioned in the dovecot docs as far as I can see. >> >> Tim > It sure sounds like that. > > Aki OK, for future reference: I think I found the issue. There was a periodic doveadm quotacheck job that was unaware of the zlib compression. I fixed that and no new corruption occured (though I still see errors from already corrupted cache data - that will hopefully dry up over time). However: I have still no idea why dovecot is complaining about the index.* files. It seems that the source of the wrong data is the dovecot-uidlist. Also I don't see why the quotacheck cmd (doveadm quota recalc -u <>) **sometimes** updates the dovecot-uidlist file. The quota itself is stored in a sql dict. Thanks to all who provided their input and especially Aki for the support! Tim

1 0