October 2020 - dovecot

Antispam plugin
by Maciej Milaszewski 17 Jan '21

17 Jan '21

Hi System centos8 + dovecot-2.3.8 from repo # 2.3.8 (9df20d2db): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.8 (b7b03ba2) # OS: Linux 4.18.0-193.19.1.el8_2.x86_64 x86_64 CentOS Linux release 8.2.2004 (Core) I need "Antispam plugin". What antispam-plugin I must use ? In older version dovecot-2.2.36.4 i use "dovecot-antispam-plugin" but in centos I had a problem with configure - probably toold wersion "./configure: line 3193: DC_DOVECOT: command not found" In wiki dovect is "antispam-plugin" I try this http://hg.dovecot.org/dovecot-antispam-plugin but get 404 Backand spamassassin

4 3

Dovecot Replication Errors (only) when using tcps: as the mail_replica Protocol
by Reuben Farrelly 18 Dec '20

18 Dec '20

Hi, I've been seeing errors logged for some time with replication processes, whereby replication sessions seem to be timing out periodically. This is with dovecot version 2.3.10.1 (a3d0e1171) and both are Gentoo x86_64. After some investigation I've determined that these timeouts are only ever occurring with tcps as the replication connection type. These errors never occur if non-encrypted tcp is configured. I've been able to validate this by changing only the replica_type on both ends of the replication configuration to tcp, and with no other changes and after a few days of operation there is not a single error logged. mail_replica = tcps:lightning.reub.net:4813 <<< periodic timeouts mail_replica = tcp:lightning.reub.net:4814 <<< works perfectly Example of the error: Jun 12 15:45:44 thunderstorm.reub.net dovecot[21149]: dsync-local(kaylene)<zx+WKTAU416UMwAAzkCIew>: Error: dsync(lightning.reub.net): I/O has stalled, no activity for 600 seconds (last sent=mailbox_delete, last recv=handshake) Jun 12 15:45:44 thunderstorm.reub.net dovecot[21149]: dsync-local(kaylene)<zx+WKTAU416UMwAAzkCIew>: Error: Timeout during state=recv_mailbox_tree doveadm: Error: Timeout during state=slave_recv_mailbox: 6 Time(s) doveadm: Error: Timeout during state=sync_mails (send=mail_requests recv=attributes): 31 Time(s) doveadm: Error: dsync(thunderstorm.reub.net): I/O has stalled, no activity for 600 seconds (last sent=mail_change (EOL), last recv=mailbox): 31 Time(s) doveadm: Error: dsync(thunderstorm.reub.net): I/O has stalled, no activity for 600 seconds (last sent=mailbox_delete, last recv=mailbox_delete): 6 Time(s) It is seen on both sides of the replication setup. The replica is offsite but only a few ms of latency away and there is no packet loss. The replication is happening over IPv6, and the local firewall is logging that sessions are always permitted, and only ever finishing due to tcp-fin or tcp-rst-from-client . SSL appears to be correctly configured, and it seems that the replication itself is for the most part working. Clients are able to use imaps just fine so I don't think there's anything much wrong from an SSL perspective else I'd be seeing complete replication failure and/or client devices unable to connect. Can anyone suggest how we can further debug this problem? Thanks, Reuben

4 3

Repo for ubuntu focal 20.04 LTS
by Sébastien Riccio 27 Nov '20

27 Nov '20

Hello Dovecot, As the new ubuntu LTS 20.04 seems to be out, I would like to ask if there are any plans to add a repo like it already exists for 18.04 LTS. We planned to upgrade our 18.04 LTS boxes to 20.04. It has it’s own dovecot packages (2.3.7 at the time of writing this), but I would prefer we use your repositories like we do with 18.04. Kind regards, Sébastien

4 4

Sieve body test
by ＠lbutlr 10 Nov '20

10 Nov '20

I have an email where I need to edit the body. I know this is generally a bad idea, but in this case I need to do it. The email comes in automatically every week or two, and so I thought that SIEVE would be the way to go. if header :contains "from" "theaddress@tehdomain" { if body :raw :contains "A string" { # Magic happens here } } It looks like what I need to do is enable and use vnd.dovecot.filter to pipe the message to a script. (Er. | not vnd.dovecot.pipe) So, if I wanted to transform the message so that the string "foobar13" was changed to "foo-bar-13" I would add filter :try "myfoobarfix.sh" After "magic happens here"> OK so far? Now, what are the requirements on the executable script? Would it literally be as simple as an executable file that simply said sed -e '|foobar13|foo-bar-13|g' Or does it need to be a shebang script and take input on $1 and … ? And lastly, where do the executable scripts need to be? https://raw.githubusercontent.com/dovecot/pigeonhole/master/doc/rfc/spec-bo… says "the external programs cannot be chosen arbitrarily; the available programs are restricted through administrator configuration", but no details on exactly how that is configured. -- "Are you pondering what I'm pondering?" Pinky: I think so, Brain. But if I put on two tutu's, would I really be wearing a four-by-four? Brain: Why do I even bother asking? Pinky: I dunno, Brain. Maybe it's all part of some huge, cosmic plot formula!

2 7

HIGHESTMODSEQ replication issue
by Gustavo Ajeitos [Office365] 09 Nov '20

09 Nov '20

Hello, I'm using dovecot 2.3.10 in OpenSUSE and I have configured a master/master replication between two backends, it works well but I have found an issue with the replication of the HIGHESTMODSEQ. When I move a set of message from a mailbox to other, e.g. move 3 messages from INBOX to Junk, the HIGHESTMODSEQ get different on each server, with no errors in the log: [Borrador] 34:34 [Sent] 63:63 [Drafts] 12:12 [Trash] 146:146 [Junk] 219:218 [INBOX] 1133:1130 It can be fixed executing a full sync: $ sudo doveadm -D -v sync -u user(a)test.priv -d -l 30 -U But it is not a solution because it happens all the time that a user move messages. I'm using ActiveSync and the ActiveSync server use the HIGHESTMODSEQ as part of the synchronization key (sync_key), so in that situation the server detect changes on the mailbox if it connects to the backend with a different HIGHESTMODSEQ that it previously cached, causing a loop beetween the ActiveSync server and the the email client, where the server constantly inform email client about nonexistent changes. Could it be a bug in Dovecot replication? How can I solve it? Best regards, Gustavo Ajeitos

4 7

doveadm sync usage ; root INBOX unsynced
by François Poulain 06 Nov '20

06 Nov '20

Hi, I am trying to import a mail IMAP account using doveadm. Following https://serverfault.com/questions/605342/migrating-from-any-imap-pop3-serve… I did it with: doveadm -D -v \ -o imapc_host=mail.oldserver.tld \ -o imapc_user=contact(a)domain.org \ -o imapc_password=xxxxxxx \ -o imapc_features=rfc822.size \ -o imapc_ssl=starttls \ -o ssl_client_ca_file=/etc/ssl/certs/ca-certificates.crt \ -o mail_fsync=never \ sync -1 -R -u contact(a)domain.org imapc: Btw I am facing with exactly the same error than: https://serverfault.com/questions/960264/doveadm-backup-doesnt-transfer-mai… I.e. it works fine for all subfolders, but the root INBOX stay empty on the new server. Those debug messages tell me something seems wrong with INBOX root but I am unable to understand them: dsync(contact(a)domain.tld): Debug: brain M: Local mailbox tree: INBOX guid=deadbeefdeabbeefdeadbeefdeabbeef uid_validity=1604137198 uid_next=1 subs=no last_change=0 last_subs=0 dsync(contact(a)domain.tld): Debug: brain S: Remote mailbox tree: INBOX guid=deadbeefdeabbeefdeadbeefdeabbeef uid_validity=1604137198 uid_next=1 subs=no last_change=0 last_subs=0 dsync(contact(a)domain.tld): Debug: brain M: Mailbox INBOX: local=deadbeefdeabbeefdeadbeefdeabbeef/0/1, remote=00000000000000000000000000000000/0/0: mailbox not selectable yet dsync(contact(a)domain.tld): Debug: brain S: Mailbox INBOX: local=00000000000000000000000000000000/0/0, remote=deadbeefdeabbeefdeadbeefdeabbeef/0/1: mailbox not selectable yet dsync(contact(a)domain.tld): Debug: brain S: Ignore nonexistent mailbox GUID deadbeefdeabbeefdeadbeefdeabbeef with -1 sync dsync(contact(a)domain.tld): Debug: brain S: We don't have mailbox deadbeefdeabbeefdeadbeefdeabbeef dsync(contact(a)domain.tld): Debug: brain M: Change during sync: Remote lost mailbox GUID deadbeefdeabbeefdeadbeefdeabbeef (maybe it was just deleted?) dsync(contact(a)domain.tld): Warning: Mailbox changes caused a desync. You may want to run dsync again: Remote lost mailbox GUID deadbeefdeabbeefdeadbeefdeabbeef (maybe it was just deleted?) Does someone have on hint? Best regards. François -- François Poulain <fpoulain(a)metrodore.fr>

3 4

Indexer error after upgrade to 2.3.11.3
by Alessio Cecchi 06 Nov '20

06 Nov '20

Hi, after the upgrade to Dovecot 2.3.11.3, from 2.3.10.1, I see frequently these errors from different users: Aug 18 11:02:35 Panic: indexer-worker(info(a)domain.com) session=<g71KISOttvS5LNVj:O3ahCyuZO18cYAAAEPCW+w>: file http-client-request.c: line 1232 (http_client_request_send_more): assertion failed: (req->payload_input != NULL) Aug 18 11:02:35 Error: indexer-worker(info(a)domain.com) session=<g71KISOttvS5LNVj:O3ahCyuZO18cYAAAEPCW+w>: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(backtrace_append+0x2f) [0x7f0ee3c828bf] -> /usr/lib64/dovecot/libdovecot.so.0(backtrace_get+0x26) [0x7f0ee3c829d6] -> /usr/lib64/dovecot/libdovecot.so.0(+0xeb7ba) [0x7f0ee3c8d7ba] -> /usr/lib64/dovecot/libdovecot.so.0(+0xeb801) [0x7f0ee3c8d801] -> /usr/lib64/dovecot/libdovecot.so.0(+0x42ff1) [0x7f0ee3be4ff1] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_request_send_more+0x415) [0x7f0ee3c2ba25] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_connection_output+0x114) [0x7f0ee3c30994] -> /usr/lib64/dovecot/libdovecot.so.0(+0x115470) [0x7f0ee3cb7470] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x55) [0x7f0ee3ca4eb5] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xdc) [0x7f0ee3ca6ebc] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x5c) [0x7f0ee3ca4fac] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f0ee3ca51f8] -> /usr/lib64/dovecot/libdovecot.so.0(+0x8a955) [0x7f0ee3c2c955] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_request_finish_payload+0x21) [0x7f0ee3c2cbd1] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(solr_connection_post_end+0x45) [0x7f0ee1c85d15] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(+0x3fa0) [0x7f0ee1c81fa0] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0x86cc) [0x7f0ee297f6cc] -> /usr/lib64/dovecot/lib20_fts_plugin.so(fts_backend_update_deinit+0x2c) [0x7f0ee297f74c] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0xfd04) [0x7f0ee2986d04] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0xff3f) [0x7f0ee2986f3f] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0xf64b) [0x7f0ee2dc764b] -> /usr/lib64/dovecot/lib01_acl_plugin.so(+0xde43) [0x7f0ee2fdce43] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit_get_changes+0x54) [0x7f0ee3f91db4] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit+0x16) [0x7f0ee3f91e76] -> dovecot/indexer-worker [info(a)domain.com INBOX](+0x291c) [0x557584acb91c] -> dovecot/indexer-worker [info(a)domain.com INBOX](+0x2e54) [0x557584acbe54] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x55) [0x7f0ee3ca4eb5] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xdc) [0x7f0ee3ca6ebc] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x5c) [0x7f0ee3ca4fac] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f0ee3ca51f8] Aug 18 11:02:35 Error: indexer: Indexer worker disconnected, discarding 1 requests for info(a)domain.com Aug 18 11:02:35 Error: imap(info(a)domain.com) session=<g71KISOttvS5LNVj>: indexer failed to index mailbox INBOX Aug 18 11:02:35 Fatal: indexer-worker(info(a)domain.com) session=<g71KISOttvS5LNVj:O3ahCyuZO18cYAAAEPCW+w>: master: service(indexer-worker): child 24604 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) I'm using FTS with Solr 6.6.5. What is it? Thanks -- Alessio Cecchi Postmaster @http://www.qboxmail.it https://www.linkedin.com/in/alessice

9 25

v2.3.11.3 solr plugin search via MUA fails to match accented ascii characters; cmd line exec of `doveadm fts lookup` PANICs (assertion failed)
by PGNet Dev 02 Nov '20

02 Nov '20

I'm running, dovecot --version 2.3.11.3 (502c39af9) solr -version 8.6.3 uname -rm 5.8.13-200.fc32.x86_64 x86_64 grep _NAME /etc/os-release PRETTY_NAME="Fedora 32 (Server Edition)" CPE_NAME="cpe:/o:fedoraproject:fedora:32" Solr FTS plugin is enabled/configured, mail_plugins = virtual acl fts fts_solr plugin { fts = solr fts_autoindex = yes fts_solr = url=https://solr.example.com:8984/solr/dovecot/ fts_enforced = body fts_filters = normalizer-icu stopwords snowball fts_language_config = /usr/share/libexttextcat/fpdb.conf fts_languages = en es de fr it pt soft_commit = yes } IMAP capability returns, a OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY PREVIEW=FUZZY STATUS=SIZE SAVEDATE SPECIAL-USE LITERAL+ NOTIFY SPECIAL-USE QUOTA ACL RIGHTS=texk] Logged in I've got two messages in my IMAP store, cd /data/vmail/example.com/myuser/Maildir/cur/ ls -altr | grep S= | /bin/tail -n2 -rw------- 1 vmail vmail 1.3K Oct 11 14:05 1602450306.M393628P65260.mx.example.com,S=1278,W=1304:2,S -rw------- 1 vmail vmail 1.3K Oct 11 14:05 1602450353.M756184P65260.mx.example.com,S=1277,W=1303:2,S that differ in BODY CONTENT -- -- one message has ascii txt with NO character accents -- the other has the same text, but with ON character accent cat "1602450306.M393628P65260.mx.example.com,S=1278,W=1304:2,S" ... From: M User <myuser(a)example.com> Subject: test Reply-To: myuser(a)example.com To: "User, My" <myuser(a)example.com> Message-ID: <6fc7ac30-b460-7dd4-f85d-ca4403ad7188(a)example.com> Date: Sun, 11 Oct 2020 14:05:06 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit !!!! también cat 1602450353.M756184P65260.mx.example.com,S=1277,W=1303:2,S ... From: M User <myuser(a)example.com> Subject: test Reply-To: myuser(a)example.com To: "User, My" <myuser(a)example.com> Message-ID: <015b3fb4-46f9-87cc-d541-060db0a13086(a)example.com> Date: Sun, 11 Oct 2020 14:05:53 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit !!!! tambien i manually re-scan & index doveadm fts rescan -u myuser(a)example.com doveadm index -u myuser(a)example.com -q '*' ... ==> /var/log/dovecot/dovecot-info.log <== 2020-10-11 15:06:34 indexer-worker(myuser(a)example.com)<OyUmLeqBg18fDAEA+IOfAw>: Info: Indexed 21 messages in accts (UIDs 14399..130699) 2020-10-11 15:06:34 indexer-worker(myuser(a)example.com)<6NnOMuqBg18fDAEA+IOfAw>: Info: Indexed 16 messages in accts/v007132 (UIDs 13414..14778) ... with no errors. then search in mail client, here TBird 78, with [X] Run Search on Server for _un_accented "tambien", match is correctly -- and quickly -- returned. in logs, ==> /var/log/dovecot/dovecot-info.log <== 2020-10-11 14:57:05 imap-login: Info: Login: user=<myuser(a)example.com>, method=PLAIN, rip=10.0.1.7, lip=10.0.1.50, mpid=67743, TLS 2020-10-11 14:57:16 indexer-worker(myuser(a)example.com)<3ZUzQ2yx2JKsHgsH:9gu0MbF/g1+hCAEA+IOfAw>: Info: Indexed 4788 messages in INBOX (UIDs 135476..140263) BUT, repeating search for ACCENTED "también" returns *no* match/result. No errors in log, simply no match. Attempting to test/debug from from cmd line, doveadm fts lookup -u myuser(a)example.com body "tambien" causes a PANIC doveadm(myuser(a)example.com): Panic: file mail-storage.c: line 2112 (mailbox_get_open_status): assertion failed: (box->opened) doveadm(myuser(a)example.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(backtrace_append+0x46) [0x7f3ee94accc6] -> /usr/lib64/dovecot/libdovecot.so.0(backtrace_get+0x22) [0x7f3ee94acde2] -> /usr/lib64/dovecot/libdovecot.so.0(+0x10025b) [0x7f3ee94b625b] -> /usr/lib64/dovecot/libdovecot.so.0(+0x100297) [0x7f3ee94b6297] -> /usr/lib64/dovecot/libdovecot.so.0(+0x59bc6) [0x7f3ee940fbc6] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x4779e) [0x7f3ee95c379e] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(+0x5849) [0x7f3ee9015849] -> /usr/lib64/dovecot/lib20_fts_plugin.so(fts_backend_lookup+0x51) [0x7f3ee8c37491] -> /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so(+0x3280) [0x7f3ee8ba9280] -> doveadm(+0x343cd) [0x5637e99443cd] -> doveadm(+0x34fe0) [0x5637e9944fe0] -> doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x22d) [0x5637e9945e2d] -> doveadm(doveadm_cmd_run_ver2+0x4e8) [0x5637e99568d8] -> doveadm(doveadm_cmd_try_run_ver2+0x3e) [0x5637e995692e] -> doveadm(main+0x1d4) [0x5637e9934cf4] -> /lib64/libc.so.6(__libc_start_main+0xf2) [0x7f3ee9071042] -> doveadm(_start+0x2e) [0x5637e99351ce] Aborted (1) What config -- dovecot &/or solr -- is needed to match on accented characters? (2) What add'l detail, if any, is needed for troubleshooting the panic?

5 37

sieve-extdata plugin breaks on pigeonhole 0.5 ish
by Ed W 01 Nov '20

01 Nov '20

Hi, I wonder if someone could give me a little help bringing the sieve-extdata plugin up to date to be usable with latest Dovecot/pigeonhole Trying to use both together at present I get an error: managesieve: Fatal: Couldn't load required plugin /usr/lib/dovecot/sieve/lib90_sieve_extdata_plugin.so: dlopen() failed: /usr/lib/dovecot/sieve/lib90_sieve_extdata_plugin.so: undefined symbol: sieve_sys_error This appears to be due to the change in error handling in pigeonhole 0.5 which appears to have removes the "sieve_sys_error()" and "sieve_sys_warning()" functions I'm unclear how this code should be change (I've commented it out for now - eek). I find this extension useful to pass in user specific parameters to my sieve scripts that are stored elsewhere in an sql database, eg spam score, size threshold for attachments, perhaps there is some other way to achieve this? Any hints on how to fixing this would be appreciated Thanks Ed W

3 2

Dovecot, sa-learn and sieve - where to save the Bayes DB
by Francis Augusto Medeiros-Logeay 31 Oct '20

31 Oct '20

Hi! I have dovecot, Spamassassin and postfix running. I started to use sa-learn to get better and more precise spam identification and filtering. I have not set the Bayes_path preference on my local.cf file, thus when I call sa-learn -u myuser(a)mydomain.org --spam, the Bayes DB gets updated at /root/.spamassassin. I decided then to create a sieve filter on dovecot, as described on Dovecot's wiki page. Basically, Dovecot's sieve plugin execute sa-learn with a -u argument, apparently the same way I do when I use it manually. BUT, the thing is that when Dovecot executes sa-learn, the bayes database is created on the virtual user's folder, ie., /var/mail/vmail/mydomain.org/user/.spamassassin. When I call it manually, the db used is /root/.spamassassin` I am now confused as I don't know if, when evaluating spam, spamd will use the Bayes DB on the user's Maildir, or if it will use the one under /root/.spamassassin. I would really like to have my spam evaluation done per user, not site wide, so I wonder: - How can I configure bayes_path for each user, individually, on a virtual users setup? - Does calling spamd with --virtual-config-dir have anything to do with this? I tried it, setting it up with my virtual users path, but it didn't seem to make any difference when it comes to what I describe in this post. With or without it set up, the results above are the same. -- Francis Augusto Medeiros-Logeay Oslo, Norway

1 0