October 2020 - dovecot

Dovecot, sa-learn and sieve - where to save the Bayes DB
by Francis Augusto Medeiros-Logeay 31 Oct '20

31 Oct '20

Hi! I have dovecot, Spamassassin and postfix running. I started to use sa-learn to get better and more precise spam identification and filtering. I have not set the Bayes_path preference on my local.cf file, thus when I call sa-learn -u myuser(a)mydomain.org --spam, the Bayes DB gets updated at /root/.spamassassin. I decided then to create a sieve filter on dovecot, as described on Dovecot's wiki page. Basically, Dovecot's sieve plugin execute sa-learn with a -u argument, apparently the same way I do when I use it manually. BUT, the thing is that when Dovecot executes sa-learn, the bayes database is created on the virtual user's folder, ie., /var/mail/vmail/mydomain.org/user/.spamassassin. When I call it manually, the db used is /root/.spamassassin` I am now confused as I don't know if, when evaluating spam, spamd will use the Bayes DB on the user's Maildir, or if it will use the one under /root/.spamassassin. I would really like to have my spam evaluation done per user, not site wide, so I wonder: - How can I configure bayes_path for each user, individually, on a virtual users setup? - Does calling spamd with --virtual-config-dir have anything to do with this? I tried it, setting it up with my virtual users path, but it didn't seem to make any difference when it comes to what I describe in this post. With or without it set up, the results above are the same. -- Francis Augusto Medeiros-Logeay Oslo, Norway

1 0

Odd replication behaviour
by James Pattinson 31 Oct '20

31 Oct '20

Hi, I have just built a new pair of similar machines both running CentOS 8.2 (selinux disabled) and Dovecot 2.3.8 (9df20d2db). One machine is a VPS (host A) and one is on my home network (host B). The idea is that they are set up in a master/master config with Dovecot replication. I seem to have this 95% working but there is one strange issue I can’t work out. Currently B is a perfect replica of A. I have pointed an instance of Thunderbird at it, and I can see all my mails. If I delete any mails or change any flags, I see the same changes almost instantly on the A side. PROBLEM: if host A receives a new mail, I don’t see it on B until I do ‘something’ to change metadata, for example deleting any random email, or marking an email as read on EITHER side causes the new email to appear almost instantly on the B side. I would have expected emails on B to appear immediately. Am I doing something wrong? Extra info - my mailboxes are in Maildir format with single OS user (vmail). I have about 4000 emails in the Inbox and about 30k in other folders. There are only 5 users and I’m using passdb as the very simple backend. Replication is via doveadm on a specified port (not SSH). Some output from dovecot -n is below. Cheers James HOST A # 2.3.8 (9df20d2db): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.8 (b7b03ba2) # OS: Linux 4.18.0-193.28.1.el8_2.x86_64 x86_64 CentOS Linux release 8.2.2004 (Core) xfs # Hostname: hosta.domain auth_mechanisms = plain login doveadm_password = # hidden, use -P to show it doveadm_port = 4040 first_valid_uid = 1000 mail_debug = yes mail_home = /srv/vmail/%u mail_location = maildir:/srv/vmail/%u mail_plugins = notify replication managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = scheme=BLF-CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { mail_replica = tcp:b.b.b.b:4040 sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /var/mail/SpamToJunk.sieve } protocols = imap lmtp service aggregator { fifo_listener replication-notify-fifo { group = root mode = 0660 user = vmail } unix_listener replication-notify { group = root mode = 0660 user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0600 user = postfix } } service doveadm { inet_listener { port = 4040 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 user = vmail } } ssl = required ssl_cert = </etc/letsencrypt/live/mail2.pattinson.org/fullchain.pem ssl_cipher_list = PROFILE=SYSTEM ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 ssl_prefer_server_ciphers = yes userdb { args = username_format=%u /etc/dovecot/users default_fields = uid=vmail gid=mail home=/srv/vmail/%u driver = passwd-file } protocol lmtp { mail_plugins = sieve } protocol lda { mail_plugins = notify replication sieve } HOST B # 2.3.8 (9df20d2db): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.8 (b7b03ba2) # OS: Linux 4.18.0-193.28.1.el8_2.x86_64 x86_64 CentOS Linux release 8.2.2004 (Core) ext4 # Hostname: hostb auth_mechanisms = plain login doveadm_password = # hidden, use -P to show it doveadm_port = 4040 first_valid_uid = 1000 mail_debug = yes mail_home = /srv/vmail/%u mail_location = maildir:/srv/vmail/%u mail_plugins = notify replication managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = scheme=BLF-CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { mail_replica = tcp:a.a.a.a:4040 sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /var/mail/SpamToJunk.sieve } protocols = imap lmtp service aggregator { fifo_listener replication-notify-fifo { group = root mode = 0660 user = vmail } unix_listener replication-notify { group = root mode = 0660 user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0600 user = postfix } } service doveadm { inet_listener { port = 4040 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 user = vmail } } ssl = required ssl_cert = </etc/letsencrypt/live/mail2.pattinson.org/fullchain.pem ssl_cipher_list = PROFILE=SYSTEM ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 ssl_prefer_server_ciphers = yes userdb { args = username_format=%u /etc/dovecot/users default_fields = uid=vmail gid=mail home=/srv/vmail/%u driver = passwd-file } protocol lmtp { mail_plugins = sieve } protocol lda { mail_plugins = notify replication sieve }

1 1

Sieve filter script EXECUTION FAILED
by ＠lbutlr 31 Oct '20

31 Oct '20

So, I have the sieve working up to the point that it calls the script, and the script is called (I get different errors if the script is not there, for example). filter action execute program `darkmode.sh' [[EXECUTION ABORTED]] There's no other error logged (for example, when I had sieve_filter_bin_dir miss-set, I would get: sieve: action filter: failed to execute program `darkmode.sh': vnd.dovecot.filter extension is unconfigured In /var/log/messages, but I fixed that. I assume there is something wrong with what I am doing in the script, bit that is something I am not finding any examples on that are proving useful. The script file is simply: Darkmode.sh: #!/bin/sh echo $1 | sed -e '|</head>|<style>* {color:white !important; background-color: black !important; } </style></head>|' Which is somehwat similar to a working script for reporting spam: sa-learn-spam.sh: #!/bin/sh exec /usr/local/bin/sa-learn -u ${1} --spam -- "Are you pondering what I'm pondering?" "I think so, Brain, but won't it go straight to my hips?!"

5 8

Re: Sieve filter script EXECUTION FAILED
by Aki Tuomi 30 Oct '20

30 Oct '20

> On 30/10/2020 19:25 Bernd Petrovitsch <bernd(a)petrovitsch.priv.at> wrote: > > > On 30/10/2020 17:11, @lbutlr wrote: > [...] > > echo $1 | sed -e '|</head>|<style>* {color:white !important; background-color: black !important; } </style></head>|' > > What should the sed stuff do? > TTBOMK '|' is not known by sed ... > > MfG, > Bernd sed accepts | as alternative for / as boundary. But I think the sed here is missing 's' from start, so this does not actually do anything... Aki

5 5

Re: Sieve filter script EXECUTION FAILED
by Aki Tuomi 30 Oct '20

30 Oct '20

> On 30/10/2020 20:34 @lbutlr <kremels(a)kreme.com> wrote: > > > On 30 Oct 2020, at 11:57, Aki Tuomi <aki.tuomi(a)open-xchange.com> wrote: > > But I think the sed here is missing 's' from start, so this does not actually do anything... > > Copy/paste/edit error. The s is there in the file. > > darkmode.sh: > #!/bin/sh > echo $1 | sed -e 's|</head>|<style>* {color:white !important; background-color: black !important; } </style></head>|' > > I am not sure about the $1. I think filter just pipes the message (or part of the message. > > I will see what happens without the echo I suppose. > > Nope, still the same. > > 32: starting `:contains' match with `i;ascii-casemap' comparator: > 32: matching value `<!DOCTYPE html><html lang="en"><head><title>29-Oct-2020 ""</title> <style>?? ...' > 32: with key `</head>' => 1 > 32: finishing match with result: matched > 32: jump if result is false > 32: not jumping > 34: filter action > 34: execute program `darkmode.sh' > 34: [[EXECUTION ABORTED]] > > -- > They looked at the drinks. > They drank the drinks. I am pretty sure that you are holding filter wrong, it expects you to process the entire mail, not just $1. Aki

1 0

how to enable automatic sharing of folders added/created in an already shared folder hiearchy?
by PGNet Dev 30 Oct '20

30 Oct '20

I'm running dovecot --version 2.3.11.3 (502c39af9) folder sharing is enabled & working. as long as I manually enable enable sharing for each folder/subfolder. e.g., if I enable sharing for each of: sharedtopfolder subfolder1 subfolder2 then all's good -- shared-to users can access according to set permissions. BUT, if I drag any folder under that shared-hierarchy e.g., sharedtopfolder subfolder1 subfolder2 + subfolder3 + subfolder4 the just-added subfolder3 subfolder4 are NOT shared. They CAN be -- manually. And, as soon as they are, they're, again, usable by target users. I'd like to have any/all newly-created, or dragged-in, folders under the already-shared "sharedtopfolder" to be **automatically** enabled for sharing. how is that explicitly setup? is it 'just' Dovecot config? &/or does it require some Sieve?

1 0

Sieve matching the raw body
by ＠lbutlr 30 Oct '20

30 Oct '20

I have tried the following to match the string '</head>' in the body of an email: if body :raw :contains "</head>" { if body :content "text/html" :contains "</head>" { if body :contains "</head>" { All three result in matching value `' with key `</head>' => 0 finishing match with result: not matched RFC 5173 says: 5.1. Body Transform ":raw" The ":raw" transform matches against the entire undecoded body of a message as a single item. Which seems like exactly what I want. :content does not seem like what I want since it appears to only match the non HTML code content of the mime part. Eventually what I want to do is to alert some of the code in the HTML portion the email. Failing that, to remove the HTML mime part entirely, but that is less desirable. NB: I am not getting sieve.log errors the code seems to parse fine, it's just not matching how I expect. -- I'm the High King of Fillory. I took a blood test.

1 1

Filtering mails with sieve filters
by Odhiambo Washington 30 Oct '20

30 Oct '20

Hello good people, I have dovecot- 2.3.7.2 (Ubuntu Ubuntu 20.04.1 LTS) running fine. However, I have a hitch with mail filtering. Using SOGo as the webmail UI, I created a filter rule which resulted into the rule below: require ["fileinto"]; if allof (address :contains "from" "odhiambo") { fileinto "INBOX/Odhiambo"; } However, the test mail I sent did not end up the folder expected. lda.log contains this info: Oct 30 07:39:07 lda(wash@mydomain)<2415266><tSKSHhvDm1+i2iQA9H6eMA>: Info: sieve: msgid=<CAAdA2WNqXpG1qmOmKp_pJPS6iQrEPeS6=vM_= o8UnvR152x54Q(a)mail.gmail.com>: stored mail into mailbox 'INBOX' Oct 30 07:39:07 lda(wash@mydomain)<2415266><tSKSHhvDm1+i2iQA9H6eMA>: Info: sieve: Execution of script /var/spool/virtual/mydomain/wash/.dovecot.sieve failed, but implicit keep was successful (user logfile /var/spool/virtual/mydomain/wash/.dovecot.sieve.log may reveal additional details) And the .dovecot.sieve.log contains: sieve: info: started log at Oct 30 07:39:07. error: msgid=<CAAdA2WNqXpG1qmOmKp_pJPS6iQrEPeS6=vM_= o8UnvR152x54Q(a)mail.gmail.com>: failed to store into mailbox 'INBOX/Odhiambo': *Invalid mailbox name: Name must not have '/' characters.* What might be the problem here?? dovecot -n output is below: # 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7.2 () # OS: Linux 5.4.0-29-generic x86_64 Ubuntu 20.04.1 LTS ext4 # Hostname: jp.mydomain auth_mechanisms = plain login auth_socket_path = /var/run/dovecot/auth-userdb debug_log_path = /var/log/dovecot/debug.log default_login_user = dovecot default_vsz_limit = 2 G disable_plaintext_auth = no first_valid_uid = 110 hostname = gw.mydomain info_log_path = /var/log/dovecot/info.log log_path = /var/log/dovecot/error.log mail_gid = Debian-exim mail_location = maildir:/var/spool/virtual/%d/%n/Maildir:INDEX=/var/spool/virtual/indexes/%d/%1n/%n mail_plugins = fts mail_privileged_group = mail mail_server_admin = postmaster@mydomain mail_uid = Debian-exim managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { fts_autoindex = yes fts_autoindex_exclude = \Junk fts_autoindex_exclude2 = \Trash fts_autoindex_exclude3 = \Drafts fts_autoindex_exclude4 = \Spam fts_enforced = yes imapsieve_mailbox1_before = file:/usr/lib/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Spam imapsieve_mailbox2_before = file:/usr/lib/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Spam imapsieve_mailbox2_name = * plugin = fts managesieve sieve sieve = file:~/sieve;active=~/.dovecot.sieve sieve_execute_bin_dir = /usr/lib/dovecot/sieve-execute sieve_filter_bin_dir = /usr/lib/dovecot/sieve-filter sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment sieve_pipe_bin_dir = /usr/lib/dovecot/sieve sieve_plugins = sieve_imapsieve sieve_extprograms } postmaster_address = postmaster@mydomain protocols = imap lmtp sieve service auth { unix_listener auth-client { mode = 0600 user = Debian-exim } unix_listener auth-userdb { group = Debian-exim user = Debian-exim } } service stats { unix_listener stats-reader { user = Debian-exim } unix_listener stats-writer { user = Debian-exim } } service submission-login { inet_listener submission { port = 2587 } } ssl_cert = </etc/letsencrypt/live/mydomain/fullchain.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { info_log_path = /var/log/dovecot/lmtp.log log_path = /var/log/dovecot/lmtp-errors.log mail_plugins = fts sieve postmaster_address = postmaster@mydomain } protocol lda { info_log_path = /var/log/dovecot/lda.log log_path = /var/log/dovecot/lda-errors.log mail_plugins = fts sieve } -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)

2 3

doveadm replicator, what decides the timing on a full resync?
by Norman King 30 Oct '20

30 Oct '20

Hello. If running dovadm replicator status, i get something like the following. root@mail2:~# doveadm replicator status Queued 'sync' requests 0 Queued 'high' requests 0 Queued 'low' requests 0 Queued 'failed' requests 0 Queued 'full resync' requests 4 Waiting 'failed' requests 0 Total number of known users 1000 When would these full resyncs happen? Is it on a schedule? and if so where is said schedule configured? doveadm replicator dsync-status shows plenty of available sessions. Thanks. -- regards Norman King Compass Foundation sysadmin/tech support Phone, 856-974-5335 ext 221 support(a)compassfoundation.io <mailto:support@compassfoundation.io> Website <https://compassfoundation.io>

1 0

When is the Dovecot 2.3.12 release due?
by Sidsel Jensen 30 Oct '20

30 Oct '20

Hi We are very eager to test the zstd compression in the upcoming Dovecot 2.3.12 release - any pointer as to when that is expected to be released? Kind Regards, Sidsel

1 0