Dovecot with Sieve unexpectedly flagging messages as \\Seen \\Deleted
Hello,
I would like to report that I am facing in essence the same Dovecot behavior as reported in the following thread:
https://dovecot.org/mailman3/archives/list/dovecot%40dovecot.org/thread/N6BF...
I am running (on CentOS 7, still):
dovecot23-lua-2.3.21-1.gf.el7.x86_64 dovecot23-devel-2.3.21-1.gf.el7.x86_64 dovecot23-2.3.21-1.gf.el7.x86_64 dovecot23-pigeonhole-2.3.21-1.gf.el7.x86_64
After almost running insane trying to resolve the issue, I found (through ChatGPT) the above historical report which matches my situation perfectly:
- Postfix delivering through Dovecot LMTP.
- Maildir storage.
- Multiple recipients expanded from a role/group alias.
- Only some recipients affected.
- The same message sometimes affected and sometimes not.
- Messages already marked \Seen \Deleted when saved.
- No corresponding flag_change event.
- No Personal Sieve scripts that set these flags (actually no personal sieve scripts at all).
Here is an excerpt from the logs (real mail addresses have been changed consistently):
... Nov 06 13:26:26 lmtp(user1)<25404><iELJNF+57Wo8YwAAcV+qjQ:R31>: Info: save: box=INBOX, uid=70160, msgid=<023901dd6de1$a5bdee20$f139ca60$@noa.gr>, size=11180, vsize=11406, from="Department Director" <techdptdir@noa.gr>, subject=Obfuscated_Subject_Text..., flags=(\Deleted \Seen) Nov 06 13:26:26 lmtp(user1)<25404><iELJNF+57Wo8YwAAcV+qjQ:R31>: Info: sieve: msgid=<023901dd6de1$a5bdee20$f139ca60$@noa.gr>, storage_id=1793964383.M932062P25404.vmail2.noa.gr,S=11180,W=11406 Nov 06 13:26:26 lmtp(user2)<24263><eNnTNF+57WrHXgAAcV+qjQ:R28>: Info: save: box=INBOX, uid=1668, msgid=<023901dd6de1$a5bdee20$f139ca60$@noa.gr>, size= 11180, vsize=11406, from="Department Director" <techdptdir@noa.gr>, subject=Obfuscated_Subject_Text..., flags=() Nov 06 13:26:26 lmtp(user2)<24263><eNnTNF+57WrHXgAAcV+qjQ:R28>: Info: sieve: msgid=<023901dd6de1$a5bdee20$f139ca60$@noa.gr>, storage_id=1793964383.M92 9906P24263.vmail2.noa.gr,S=11180,W=11406 ...
In postfix log, at the same time, the above messages were delivered as shown (real mail addresses have been changed consistently):
... Nov 06 13:26:28 vmail2 postfix/lmtp[22903]: 7330881E1B641: to=<user1@noa.gr>, orig_to=<tech_dpt@noa.gr>, relay=vmail2.noa.gr[private/dovecot-lmtp], delay=5.2, delays=0.32/0.1/0.002/4.8, dsn=2.0.0, status=sent (250 2.0.0 <user1@noa.gr> iELJNF+57Wo8YwAAcV+qjQ:R31 Saved) ... Nov 06 13:26:28 vmail2 postfix/lmtp[24262]: 7330881E1B641: to=<user2@noa.gr>, orig_to=<tech_dpt@noa.gr>, relay=vmail2.noa.gr[private/dovecot-lmtp], delay=5.4, delays=0.32/0.1/0.001/5, dsn=2.0.0, status=sent (250 2.0.0 <user2@noa.gr> eNnTNF+57WrHXgAAcV+qjQ:R28 Saved) ...
So, most users (expanded from an ldap-based group alias) receive mail like user2, i.e. with flags=(), while very few (not the same every time) receive mail like user1, i.e. with flags=(\Deleted \Seen). No personal sieve scripts exist for any of these users.
As a workaround, I decided I should use the solution described in that thread:
- Create a global Sieve script:
/etc/dovecot/sieve/clear-delivery-flags.sieve require ["imap4flags"]; removeflag "\\Deleted"; removeflag "\\Seen";
Compile it: sievec /etc/dovecot/sieve/clear-delivery-flags.sieve
Add to config:
plugin { sieve_before = /etc/dovecot/sieve/clear-delivery-flags.sieve }
- Validate and reload: doveconf -n systemctl reload dovecot
At the end of this mail, I post my config (and dovecot -n) for your reference.
I know that versions 2.3.x are EOL and I have not managed to upgrade/test on 2.4.x versions, but nevertheless I thought I should report it here, just in case some obscure bug is surviving across versions and has managed to reach into 2.4.x versions.
It is quite strange that the exact same behavior seems to be observed (under some unclear circumstances) in both v2.1.7 and v2.3.21.
If you would like me to do any tests and/or provide additional data, I would be happy to do so.
(My only reservation is that this server is I/O overloaded as it's a VPS with a slow/overloaded storage backend and I would avoid to run debugging scripts that might deteriorate performance even more.)
Best regards, Nick
Dovecot config file (dovecot.conf):
================================================================
protocols = imap pop3 sieve lmtp
login_greeting = Dovecot NOA ICXC-NIKA
log_path = /var/log/dove.log
mail_location = maildir:~/Maildir/
mail_gid = 500 mail_uid = 500
auth_mechanisms = plain login auth_username_format = %Ln
auth_cache_size = 10M auth_cache_ttl = 1 hour auth_cache_negative_ttl = 5 mins
auth_verbose = no auth_debug = no mail_debug = no
disable_plaintext_auth = no
deliver_log_format = msgid=%m, storage_id=%{storage_id}
mail_plugins = quota mail_log notify
protocol imap { imap_client_workarounds = "delay-newmail"
mail_plugins = $mail_plugins imap_quota
mail_max_userip_connections = 400
namespace inbox { mailbox Trash { autoexpunge = 15d } } }
protocol pop3 { mail_max_userip_connections = 3
mail_plugins = $mail_plugins
pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv
namespace inbox { mailbox Trash { autoexpunge = 15d } } }
protocol lmtp { auth_socket_path = /var/run/dovecot/auth-master postmaster_address = sysadm@noa.gr
mail_plugins = $mail_plugins sieve
sendmail_path = /usr/lib/sendmail }
protocol sieve { managesieve_max_line_length = 65536 mail_max_userip_connections = 10 managesieve_logout_format = bytes=%i/%o
managesieve_max_compile_errors = 10 }
userdb { args = /etc/dovecot/dovecot-usrdb-ldap.conf driver = ldap }
passdb { args = /etc/dovecot/dovecot-passdb-ldap.conf driver = ldap }
plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change save mailbox_create mail_log_fields = uid box msgid size flags vsize from subject
quota = maildir:User quota quota_rule = *:storage=15G quota_rule2 = Trash:storage=+3%% quota_warning = storage=75%% quota-warning 75 %u quota_warning2 = storage=90%% quota-warning 90 %u
sieve = file:~/sieve;active=~/.dovecot.sieve sieve_max_script_size = 0 sieve_max_actions = 0 sieve_max_redirects = 2
sieve_before = /etc/dovecot/sieve/clear-delivery-flags.sieve
sieve_trace_debug = no }
service quota-warning { executable = script /opt/mail1.sh user = vmail unix_listener quota-warning { user = vmail } }
service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = vmail mode = 0660 user = vmail } user = root }
service imap-login { service_count = 100 vsz_limit = 64 M process_limit = 500 }
service pop3-login { service_count = 100 vsz_limit = 64 M }
service managesieve-login { inet_listener sieve { port = 4190 }
service_count = 100 process_min_avail = 0 vsz_limit = 64M }
service managesieve { process_limit = 1024 }
service imap { process_limit = 300 vsz_limit = 512 M }
service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } }
ssl = yes
ssl_cert = </etc/pki/tls/certs/cert.pem ssl_key = </etc/pki/tls/private/cert.key
namespace inbox {
separator = . prefix = inbox = yes
mailbox Drafts { special_use = \Drafts auto = subscribe } mailbox Junk { special_use = \Junk auto = subscribe } mailbox Trash { special_use = \Trash auto = subscribe } mailbox Sent { special_use = \Sent auto = subscribe } }
======================== dovecot -n ==========================
dovecot -n
2.3.21 (47349e2482): /etc/dovecot/dovecot.conf
Pigeonhole version 0.5.21 (f6cd4b8e)
OS: Linux 3.10.0-1160.119.1.el7.x86_64 x86_64 CentOS Linux release
7.9.2009 (Core)
Hostname: vmail2.noa.gr
auth_cache_negative_ttl = 5 mins auth_cache_size = 10 M auth_mechanisms = plain login auth_username_format = %Ln deliver_log_format = msgid=%m, storage_id=%{storage_id} disable_plaintext_auth = no log_path = /var/log/dove.log login_greeting = Dovecot NOA ICXC-NIKA mail_gid = 500 mail_location = maildir:~/Maildir/ mail_plugins = quota mail_log notify mail_uid = 500 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = . } passdb { args = /etc/dovecot/dovecot-passdb-ldap.conf driver = ldap } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change save mailbox_create mail_log_fields = uid box msgid size flags vsize from subject quota = maildir:User quota quota_rule = *:storage=15G quota_rule2 = Trash:storage=+3%% quota_warning = storage=75%% quota-warning 75 %u quota_warning2 = storage=90%% quota-warning 90 %u sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /etc/dovecot/sieve/clear-delivery-flags.sieve sieve_max_actions = 0 sieve_max_redirects = 2 sieve_max_script_size = 0 sieve_trace_debug = no } protocols = imap pop3 sieve lmtp service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = vmail mode = 0660 user = vmail } user = root } service imap-login { process_limit = 500 service_count = 100 vsz_limit = 64 M } service imap { process_limit = 300 vsz_limit = 512 M } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 0 service_count = 100 vsz_limit = 64 M } service managesieve { process_limit = 1024 } service pop3-login { service_count = 100 vsz_limit = 64 M } service quota-warning { executable = script /opt/mail1.sh unix_listener quota-warning { user = vmail } user = vmail } ssl_cert = </etc/pki/tls/certs/cert.pem ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-usrdb-ldap.conf driver = ldap } protocol imap { imap_client_workarounds = delay-newmail mail_max_userip_connections = 400 mail_plugins = quota mail_log notify imap_quota namespace inbox { location = mailbox Trash { autoexpunge = 15 days } prefix = } } protocol pop3 { mail_max_userip_connections = 3 mail_plugins = quota mail_log notify namespace inbox { location = mailbox Trash { autoexpunge = 15 days } prefix = } pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv } protocol lmtp { auth_socket_path = /var/run/dovecot/auth-master mail_plugins = quota mail_log notify sieve postmaster_address = sysadm@noa.gr sendmail_path = /usr/lib/sendmail } protocol sieve { mail_max_userip_connections = 10 managesieve_logout_format = bytes=%i/%o managesieve_max_compile_errors = 10 managesieve_max_line_length = 64 k }
================================================================
participants (1)
-
Nikolaos Milas