replication and spam removal ("doveadm expunge")
Olaf Hopp
Olaf.Hopp at kit.edu
Mon Apr 6 13:40:19 EEST 2020
Hi Aki,
On 4/4/20 8:12 PM, Aki Tuomi wrote:
> Can you provide doveconf -n and try turning on mail_debug=yes on both ends and try doveadm -Dv expunge ....
mail_debug=yes
is on on both ends and dovecot was restarted but anyway nothing is logged when I issue "doveadm -Dv expunge "
In the shell where I issue the "expunge" I see the following:
# /usr/bin/doveadm -Dv expunge -u test4 mailbox INBOX.spambox BEFORE 13m
Debug: Loading modules from directory: /usr/lib64/dovecot
Debug: Module loaded: /usr/lib64/dovecot/lib15_notify_plugin.so
Debug: Module loaded: /usr/lib64/dovecot/lib20_replication_plugin.so
Debug: Loading modules from directory: /usr/lib64/dovecot/doveadm
Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message)
Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib10_doveadm_sieve_plugin.so
Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message)
doveadm(test4)<19830><>: Debug: auth-master: userdb lookup(test4): Started userdb lookup
doveadm(test4)<19830><>: Debug: auth-master: conn unix:/var/run/dovecot/auth-userdb: Connecting
doveadm(test4)<19830><>: Debug: auth-master: conn unix:/var/run/dovecot/auth-userdb (pid=15116,uid=0): Client connected (fd=10)
doveadm(test4)<19830><>: Debug: auth-master: userdb lookup(test4): auth USER input: test4 system_groups_user=test4 uid=1805 gid=2300 home=/home/irams1-test/test4
doveadm(test4)<19830><>: Debug: auth-master: userdb lookup(test4): Finished userdb lookup (username=test4 system_groups_user=test4 uid=1805 gid=2300 home=/home/irams1-test/test4)
doveadm(test4): Debug: Effective uid=1805, gid=2300, home=/home/irams1-test/test4
doveadm(test4): Debug: Namespace inbox: type=private, prefix=INBOX., sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir
doveadm(test4): Debug: maildir++: root=/home/irams1-test/test4/Maildir, index=, indexpvt=, control=, inbox=/home/irams1-test/test4/Maildir, alt=
doveadm(test4): Debug: Namespace : type=private, prefix=, sep=, inbox=no, hidden=yes, list=no, subscriptions=no location=fail::LAYOUT=none
doveadm(test4): Debug: none: root=, index=, indexpvt=, control=, inbox=, alt=
doveadm(test4): Debug: Mailbox INBOX.spambox: Mailbox opened because: expunge
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=38
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=39
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=40
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=41
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=42
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=43
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=44
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=45
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=46
doveadm(test4): Debug: auth-master: conn unix:/var/run/dovecot/auth-userdb (pid=15116,uid=0): Disconnected: Connection closed (fd=10)
A few mails are deleted from the spambox but the deletion is not synced to the other side
Even if I do a "doveadm force-resync -u test4 '*'" on both sides the deletions are not replicated
but anyway
#doveadm replicator status test4
username priority fast sync full sync success sync failed
test4 none 00:26:47 18:19:46 00:26:47 -
on both sides.
If new spam arrives it got well replicated to the other side as expected
"doveconf -n" ist attached below.
Thanks,
Olaf
>
> Aki
>> On 04/04/2020 20:03 Olaf Hopp < olaf.hopp at kit.edu <mailto:olaf.hopp at kit.edu>> wrote:
>>
>>
>> Nobody ? :-(
>>
>> On 3/30/20 5:26 PM, Olaf Hopp wrote:
>>> Hello everybody,
>>> since now I did no replication and spam is delivered into users folder "spambox"
>>> Every night there is a cronjob which deletes spam older than 30 days via something like
>>> "find .... -ctime +30 -delete"
>>> Now I'm going to set up replication (two way) and I thought that
>>> doing "rm" is not a good idea.
>>> So I modified the job to something like
>>> /usr/bin/doveadm expunge -u test1 mailbox INBOX.spambox BEFORE 30d
>>> which works like intended, but I see that on the replication
>>> partner the spam isn't deleted. Even if I do a
>>> doveadm force-resync -u test1 '*'
>>> or log in via imap to the replication partner I still see the old spam.
>>>
>>> So my question is:
>>> is this the intended behaviour and I have to run the "doveadm expunge" on both replication partners
>>> or should the deletion via "expunge" being replicated to the partner and I have a misconfiguration
>>> or maybe I hit a bug ?
>>>
>>> Regards, Olaf
>>>
>>>
>>> /etc/dovecot/conf.d/12-replication.conf:
>>>
>>>
>>> mail_plugins = $mail_plugins notify replication
>>> service aggregator {
>>> fifo_listener replication-notify-fifo {
>>> mode = 0666
>>> }
>>> unix_listener replication-notify {
>>> mode = 0666
>>> }
>>> }
>>> service replicator {
>>> process_min_avail = 1
>>> unix_listener replicator-doveadm {
>>> mode = 0666
>>> }
>>> }
>>> service doveadm {
>>> inet_listener {
>>> port = 1109
>>> }
>>> }
>>> doveadm_password = XXXXX
>>> plugin {
>>> mail_replica = tcp:X.Y.Z.X:1109
>>> }
>>>
>>>
>>
>> --
>> Karlsruher Institut für Technologie (KIT)
>> ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik
>>
>> Dipl.-Geophys. Olaf Hopp
>> - Leitung IT-Dienste -
>>
>> Am Fasanengarten 5, Gebäude 50.34, Raum 009
>> 76131 Karlsruhe
>> Telefon: +49 721 608-43973
>> Fax: +49 721 608-46699
>> E-Mail: Olaf.Hopp at kit.edu <mailto:Olaf.Hopp at kit.edu>
>> www.atis.informatik.kit.edu
>>
>> www.kit.edu
>>
>> KIT - Die Forschungsuniversität in der Helmholtz-Gemeinschaft
>>
>> Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.
>
> ---
> Aki Tuomi
>
--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik
Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -
Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: Olaf.Hopp at kit.edu
www.atis.informatik.kit.edu
www.kit.edu
KIT - Die Forschungsuniversität in der Helmholtz-Gemeinschaft
Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.
-------------- next part --------------
# 2.3.10 (0da0eff44): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.10 (bf8ef1c2)
# OS: Linux 4.18.0-147.5.1.el8_1.x86_64 x86_64 CentOS Linux release 8.1.1911 (Core)
# Hostname: irams1-test2.ira.uka.de
auth_failure_delay = 3 secs
auth_master_user_separator = *
auth_mechanisms = plain login
auth_username_format = %Ln
auth_verbose = yes
auth_verbose_passwords = plain
auth_worker_max_count = 60
default_client_limit = 2000
default_process_limit = 3000
default_vsz_limit = 512 M
doveadm_password = # hidden, use -P to show it
first_valid_uid = 1000
last_valid_uid = 65533
mail_debug = yes
mail_location = maildir:~/Maildir
mail_plugins = " notify replication"
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext
mbox_write_locks = fcntl
namespace inbox {
inbox = yes
location =
mailbox "Deleted Items" {
autoexpunge = 30 days
special_use = \Trash
}
mailbox "Deleted Messages" {
autoexpunge = 30 days
special_use = \Trash
}
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox "Gelöschte Objekte" {
autoexpunge = 30 days
special_use = \Trash
}
mailbox "Gel&APY-schte Objekte" {
autoexpunge = 30 days
special_use = \Trash
}
mailbox Papierkorb {
autoexpunge = 30 days
special_use = \Trash
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox Trash {
auto = subscribe
autoexpunge = 30 days
special_use = \Trash
}
mailbox spambox {
auto = create
special_use = \Junk
}
prefix = INBOX.
separator = .
}
passdb {
args = /etc/dovecot/master-users
driver = passwd-file
master = yes
}
passdb {
args = dovecot
driver = pam
}
plugin {
mail_replica = tcp:141.3.10.15:1109
sieve = file:~/sieve;active=~/.dovecot.sieve
sieve_before = /etc/dovecot/sieve-master
sieve_max_redirects = 20
}
postmaster_address = postmaster at ira.uka.de
protocols = imap pop3 lmtp sieve sieve
quota_full_tempfail = yes
service aggregator {
fifo_listener replication-notify-fifo {
mode = 0666
}
unix_listener replication-notify {
mode = 0666
}
}
service doveadm {
inet_listener {
port = 1109
}
}
service imap-login {
process_limit = 8192
process_min_avail = 16
service_count = 0
}
service imap {
process_limit = 8192
}
service lmtp {
executable = lmtp -L
}
service managesieve-login {
inet_listener sieve {
port = 4190
}
}
service pop3 {
process_limit = 8192
}
service replicator {
process_min_avail = 1
unix_listener replicator-doveadm {
mode = 0666
}
}
ssl = required
ssl_ca = </etc/pki/tls/certs/kitchain.crt
ssl_cert = </etc/pki/tls/certs/imap.informatik.pem
ssl_cipher_list = ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA
ssl_dh = # hidden, use -P to show it
ssl_key = # hidden, use -P to show it
ssl_prefer_server_ciphers = yes
syslog_facility = local3
userdb {
driver = passwd
}
verbose_proctitle = yes
protocol lmtp {
mail_plugins = " notify replication sieve"
}
protocol imap {
mail_max_userip_connections = 100
ssl_cert = </etc/pki/tls/certs/imap_with_chain.informatik.kit.edu.pem
ssl_key = # hidden, use -P to show it
}
protocol pop3 {
ssl_cert = </etc/pki/tls/certs/pop_with_chain.informatik.kit.edu.pem
ssl_key = # hidden, use -P to show it
}
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5212 bytes
Desc: S/MIME Cryptographic Signature
URL: <https://dovecot.org/pipermail/dovecot/attachments/20200406/14430fdf/attachment-0001.p7s>
More information about the dovecot
mailing list