Random duplicated emails
Hello,
I migrated our mail infrastructure to Dovecot on Ubuntu 18.04 some months ago. It works fine, but recently some users told me that they sometime receive duplicated emails. Same email content, same headers including message-id.
I'm using two dovecot servers on two sites. Both server are in cluster. We don't use shared folders. All users that reported this issue so far are using the same server instance. The problematic emails are coming from local users on that instance too. The examples they given to me was emails with many recipients (To/CC). A specific message can be received twice (or more) by recipient A but only once by recipient B. I didn't see anything in the logs about sieve rules that redirect emails to others recipients.
Where should I look to diagnostic this issue?
Thanks.
Server config: # 2.2.33.2 (d6601f4ec): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.21 (92477967) doveconf: Warning: service auth { client_limit=1000 } is lower than required under max. load (2500) doveconf: Warning: service anvil { client_limit=1000 } is lower than required under max. load (1503) # OS: Linux 4.15.0-60-generic x86_64 Ubuntu 18.04.3 LTS auth_default_realm = **edited** auth_master_user_separator = * auth_mechanisms = plain login default_process_limit = 500 first_valid_uid = 200 last_valid_uid = 200 mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_plugins = " notify replication zlib quota" mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vnd.dovecot.debug mdbox_rotate_size = 10 M namespace inbox { inbox = yes location = mailbox "Éléments envoyés" { special_use = \Sent } mailbox "Éléments supprimés" { special_use = \Trash } mailbox Brouillon { special_use = \Drafts } mailbox "Courrier indésirable" { special_use = \Junk } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = INBOX/ separator = / } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes } plugin { mail_replica = tcps:**edited**:53683 quota = count:User quota quota_grace = 10%% quota_rule = *:storage=10G quota_rule2 = INBOX/Trash:storage=+100M quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 La boite courriel du destinataire est pleine - Mailbox is full quota_status_success = DUNNO quota_vsizes = yes quota_warning = storage=100%% quota-warning 100 %u quota_warning2 = storage=95%% quota-warning 95 %u quota_warning3 = storage=90%% quota-warning 90 %u quota_warning4 = storage=85%% quota-warning 85 %u quota_warning5 = storage=75%% quota-warning 75 %u quota_warning6 = -storage=80%% quota-warning '-80' %u quota_warning7 = -storage=100%% quota-warning '-100' %u sieve = /var/vmail/domains/%Ld/%Ln/.dovecot.sieve sieve_before = /var/vmail/sieve/before.sieve sieve_default = /var/vmail/sieve/default.sieve sieve_extensions = +vnd.dovecot.debug sieve_global = /var/vmail/sieve/global zlib_save = lz4 } protocols = " imap lmtp sieve" service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { group = vmail mode = 0666 user = vmail } } service imap-login { inet_listener imap { port = 0 } process_min_avail = 10 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { address = **edited** port = 53684 } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } } ssl = required ssl_cert = </etc/ssl/**edited**/**edited**.crtbundle ssl_client_ca_dir = /etc/ssl/certs ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol lmtp { mail_plugins = " notify replication zlib quota sieve" postmaster_address = postmaster@**edited** } protocol imap { mail_plugins = " notify replication zlib quota imap_zlib imap_quota" }
-- Francis
On 9 Sep 2019, at 10:08, Francis <francisd@gmail.com> wrote:
Where should I look to diagnostic this issue?
Look at the raw stored messages. Your MTA probably has an internal ID (queuid in postfix) that should tell you if the messages are getting duped before they get to dovecot.
the raw messages should have time stamps showing when they were written, so that will give you something else to check in the logs.
Both you mail and dovecot logs, of course.
Since these are local emails on one server, is it possible that server is sending mail to the other server that is coming back as a second copy?
-- Try to realize it's all within yourself/No one else can make you change
I migrated our mail infrastructure to Dovecot on Ubuntu 18.04 some months ago. It works fine, but recently some users told me that they sometime receive duplicated emails. Same email content, same headers including message-id.
I'm using two dovecot servers on two sites. Both server are in cluster. We don't use shared folders. All users that reported this issue so far are using the same server instance. The problematic emails are coming from local users on that instance too. The examples they given to me was emails with many recipients (To/CC). A specific message can be received twice (or more) by recipient A but only once by recipient B. I didn't see anything in the logs about sieve rules that redirect emails to others recipients.
Where should I look to diagnostic this issue?
Thanks.
Server config: # 2.2.33.2 (d6601f4ec): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.21 (92477967)
may be you hit this problem:
https://dovecot.org/list/dovecot/2018-March/111422.html <https://dovecot.org/list/dovecot/2018-March/111422.html>
I don't know if it's fixed yet.
You could log mail events and check if it's related to dsync:
mail_plugins = ... mail_log
plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size subject }
Best regards, Gerald
Le lun. 9 sept. 2019 à 12:08, Francis <francisd@gmail.com> a écrit :
Hello,
I migrated our mail infrastructure to Dovecot on Ubuntu 18.04 some months ago. It works fine, but recently some users told me that they sometime receive duplicated emails. Same email content, same headers including message-id.
I'm using two dovecot servers on two sites. Both server are in cluster. We don't use shared folders. All users that reported this issue so far are using the same server instance. The problematic emails are coming from local users on that instance too. The examples they given to me was emails with many recipients (To/CC). A specific message can be received twice (or more) by recipient A but only once by recipient B. I didn't see anything in the logs about sieve rules that redirect emails to others recipients.
Where should I look to diagnostic this issue?
Hello,
Thanks for your hints. The message from Битус Павел in another thread today seem to be similar to my problem. I asked my users if the affected messages are processed by a local rules in Thunderbird and they said yes. I asked them to replaces theirs local rules by sieve rules instead. We'll see if it fixes the problem.
Thanks!
-- Francis
Hello,
When using a sieve, there is no problem with duplicates, but it is impossible to transfer all users to a sieve.
10.09.2019 20:17, Francis via dovecot пишет:
Le lun. 9 sept. 2019 à 12:08, Francis <francisd@gmail.com <mailto:francisd@gmail.com>> a écrit :
Hello, I migrated our mail infrastructure to Dovecot on Ubuntu 18.04 some months ago. It works fine, but recently some users told me that they sometime receive duplicated emails. Same email content, same headers including message-id. I'm using two dovecot servers on two sites. Both server are in cluster. We don't use shared folders. All users that reported this issue so far are using the same server instance. The problematic emails are coming from local users on that instance too. The examples they given to me was emails with many recipients (To/CC). A specific message can be received twice (or more) by recipient A but only once by recipient B. I didn't see anything in the logs about sieve rules that redirect emails to others recipients. Where should I look to diagnostic this issue?Hello,
Thanks for your hints. The message from Битус Павел in another thread today seem to be similar to my problem. I asked my users if the affected messages are processed by a local rules in Thunderbird and they said yes. I asked them to replaces theirs local rules by sieve rules instead. We'll see if it fixes the problem.
Thanks!
-- Francis
ROZETKA
Bitus Pavlo System administrator rozetka.ua <http://www.rozetka.ua>
participants (4)
-
＠lbutlr
-
Francis
-
Gerald Galster
-
Битус Павел