Avoid duplicate emails in the sent folder
Hello,
I have a webmail program, that I cannot configure to not save sent emails in the Sent folder. My emails to the SOGo team don't seem to reach the mailing list anymore.
So, I have no other choice to think the problem differently, and _maybe_ use Dovecot to fix this issue. Moreover, I think this could be useful for other email clients.
First, I have tried to use Sieve, but no success so far:
imapsieve_mailbox4_name = * imapsieve_mailbox4_from = Sent imapsieve_mailbox4_causes = APPEND, COPY imapsieve_mailbox4_after = file:/var/lib/dovecot/sieve/sent- checks.sieve
And this sieve script:
Message-ID header. require [ "variables", "environment", "duplicate", "imapsieve" ];
[…]
if duplicate :seconds 10 { discard; }
Otherwise, can I configure dovecot in a certain way, for instance create a virtual Sent folder. This would not save the email when appended, but still returns the list of sent emails ?
But this is not working. Any idea?
Thanks for your help!
André
I have a sieve filter called "sent.sieve"
Where I just do this: discard;
Then in sieve settings:
plugin { sieve_plugins = sieve_imapsieve sieve_extprograms sieve_global_extensions = +vnd.dovecot.execute sieve_execute_bin_dir = /usr/bin/ imapsieve_mailbox2_name = Sent imapsieve_mailbox2_causes = APPEND imapsieve_mailbox2_before = file:/etc/dovecot/sieve/sent.sieve imapsieve_expunge_discarded=yes }
And then in Exim4 SMTP server:
sentfolder.filter: if $sender_address_domain is sebbe.eu then unseen save /var/mail/asterisk2/Sent endif
in exim4 config: system_filter = /etc/exim4/sentfolder.filter system_filter_file_transport = sentfolder
in transports of exim4.conf:
sentfolder: debug_print = "T: Sentfolder for $local_part@$domain" headers_add = Status: RO headers_remove = x-dkim-signature headers_remove = x-spf-signature headers_remove = x-trusted-domain headers_remove = keywords headers_remove = x-priority headers_remove = x-status driver = appendfile delivery_date_add envelope_to_add return_path_add use_flock_lock = yes use_fcntl_lock = no use_lockfile = no group = mail mode = 0777 mode_fail_narrower = false current_directory = /
that should solve it.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 12:48 Till: dovecot@dovecot.org Kopia: users@sogo.nu Ämne: Avoid duplicate emails in the sent folder
Hello,
I have a webmail program, that I cannot configure to not save sent emails in the Sent folder. My emails to the SOGo team don't seem to reach the mailing list anymore.
So, I have no other choice to think the problem differently, and _maybe_ use Dovecot to fix this issue. Moreover, I think this could be useful for other email clients.
First, I have tried to use Sieve, but no success so far:
imapsieve_mailbox4_name = * imapsieve_mailbox4_from = Sent imapsieve_mailbox4_causes = APPEND, COPY imapsieve_mailbox4_after = file:/var/lib/dovecot/sieve/sent- checks.sieve
And this sieve script:
Message-ID header. require [ "variables", "environment", "duplicate", "imapsieve" ];
[…]
if duplicate :seconds 10 { discard; }
Otherwise, can I configure dovecot in a certain way, for instance create a virtual Sent folder. This would not save the email when appended, but still returns the list of sent emails ?
But this is not working. Any idea?
Thanks for your help!
André
Thanks, Sebastian.
I just tried this, but it is working too well.
If I want to import emails from another Sent folder, using drag & drop from my email client, the emails are deleted in the background.
This is why I am trying to use the duplicate filter.
Kind regards, André Rodier
On Sun, 2021-04-04 at 13:04 +0200, Sebastian wrote:
I have a sieve filter called "sent.sieve"
Where I just do this: discard;
Then in sieve settings:
plugin { sieve_plugins = sieve_imapsieve sieve_extprograms sieve_global_extensions = +vnd.dovecot.execute sieve_execute_bin_dir = /usr/bin/ imapsieve_mailbox2_name = Sent imapsieve_mailbox2_causes = APPEND imapsieve_mailbox2_before = file:/etc/dovecot/sieve/sent.sieve imapsieve_expunge_discarded=yes }
And then in Exim4 SMTP server:
sentfolder.filter: if $sender_address_domain is sebbe.eu then unseen save /var/mail/asterisk2/Sent endif
in exim4 config: system_filter = /etc/exim4/sentfolder.filter system_filter_file_transport = sentfolder
in transports of exim4.conf:
sentfolder: debug_print = "T: Sentfolder for $local_part@$domain" headers_add = Status: RO headers_remove = x-dkim-signature headers_remove = x-spf-signature headers_remove = x-trusted-domain headers_remove = keywords headers_remove = x-priority headers_remove = x-status driver = appendfile delivery_date_add envelope_to_add return_path_add use_flock_lock = yes use_fcntl_lock = no use_lockfile = no group = mail mode = 0777 mode_fail_narrower = false current_directory = /
that should solve it.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 12:48 Till: dovecot@dovecot.org Kopia: users@sogo.nu Ämne: Avoid duplicate emails in the sent folder
Hello,
I have a webmail program, that I cannot configure to not save sent emails in the Sent folder. My emails to the SOGo team don't seem to reach the mailing list anymore.
So, I have no other choice to think the problem differently, and _maybe_ use Dovecot to fix this issue. Moreover, I think this could be useful for other email clients.
First, I have tried to use Sieve, but no success so far:
imapsieve_mailbox4_name = * imapsieve_mailbox4_from = Sent imapsieve_mailbox4_causes = APPEND, COPY imapsieve_mailbox4_after = file:/var/lib/dovecot/sieve/sent- checks.sieve
And this sieve script:
Message-ID header. require [ "variables", "environment", "duplicate", "imapsieve" ];
[…]
if duplicate :seconds 10 { discard; }
Otherwise, can I configure dovecot in a certain way, for instance create a virtual Sent folder. This would not save the email when appended, but still returns the list of sent emails ?
But this is not working. Any idea?
Thanks for your help!
André
Thats why you need the exim4 filter too.
Because, my version basically prohibits the IMAP client from writing to the Sent folder at all, and gives that job to the SMTP server instead. Works perfectly for me, all sent emails gets into the sent folder without any hitch.
Without any duplicates since no IMAP client is permitted to write into the Sent folder.
Note that only APPEND should be specifyed, not COPY, meaning that if you COPY from the same IMAP account to Sent folder, it will be permitted, but if you APPEND (write a new email) to the Sent folder, it will get auto-deleted.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 13:57 Till: Dovecot Mailing List dovecot@dovecot.org Ämne: Re: Sv: Avoid duplicate emails in the sent folder
Thanks, Sebastian.
I just tried this, but it is working too well.
If I want to import emails from another Sent folder, using drag & drop from my email client, the emails are deleted in the background.
This is why I am trying to use the duplicate filter.
Kind regards, André Rodier
On Sun, 2021-04-04 at 13:04 +0200, Sebastian wrote:
I have a sieve filter called "sent.sieve"
Where I just do this: discard;
Then in sieve settings:
plugin { sieve_plugins = sieve_imapsieve sieve_extprograms sieve_global_extensions = +vnd.dovecot.execute sieve_execute_bin_dir = /usr/bin/ imapsieve_mailbox2_name = Sent imapsieve_mailbox2_causes = APPEND imapsieve_mailbox2_before = file:/etc/dovecot/sieve/sent.sieve imapsieve_expunge_discarded=yes }
And then in Exim4 SMTP server:
sentfolder.filter: if $sender_address_domain is sebbe.eu then unseen save /var/mail/asterisk2/Sent endif
in exim4 config: system_filter = /etc/exim4/sentfolder.filter system_filter_file_transport = sentfolder
in transports of exim4.conf:
sentfolder: debug_print = "T: Sentfolder for $local_part@$domain" headers_add = Status: RO headers_remove = x-dkim-signature headers_remove = x-spf-signature headers_remove = x-trusted-domain headers_remove = keywords headers_remove = x-priority headers_remove = x-status driver = appendfile delivery_date_add envelope_to_add return_path_add use_flock_lock = yes use_fcntl_lock = no use_lockfile = no group = mail mode = 0777 mode_fail_narrower = false current_directory = /
that should solve it.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 12:48 Till: dovecot@dovecot.org Kopia: users@sogo.nu Ämne: Avoid duplicate emails in the sent folder
Hello,
I have a webmail program, that I cannot configure to not save sent emails in the Sent folder. My emails to the SOGo team don't seem to reach the mailing list anymore.
So, I have no other choice to think the problem differently, and _maybe_ use Dovecot to fix this issue. Moreover, I think this could be useful for other email clients.
First, I have tried to use Sieve, but no success so far:
imapsieve_mailbox4_name = * imapsieve_mailbox4_from = Sent imapsieve_mailbox4_causes = APPEND, COPY imapsieve_mailbox4_after = file:/var/lib/dovecot/sieve/sent- checks.sieve
And this sieve script:
Message-ID header. require [ "variables", "environment", "duplicate", "imapsieve" ];
[…]
if duplicate :seconds 10 { discard; }
Otherwise, can I configure dovecot in a certain way, for instance create a virtual Sent folder. This would not save the email when appended, but still returns the list of sent emails ?
But this is not working. Any idea?
Thanks for your help!
André
Hello Sebastian,
On Sun, 2021-04-04 at 14:00 +0200, Sebastian wrote:
Thats why you need the exim4 filter too.
I obtained the exact same result with postfix, using +Sent and a sieve filter. Sent emails are automatically bcc'ed to the user, using the +Sent email address.
However, I still want to be able to drag and drop emails to the sent folder, which unfortunately uses the APPEND verb.
Also, there is a mail import wizard that import emails from gmail/yahoo/etc, and there the verb used is APPEND.
I am now investigating the doveadm deduplicate program.
Kind regards, André
Because, my version basically prohibits the IMAP client from writing to the Sent folder at all, and gives that job to the SMTP server instead. Works perfectly for me, all sent emails gets into the sent folder without any hitch.
Without any duplicates since no IMAP client is permitted to write into the Sent folder.
Note that only APPEND should be specifyed, not COPY, meaning that if you COPY from the same IMAP account to Sent folder, it will be permitted, but if you APPEND (write a new email) to the Sent folder, it will get auto-deleted.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 13:57 Till: Dovecot Mailing List dovecot@dovecot.org Ämne: Re: Sv: Avoid duplicate emails in the sent folder
Thanks, Sebastian.
I just tried this, but it is working too well.
If I want to import emails from another Sent folder, using drag & drop from my email client, the emails are deleted in the background.
This is why I am trying to use the duplicate filter.
Kind regards, André Rodier
On Sun, 2021-04-04 at 13:04 +0200, Sebastian wrote:
I have a sieve filter called "sent.sieve"
Where I just do this: discard;
Then in sieve settings:
plugin { sieve_plugins = sieve_imapsieve sieve_extprograms sieve_global_extensions = +vnd.dovecot.execute sieve_execute_bin_dir = /usr/bin/ imapsieve_mailbox2_name = Sent imapsieve_mailbox2_causes = APPEND imapsieve_mailbox2_before = file:/etc/dovecot/sieve/sent.sieve imapsieve_expunge_discarded=yes }
And then in Exim4 SMTP server:
sentfolder.filter: if $sender_address_domain is sebbe.eu then unseen save /var/mail/asterisk2/Sent endif
in exim4 config: system_filter = /etc/exim4/sentfolder.filter system_filter_file_transport = sentfolder
in transports of exim4.conf:
sentfolder: debug_print = "T: Sentfolder for $local_part@$domain" headers_add = Status: RO headers_remove = x-dkim-signature headers_remove = x-spf-signature headers_remove = x-trusted-domain headers_remove = keywords headers_remove = x-priority headers_remove = x-status driver = appendfile delivery_date_add envelope_to_add return_path_add use_flock_lock = yes use_fcntl_lock = no use_lockfile = no group = mail mode = 0777 mode_fail_narrower = false current_directory = /
that should solve it.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 12:48 Till: dovecot@dovecot.org Kopia: users@sogo.nu Ämne: Avoid duplicate emails in the sent folder
Hello,
I have a webmail program, that I cannot configure to not save sent emails in the Sent folder. My emails to the SOGo team don't seem to reach the mailing list anymore.
So, I have no other choice to think the problem differently, and _maybe_ use Dovecot to fix this issue. Moreover, I think this could be useful for other email clients.
First, I have tried to use Sieve, but no success so far:
imapsieve_mailbox4_name = * imapsieve_mailbox4_from = Sent imapsieve_mailbox4_causes = APPEND, COPY imapsieve_mailbox4_after = file:/var/lib/dovecot/sieve/sent- checks.sieve
And this sieve script:
Message-ID header. require [ "variables", "environment", "duplicate", "imapsieve" ];
[…]
if duplicate :seconds 10 { discard; }
Otherwise, can I configure dovecot in a certain way, for instance create a virtual Sent folder. This would not save the email when appended, but still returns the list of sent emails ?
But this is not working. Any idea?
Thanks for your help!
André
Aah yeah. A solution would be to create a folder, called Sent2. Then you APPEND all emails into the Sent2 folder.
Then you COPY/MOVE all emails into the Sent folder.
What I have understand, your problem is that some clients (like webmail and certain email clients) write their own copy into the Sent folder, thus giving duplicates. Thats why you completely disable APPENDing into the Sent folder.
Dragging and dropping has to be done with COPY/MOVE - which only happens if you Drag and drop from the very same IMAP account. If you drag and drop from different email accounts, of course, from one account's perspective, you are reading a email and then DISCARD'ing it, and from the other account's perspective, you are APPENDing a completely new, unseen email.
Thats why I suggest creating a new Sent2 folder to transfer emails between accounts. Then you use Sent2 to move into Sent folder.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 14:10 Till: Dovecot Mailing List dovecot@dovecot.org Ämne: Re: Sv: Sv: Avoid duplicate emails in the sent folder
Hello Sebastian,
On Sun, 2021-04-04 at 14:00 +0200, Sebastian wrote:
Thats why you need the exim4 filter too.
I obtained the exact same result with postfix, using +Sent and a sieve filter. Sent emails are automatically bcc'ed to the user, using the +Sent email address.
However, I still want to be able to drag and drop emails to the sent folder, which unfortunately uses the APPEND verb.
Also, there is a mail import wizard that import emails from gmail/yahoo/etc, and there the verb used is APPEND.
I am now investigating the doveadm deduplicate program.
Kind regards, André
Because, my version basically prohibits the IMAP client from writing to the Sent folder at all, and gives that job to the SMTP server instead. Works perfectly for me, all sent emails gets into the sent folder without any hitch.
Without any duplicates since no IMAP client is permitted to write into the Sent folder.
Note that only APPEND should be specifyed, not COPY, meaning that if you COPY from the same IMAP account to Sent folder, it will be permitted, but if you APPEND (write a new email) to the Sent folder, it will get auto-deleted.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 13:57 Till: Dovecot Mailing List dovecot@dovecot.org Ämne: Re: Sv: Avoid duplicate emails in the sent folder
Thanks, Sebastian.
I just tried this, but it is working too well.
If I want to import emails from another Sent folder, using drag & drop from my email client, the emails are deleted in the background.
This is why I am trying to use the duplicate filter.
Kind regards, André Rodier
On Sun, 2021-04-04 at 13:04 +0200, Sebastian wrote:
I have a sieve filter called "sent.sieve"
Where I just do this: discard;
Then in sieve settings:
plugin { sieve_plugins = sieve_imapsieve sieve_extprograms sieve_global_extensions = +vnd.dovecot.execute sieve_execute_bin_dir = /usr/bin/ imapsieve_mailbox2_name = Sent imapsieve_mailbox2_causes = APPEND imapsieve_mailbox2_before = file:/etc/dovecot/sieve/sent.sieve imapsieve_expunge_discarded=yes }
And then in Exim4 SMTP server:
sentfolder.filter: if $sender_address_domain is sebbe.eu then unseen save /var/mail/asterisk2/Sent endif
in exim4 config: system_filter = /etc/exim4/sentfolder.filter system_filter_file_transport = sentfolder
in transports of exim4.conf:
sentfolder: debug_print = "T: Sentfolder for $local_part@$domain" headers_add = Status: RO headers_remove = x-dkim-signature headers_remove = x-spf-signature headers_remove = x-trusted-domain headers_remove = keywords headers_remove = x-priority headers_remove = x-status driver = appendfile delivery_date_add envelope_to_add return_path_add use_flock_lock = yes use_fcntl_lock = no use_lockfile = no group = mail mode = 0777 mode_fail_narrower = false current_directory = /
that should solve it.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 12:48 Till: dovecot@dovecot.org Kopia: users@sogo.nu Ämne: Avoid duplicate emails in the sent folder
Hello,
I have a webmail program, that I cannot configure to not save sent emails in the Sent folder. My emails to the SOGo team don't seem to reach the mailing list anymore.
So, I have no other choice to think the problem differently, and _maybe_ use Dovecot to fix this issue. Moreover, I think this could be useful for other email clients.
First, I have tried to use Sieve, but no success so far:
imapsieve_mailbox4_name = * imapsieve_mailbox4_from = Sent imapsieve_mailbox4_causes = APPEND, COPY imapsieve_mailbox4_after = file:/var/lib/dovecot/sieve/sent- checks.sieve
And this sieve script:
Message-ID header. require [ "variables", "environment", "duplicate", "imapsieve" ];
[…]
if duplicate :seconds 10 { discard; }
Otherwise, can I configure dovecot in a certain way, for instance create a virtual Sent folder. This would not save the email when appended, but still returns the list of sent emails ?
But this is not working. Any idea?
Thanks for your help!
André
Also another important thing to take in consideration:
An email that is sent outgoing from an SMTP server, isn't a exact copy of the email put into the SMTP server. Thats why a deduplicate program/command/filter won't work, as the 2 emails aren't 100% identical.
The 2 "duplicates" arent technically 100% identical. Thats why the only solution is to prevent the IMAP client from writing new "unknown" emails into the Sent folder, and then instead use a separate folder for importing emails across accounts (from which you then COPY/MOVE emails from internally in the same acount, to the Sent folder).
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 14:10 Till: Dovecot Mailing List dovecot@dovecot.org Ämne: Re: Sv: Sv: Avoid duplicate emails in the sent folder
Hello Sebastian,
On Sun, 2021-04-04 at 14:00 +0200, Sebastian wrote:
Thats why you need the exim4 filter too.
I obtained the exact same result with postfix, using +Sent and a sieve filter. Sent emails are automatically bcc'ed to the user, using the +Sent email address.
However, I still want to be able to drag and drop emails to the sent folder, which unfortunately uses the APPEND verb.
Also, there is a mail import wizard that import emails from gmail/yahoo/etc, and there the verb used is APPEND.
I am now investigating the doveadm deduplicate program.
Kind regards, André
Because, my version basically prohibits the IMAP client from writing to the Sent folder at all, and gives that job to the SMTP server instead. Works perfectly for me, all sent emails gets into the sent folder without any hitch.
Without any duplicates since no IMAP client is permitted to write into the Sent folder.
Note that only APPEND should be specifyed, not COPY, meaning that if you COPY from the same IMAP account to Sent folder, it will be permitted, but if you APPEND (write a new email) to the Sent folder, it will get auto-deleted.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 13:57 Till: Dovecot Mailing List dovecot@dovecot.org Ämne: Re: Sv: Avoid duplicate emails in the sent folder
Thanks, Sebastian.
I just tried this, but it is working too well.
If I want to import emails from another Sent folder, using drag & drop from my email client, the emails are deleted in the background.
This is why I am trying to use the duplicate filter.
Kind regards, André Rodier
On Sun, 2021-04-04 at 13:04 +0200, Sebastian wrote:
I have a sieve filter called "sent.sieve"
Where I just do this: discard;
Then in sieve settings:
plugin { sieve_plugins = sieve_imapsieve sieve_extprograms sieve_global_extensions = +vnd.dovecot.execute sieve_execute_bin_dir = /usr/bin/ imapsieve_mailbox2_name = Sent imapsieve_mailbox2_causes = APPEND imapsieve_mailbox2_before = file:/etc/dovecot/sieve/sent.sieve imapsieve_expunge_discarded=yes }
And then in Exim4 SMTP server:
sentfolder.filter: if $sender_address_domain is sebbe.eu then unseen save /var/mail/asterisk2/Sent endif
in exim4 config: system_filter = /etc/exim4/sentfolder.filter system_filter_file_transport = sentfolder
in transports of exim4.conf:
sentfolder: debug_print = "T: Sentfolder for $local_part@$domain" headers_add = Status: RO headers_remove = x-dkim-signature headers_remove = x-spf-signature headers_remove = x-trusted-domain headers_remove = keywords headers_remove = x-priority headers_remove = x-status driver = appendfile delivery_date_add envelope_to_add return_path_add use_flock_lock = yes use_fcntl_lock = no use_lockfile = no group = mail mode = 0777 mode_fail_narrower = false current_directory = /
that should solve it.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 12:48 Till: dovecot@dovecot.org Kopia: users@sogo.nu Ämne: Avoid duplicate emails in the sent folder
Hello,
I have a webmail program, that I cannot configure to not save sent emails in the Sent folder. My emails to the SOGo team don't seem to reach the mailing list anymore.
So, I have no other choice to think the problem differently, and _maybe_ use Dovecot to fix this issue. Moreover, I think this could be useful for other email clients.
First, I have tried to use Sieve, but no success so far:
imapsieve_mailbox4_name = * imapsieve_mailbox4_from = Sent imapsieve_mailbox4_causes = APPEND, COPY imapsieve_mailbox4_after = file:/var/lib/dovecot/sieve/sent- checks.sieve
And this sieve script:
Message-ID header. require [ "variables", "environment", "duplicate", "imapsieve" ];
[…]
if duplicate :seconds 10 { discard; }
Otherwise, can I configure dovecot in a certain way, for instance create a virtual Sent folder. This would not save the email when appended, but still returns the list of sent emails ?
But this is not working. Any idea?
Thanks for your help!
André
On Sun, 2021-04-04 at 14:31 +0200, Sebastian wrote:
Also another important thing to take in consideration:
An email that is sent outgoing from an SMTP server, isn't a exact copy of the email put into the SMTP server. Thats why a deduplicate program/command/filter won't work, as the 2 emails aren't 100% identical. Thanks, I see this as the main issue
The 2 "duplicates" arent technically 100% identical. Thats why the only solution is to prevent the IMAP client from writing new "unknown" emails into the Sent folder, and then instead use a separate folder for importing emails across accounts (from which you then COPY/MOVE emails from internally in the same acount, to the Sent folder).
I continue to search on a simple solution, and I will come back to you!
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 14:10 Till: Dovecot Mailing List dovecot@dovecot.org Ämne: Re: Sv: Sv: Avoid duplicate emails in the sent folder
Hello Sebastian,
On Sun, 2021-04-04 at 14:00 +0200, Sebastian wrote:
Thats why you need the exim4 filter too.
I obtained the exact same result with postfix, using +Sent and a sieve filter. Sent emails are automatically bcc'ed to the user, using the +Sent email address.
However, I still want to be able to drag and drop emails to the sent folder, which unfortunately uses the APPEND verb.
Also, there is a mail import wizard that import emails from gmail/yahoo/etc, and there the verb used is APPEND.
I am now investigating the doveadm deduplicate program.
Kind regards, André
Because, my version basically prohibits the IMAP client from writing to the Sent folder at all, and gives that job to the SMTP server instead. Works perfectly for me, all sent emails gets into the sent folder without any hitch.
Without any duplicates since no IMAP client is permitted to write into the Sent folder.
Note that only APPEND should be specifyed, not COPY, meaning that if you COPY from the same IMAP account to Sent folder, it will be permitted, but if you APPEND (write a new email) to the Sent folder, it will get auto-deleted.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 13:57 Till: Dovecot Mailing List dovecot@dovecot.org Ämne: Re: Sv: Avoid duplicate emails in the sent folder
Thanks, Sebastian.
I just tried this, but it is working too well.
If I want to import emails from another Sent folder, using drag & drop from my email client, the emails are deleted in the background.
This is why I am trying to use the duplicate filter.
Kind regards, André Rodier
On Sun, 2021-04-04 at 13:04 +0200, Sebastian wrote:
I have a sieve filter called "sent.sieve"
Where I just do this: discard;
Then in sieve settings:
plugin { sieve_plugins = sieve_imapsieve sieve_extprograms sieve_global_extensions = +vnd.dovecot.execute sieve_execute_bin_dir = /usr/bin/ imapsieve_mailbox2_name = Sent imapsieve_mailbox2_causes = APPEND imapsieve_mailbox2_before = file:/etc/dovecot/sieve/sent.sieve imapsieve_expunge_discarded=yes }
And then in Exim4 SMTP server:
sentfolder.filter: if $sender_address_domain is sebbe.eu then unseen save /var/mail/asterisk2/Sent endif
in exim4 config: system_filter = /etc/exim4/sentfolder.filter system_filter_file_transport = sentfolder
in transports of exim4.conf:
sentfolder: debug_print = "T: Sentfolder for $local_part@$domain" headers_add = Status: RO headers_remove = x-dkim-signature headers_remove = x-spf-signature headers_remove = x-trusted-domain headers_remove = keywords headers_remove = x-priority headers_remove = x-status driver = appendfile delivery_date_add envelope_to_add return_path_add use_flock_lock = yes use_fcntl_lock = no use_lockfile = no group = mail mode = 0777 mode_fail_narrower = false current_directory = /
that should solve it.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 12:48 Till: dovecot@dovecot.org Kopia: users@sogo.nu Ämne: Avoid duplicate emails in the sent folder
Hello,
I have a webmail program, that I cannot configure to not save sent emails in the Sent folder. My emails to the SOGo team don't seem to reach the mailing list anymore.
So, I have no other choice to think the problem differently, and _maybe_ use Dovecot to fix this issue. Moreover, I think this could be useful for other email clients.
First, I have tried to use Sieve, but no success so far:
imapsieve_mailbox4_name = * imapsieve_mailbox4_from = Sent imapsieve_mailbox4_causes = APPEND, COPY imapsieve_mailbox4_after = file:/var/lib/dovecot/sieve/sent- checks.sieve
And this sieve script:
Message-ID header. require [ "variables", "environment", "duplicate", "imapsieve" ];
[…]
if duplicate :seconds 10 { discard; }
Otherwise, can I configure dovecot in a certain way, for instance create a virtual Sent folder. This would not save the email when appended, but still returns the list of sent emails ?
But this is not working. Any idea?
Thanks for your help!
André
Why won't it work to just, as I said, create a separate folder, like Sent2, which you use as a "midpoint" to land all your externally imported emails, and then you just move everything from Sent2 to Sent.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 14:40 Till: Dovecot Mailing List dovecot@dovecot.org Ämne: Re: Sv: Sv: Sv: Avoid duplicate emails in the sent folder
On Sun, 2021-04-04 at 14:31 +0200, Sebastian wrote:
Also another important thing to take in consideration:
An email that is sent outgoing from an SMTP server, isn't a exact copy of the email put into the SMTP server. Thats why a deduplicate program/command/filter won't work, as the 2 emails aren't 100% identical. Thanks, I see this as the main issue
The 2 "duplicates" arent technically 100% identical. Thats why the only solution is to prevent the IMAP client from writing new "unknown" emails into the Sent folder, and then instead use a separate folder for importing emails across accounts (from which you then COPY/MOVE emails from internally in the same acount, to the Sent folder).
I continue to search on a simple solution, and I will come back to you!
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 14:10 Till: Dovecot Mailing List dovecot@dovecot.org Ämne: Re: Sv: Sv: Avoid duplicate emails in the sent folder
Hello Sebastian,
On Sun, 2021-04-04 at 14:00 +0200, Sebastian wrote:
Thats why you need the exim4 filter too.
I obtained the exact same result with postfix, using +Sent and a sieve filter. Sent emails are automatically bcc'ed to the user, using the +Sent email address.
However, I still want to be able to drag and drop emails to the sent folder, which unfortunately uses the APPEND verb.
Also, there is a mail import wizard that import emails from gmail/yahoo/etc, and there the verb used is APPEND.
I am now investigating the doveadm deduplicate program.
Kind regards, André
Because, my version basically prohibits the IMAP client from writing to the Sent folder at all, and gives that job to the SMTP server instead. Works perfectly for me, all sent emails gets into the sent folder without any hitch.
Without any duplicates since no IMAP client is permitted to write into the Sent folder.
Note that only APPEND should be specifyed, not COPY, meaning that if you COPY from the same IMAP account to Sent folder, it will be permitted, but if you APPEND (write a new email) to the Sent folder, it will get auto-deleted.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 13:57 Till: Dovecot Mailing List dovecot@dovecot.org Ämne: Re: Sv: Avoid duplicate emails in the sent folder
Thanks, Sebastian.
I just tried this, but it is working too well.
If I want to import emails from another Sent folder, using drag & drop from my email client, the emails are deleted in the background.
This is why I am trying to use the duplicate filter.
Kind regards, André Rodier
On Sun, 2021-04-04 at 13:04 +0200, Sebastian wrote:
I have a sieve filter called "sent.sieve"
Where I just do this: discard;
Then in sieve settings:
plugin { sieve_plugins = sieve_imapsieve sieve_extprograms sieve_global_extensions = +vnd.dovecot.execute sieve_execute_bin_dir = /usr/bin/ imapsieve_mailbox2_name = Sent imapsieve_mailbox2_causes = APPEND imapsieve_mailbox2_before = file:/etc/dovecot/sieve/sent.sieve imapsieve_expunge_discarded=yes }
And then in Exim4 SMTP server:
sentfolder.filter: if $sender_address_domain is sebbe.eu then unseen save /var/mail/asterisk2/Sent endif
in exim4 config: system_filter = /etc/exim4/sentfolder.filter system_filter_file_transport = sentfolder
in transports of exim4.conf:
sentfolder: debug_print = "T: Sentfolder for $local_part@$domain" headers_add = Status: RO headers_remove = x-dkim-signature headers_remove = x-spf-signature headers_remove = x-trusted-domain headers_remove = keywords headers_remove = x-priority headers_remove = x-status driver = appendfile delivery_date_add envelope_to_add return_path_add use_flock_lock = yes use_fcntl_lock = no use_lockfile = no group = mail mode = 0777 mode_fail_narrower = false current_directory = /
that should solve it.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 12:48 Till: dovecot@dovecot.org Kopia: users@sogo.nu Ämne: Avoid duplicate emails in the sent folder
Hello,
I have a webmail program, that I cannot configure to not save sent emails in the Sent folder. My emails to the SOGo team don't seem to reach the mailing list anymore.
So, I have no other choice to think the problem differently, and _maybe_ use Dovecot to fix this issue. Moreover, I think this could be useful for other email clients.
First, I have tried to use Sieve, but no success so far:
imapsieve_mailbox4_name = * imapsieve_mailbox4_from = Sent imapsieve_mailbox4_causes = APPEND, COPY imapsieve_mailbox4_after = file:/var/lib/dovecot/sieve/sent- checks.sieve
And this sieve script:
Message-ID header. require [ "variables", "environment", "duplicate", "imapsieve" ];
[…]
if duplicate :seconds 10 { discard; }
Otherwise, can I configure dovecot in a certain way, for instance create a virtual Sent folder. This would not save the email when appended, but still returns the list of sent emails ?
But this is not working. Any idea?
Thanks for your help!
André
Sorry for the top posting.
Here the solution I found, without using a secondary Sent folder:
My sieve script is like this:
———————————————————————————————————————— require [ "imapsieve", "include" ];
include :personal :optional "pre-sent-checks"; discard; ————————————————————————————————————————
During the import phase, the sieve user script "pre-sent-checks.sieve" is created with this content:
———————————————————————————————————————— keep;stop; ————————————————————————————————————————
Once the import phase is finished, the user script contains this:
———————————————————————————————————————— require ["include"]; return; ————————————————————————————————————————
This is very well enough for me. The import takes just a few seconds, I don't have race conditions for now. I prefer doing this than createing a secondary Sent folder.
On Sun, 2021-04-04 at 14:43 +0200, Sebastian wrote:
Why won't it work to just, as I said, create a separate folder, like Sent2, which you use as a "midpoint" to land all your externally imported emails, and then you just move everything from Sent2 to Sent.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 14:40 Till: Dovecot Mailing List dovecot@dovecot.org Ämne: Re: Sv: Sv: Sv: Avoid duplicate emails in the sent folder
On Sun, 2021-04-04 at 14:31 +0200, Sebastian wrote:
Also another important thing to take in consideration:
An email that is sent outgoing from an SMTP server, isn't a exact copy of the email put into the SMTP server. Thats why a deduplicate program/command/filter won't work, as the 2 emails aren't 100% identical. Thanks, I see this as the main issue
The 2 "duplicates" arent technically 100% identical. Thats why the only solution is to prevent the IMAP client from writing new "unknown" emails into the Sent folder, and then instead use a separate folder for importing emails across accounts (from which you then COPY/MOVE emails from internally in the same acount, to the Sent folder).
I continue to search on a simple solution, and I will come back to you!
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 14:10 Till: Dovecot Mailing List dovecot@dovecot.org Ämne: Re: Sv: Sv: Avoid duplicate emails in the sent folder
Hello Sebastian,
On Sun, 2021-04-04 at 14:00 +0200, Sebastian wrote:
Thats why you need the exim4 filter too.
I obtained the exact same result with postfix, using +Sent and a sieve filter. Sent emails are automatically bcc'ed to the user, using the +Sent email address.
However, I still want to be able to drag and drop emails to the sent folder, which unfortunately uses the APPEND verb.
Also, there is a mail import wizard that import emails from gmail/yahoo/etc, and there the verb used is APPEND.
I am now investigating the doveadm deduplicate program.
Kind regards, André
Because, my version basically prohibits the IMAP client from writing to the Sent folder at all, and gives that job to the SMTP server instead. Works perfectly for me, all sent emails gets into the sent folder without any hitch.
Without any duplicates since no IMAP client is permitted to write into the Sent folder.
Note that only APPEND should be specifyed, not COPY, meaning that if you COPY from the same IMAP account to Sent folder, it will be permitted, but if you APPEND (write a new email) to the Sent folder, it will get auto-deleted.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 13:57 Till: Dovecot Mailing List dovecot@dovecot.org Ämne: Re: Sv: Avoid duplicate emails in the sent folder
Thanks, Sebastian.
I just tried this, but it is working too well.
If I want to import emails from another Sent folder, using drag & drop from my email client, the emails are deleted in the background.
This is why I am trying to use the duplicate filter.
Kind regards, André Rodier
On Sun, 2021-04-04 at 13:04 +0200, Sebastian wrote:
I have a sieve filter called "sent.sieve"
Where I just do this: discard;
Then in sieve settings:
plugin { sieve_plugins = sieve_imapsieve sieve_extprograms sieve_global_extensions = +vnd.dovecot.execute sieve_execute_bin_dir = /usr/bin/ imapsieve_mailbox2_name = Sent imapsieve_mailbox2_causes = APPEND imapsieve_mailbox2_before = file:/etc/dovecot/sieve/sent.sieve imapsieve_expunge_discarded=yes }
And then in Exim4 SMTP server:
sentfolder.filter: if $sender_address_domain is sebbe.eu then unseen save /var/mail/asterisk2/Sent endif
in exim4 config: system_filter = /etc/exim4/sentfolder.filter system_filter_file_transport = sentfolder
in transports of exim4.conf:
sentfolder: debug_print = "T: Sentfolder for $local_part@$domain" headers_add = Status: RO headers_remove = x-dkim-signature headers_remove = x-spf-signature headers_remove = x-trusted-domain headers_remove = keywords headers_remove = x-priority headers_remove = x-status driver = appendfile delivery_date_add envelope_to_add return_path_add use_flock_lock = yes use_fcntl_lock = no use_lockfile = no group = mail mode = 0777 mode_fail_narrower = false current_directory = /
that should solve it.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org
För André Rodier Skickat: den 4 april 2021 12:48 Till: dovecot@dovecot.org Kopia: users@sogo.nu Ämne: Avoid duplicate emails in the sent folder
Hello,
I have a webmail program, that I cannot configure to not save sent emails in the Sent folder. My emails to the SOGo team don't seem to reach the mailing list anymore.
So, I have no other choice to think the problem differently, and _maybe_ use Dovecot to fix this issue. Moreover, I think this could be useful for other email clients.
First, I have tried to use Sieve, but no success so far:
imapsieve_mailbox4_name = * imapsieve_mailbox4_from = Sent imapsieve_mailbox4_causes = APPEND, COPY imapsieve_mailbox4_after = file:/var/lib/dovecot/sieve/sent- checks.sieve
And this sieve script:
Message-ID header. require [ "variables", "environment", "duplicate", "imapsieve" ];
[…]
if duplicate :seconds 10 { discard; }
Otherwise, can I configure dovecot in a certain way, for instance create a virtual Sent folder. This would not save the email when appended, but still returns the list of sent emails ?
But this is not working. Any idea?
Thanks for your help!
André
Ooh now I understand - you basically disable/bypass the automatic discarding of the duplicates during import, and then enable the discarding again. That works nice too.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 5 april 2021 18:45 Till: Dovecot Mailing List dovecot@dovecot.org Ämne: Re: Sv: Sv: Sv: Sv: Avoid duplicate emails in the sent folder
Sorry for the top posting.
Here the solution I found, without using a secondary Sent folder:
My sieve script is like this:
———————————————————————————————————————— require [ "imapsieve", "include" ];
include :personal :optional "pre-sent-checks"; discard; ————————————————————————————————————————
During the import phase, the sieve user script "pre-sent-checks.sieve" is created with this content:
———————————————————————————————————————— keep;stop; ————————————————————————————————————————
Once the import phase is finished, the user script contains this:
———————————————————————————————————————— require ["include"]; return; ————————————————————————————————————————
This is very well enough for me. The import takes just a few seconds, I don't have race conditions for now. I prefer doing this than createing a secondary Sent folder.
On Sun, 2021-04-04 at 14:43 +0200, Sebastian wrote:
Why won't it work to just, as I said, create a separate folder, like Sent2, which you use as a "midpoint" to land all your externally imported emails, and then you just move everything from Sent2 to Sent.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 14:40 Till: Dovecot Mailing List dovecot@dovecot.org Ämne: Re: Sv: Sv: Sv: Avoid duplicate emails in the sent folder
On Sun, 2021-04-04 at 14:31 +0200, Sebastian wrote:
Also another important thing to take in consideration:
An email that is sent outgoing from an SMTP server, isn't a exact copy of the email put into the SMTP server. Thats why a deduplicate program/command/filter won't work, as the 2 emails aren't 100% identical. Thanks, I see this as the main issue
The 2 "duplicates" arent technically 100% identical. Thats why the only solution is to prevent the IMAP client from writing new "unknown" emails into the Sent folder, and then instead use a separate folder for importing emails across accounts (from which you then COPY/MOVE emails from internally in the same acount, to the Sent folder).
I continue to search on a simple solution, and I will come back to you!
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 14:10 Till: Dovecot Mailing List dovecot@dovecot.org Ämne: Re: Sv: Sv: Avoid duplicate emails in the sent folder
Hello Sebastian,
On Sun, 2021-04-04 at 14:00 +0200, Sebastian wrote:
Thats why you need the exim4 filter too.
I obtained the exact same result with postfix, using +Sent and a sieve filter. Sent emails are automatically bcc'ed to the user, using the +Sent email address.
However, I still want to be able to drag and drop emails to the sent folder, which unfortunately uses the APPEND verb.
Also, there is a mail import wizard that import emails from gmail/yahoo/etc, and there the verb used is APPEND.
I am now investigating the doveadm deduplicate program.
Kind regards, André
Because, my version basically prohibits the IMAP client from writing to the Sent folder at all, and gives that job to the SMTP server instead. Works perfectly for me, all sent emails gets into the sent folder without any hitch.
Without any duplicates since no IMAP client is permitted to write into the Sent folder.
Note that only APPEND should be specifyed, not COPY, meaning that if you COPY from the same IMAP account to Sent folder, it will be permitted, but if you APPEND (write a new email) to the Sent folder, it will get auto-deleted.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För André Rodier Skickat: den 4 april 2021 13:57 Till: Dovecot Mailing List dovecot@dovecot.org Ämne: Re: Sv: Avoid duplicate emails in the sent folder
Thanks, Sebastian.
I just tried this, but it is working too well.
If I want to import emails from another Sent folder, using drag & drop from my email client, the emails are deleted in the background.
This is why I am trying to use the duplicate filter.
Kind regards, André Rodier
On Sun, 2021-04-04 at 13:04 +0200, Sebastian wrote:
I have a sieve filter called "sent.sieve"
Where I just do this: discard;
Then in sieve settings:
plugin { sieve_plugins = sieve_imapsieve sieve_extprograms sieve_global_extensions = +vnd.dovecot.execute sieve_execute_bin_dir = /usr/bin/ imapsieve_mailbox2_name = Sent imapsieve_mailbox2_causes = APPEND imapsieve_mailbox2_before = file:/etc/dovecot/sieve/sent.sieve imapsieve_expunge_discarded=yes }
And then in Exim4 SMTP server:
sentfolder.filter: if $sender_address_domain is sebbe.eu then unseen save /var/mail/asterisk2/Sent endif
in exim4 config: system_filter = /etc/exim4/sentfolder.filter system_filter_file_transport = sentfolder
in transports of exim4.conf:
sentfolder: debug_print = "T: Sentfolder for $local_part@$domain" headers_add = Status: RO headers_remove = x-dkim-signature headers_remove = x-spf-signature headers_remove = x-trusted-domain headers_remove = keywords headers_remove = x-priority headers_remove = x-status driver = appendfile delivery_date_add envelope_to_add return_path_add use_flock_lock = yes use_fcntl_lock = no use_lockfile = no group = mail mode = 0777 mode_fail_narrower = false current_directory = /
that should solve it.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org
För André Rodier Skickat: den 4 april 2021 12:48 Till: dovecot@dovecot.org Kopia: users@sogo.nu Ämne: Avoid duplicate emails in the sent folder
Hello,
I have a webmail program, that I cannot configure to not save sent emails in the Sent folder. My emails to the SOGo team don't seem to reach the mailing list anymore.
So, I have no other choice to think the problem differently, and _maybe_ use Dovecot to fix this issue. Moreover, I think this could be useful for other email clients.
First, I have tried to use Sieve, but no success so far:
imapsieve_mailbox4_name = * imapsieve_mailbox4_from = Sent imapsieve_mailbox4_causes = APPEND, COPY imapsieve_mailbox4_after = file:/var/lib/dovecot/sieve/sent- checks.sieve
And this sieve script:
Message-ID header. require [ "variables", "environment", "duplicate", "imapsieve" ];
[…]
if duplicate :seconds 10 { discard; }
Otherwise, can I configure dovecot in a certain way, for instance create a virtual Sent folder. This would not save the email when appended, but still returns the list of sent emails ?
But this is not working. Any idea?
Thanks for your help!
André
On 04 Apr 2021, at 06:39, André Rodier andre@rodier.me wrote:
I continue to search on a simple solution, and I will come back to you!
Seems the best solution is to get a web client that properly puts messages into DRAFTS and not appends them to Sent, but I guess that is too much to hope for?
I'd certainly at least complain about this misbehavior.
-- The Earth is bipolar
On Sun, 2021-04-04 at 06:45 -0600, @lbutlr wrote:
On 04 Apr 2021, at 06:39, André Rodier andre@rodier.me wrote:
I continue to search on a simple solution, and I will come back to you!
Seems the best solution is to get a web client that properly puts messages into DRAFTS and not appends them to Sent, but I guess that is too much to hope for?
I'd certainly at least complain about this misbehavior.
I keep sending emails to the SOGo mailing, but it is very hard - if not impossible - to get an aswer.
Even worst, my emails to the list are now silently dropped.
I am not proficient enough in Objective-C to submit a patch, but yes, the behaviour is despicable.
André
Why? Isn't it better to fix misbehaving clients server-side?
For example, some clients I have (OEM built-in clients) are for example putting the date 1970-01-01 on all outgoing emails (and also on incoming emails which lacks a Date: header) causing them to land at the bottom of the inbox view and then I miss the emails. Or I sit there wondering where the lingering "unread" email is.
I fixed it simple with a simple ACL filter which replaces the Date: header on all incoming and outgoing emails with the Date: header having the current date/time of the server.
Problem solved!
Thats why its so nice with email servers having "scripting" or "filter" functionality, because you can usually fix misbehaving or glitchy or weird clients server-side so the experience becomes the same as a behaving client for the end-user.
-----Ursprungligt meddelande----- Från: dovecot-bounces@dovecot.org dovecot-bounces@dovecot.org För @lbutlr Skickat: den 4 april 2021 14:46 Till: dovecot mailing list dovecot@dovecot.org Ämne: Re: Avoid duplicate emails in the sent folder
On 04 Apr 2021, at 06:39, André Rodier andre@rodier.me wrote:
I continue to search on a simple solution, and I will come back to you!
Seems the best solution is to get a web client that properly puts messages into DRAFTS and not appends them to Sent, but I guess that is too much to hope for?
I'd certainly at least complain about this misbehavior.
-- The Earth is bipolar
participants (3)
-
＠lbutlr
-
André Rodier
-
Sebastian