Automatic purging of old email in all mailboxes
I am using a postfix + Dovecot server as a test mail server for which some applications in our test environment use as a target to deliver email so that our real endusers don't receive messages from our test servers.
A few of the mailboxes in Dovecot receive hundreds of emails per day. I'd like to automatically remove all emails in all mailboxes and mailbox folders that were received more than 90 days prior to the received date. As I'm a novice Dovecot administrator, I'm not exactly sure what the best way to accomplish this is, but I've started looking at the "doveadm expunge" command. I figure I could create a cron job that calls this command with the -A switch and that has a search query that finds all emails older than 90 days.
I'm aware that I can test my query by using the "doveadm search" command. The problem is that when I do any sort of search query with that command (e.g. "doveadm search -A NEW"), I receive the following error messages:
Error: User listing returned failure doveadm: Error: Failed to iterate through some users
I've searched for help with this error, but most of the guidance I'm seeing refers to making sure that dovecot-sql is configured correctly; however, I'm using "passwd-file" for the passdb and not a true database.
Here is doveconf -n:
# 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.13 (7b14904) # OS: Linux 4.4.0-34-generic x86_64 Ubuntu 16.04.1 LTS ext4 auth_mechanisms = plain login hostname = mail.domain.test info_log_path = /var/log/dovecot.log log_path = /var/log/dovecot.log mail_location = maildir:/var/mail/vhosts/%d/%n namespace inbox { inbox = yes location = mailbox "Deleted Items" { special_use = \Trash } mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox "Junk E-Mail" { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Items" { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=PLAIN username_format=%u /etc/dovecot/dovecot-users driver = passwd-file } protocols = " imap lmtp pop3" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } } service imap-login { inet_listener imaps { port = 993 ssl = yes } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service pop3-login { inet_listener pop3s { port = 995 ssl = yes } } ssl = required ssl_cert =
Thank you in advance!
On Mon, Aug 29, 2016 at 07:40:41PM +0000, Scott W. Sander wrote:
I am using a postfix + Dovecot server as a test mail server for which some applications in our test environment use as a target to deliver email so that our real endusers don't receive messages from our test servers.
A few of the mailboxes in Dovecot receive hundreds of emails per day. I'd like to automatically remove all emails in all mailboxes and mailbox folders that were received more than 90 days prior to the received date. As I'm a novice Dovecot administrator, I'm not exactly sure what the best way to accomplish this is, but I've started looking at the "doveadm expunge" command. I figure I could create a cron job that calls this command with the -A switch and that has a search query that finds all emails older than 90 days.
I'm aware that I can test my query by using the "doveadm search" command. The problem is that when I do any sort of search query with that command (e.g. "doveadm search -A NEW"), I receive the following error messages:
Error: User listing returned failure doveadm: Error: Failed to iterate through some users
I've searched for help with this error, but most of the guidance I'm seeing refers to making sure that dovecot-sql is configured correctly; however, I'm using "passwd-file" for the passdb and not a true database.
Since you're using maildirs as the storage backend, it might be easier to just write a small script in the language of your preference (bash, python, perl, etc.) that walks the directory tree and deletes files based on their mtime.
I personally use a python script to delete messages that have been in my trash folder for more than 30 days, and it works very well.
--Sean
http://wiki.dovecot.org/MailboxSettings
I just started using auto expunge so can't tell you how well it works, but it's worth a look.
Sent from my iPhone
On Aug 29, 2016, at 2:40 PM, Scott W. Sander scottwsx96@gmail.com wrote:
I am using a postfix + Dovecot server as a test mail server for which some applications in our test environment use as a target to deliver email so that our real endusers don't receive messages from our test servers.
A few of the mailboxes in Dovecot receive hundreds of emails per day. I'd like to automatically remove all emails in all mailboxes and mailbox folders that were received more than 90 days prior to the received date. As I'm a novice Dovecot administrator, I'm not exactly sure what the best way to accomplish this is, but I've started looking at the "doveadm expunge" command. I figure I could create a cron job that calls this command with the -A switch and that has a search query that finds all emails older than 90 days.
I'm aware that I can test my query by using the "doveadm search" command. The problem is that when I do any sort of search query with that command (e.g. "doveadm search -A NEW"), I receive the following error messages:
Error: User listing returned failure doveadm: Error: Failed to iterate through some users
I've searched for help with this error, but most of the guidance I'm seeing refers to making sure that dovecot-sql is configured correctly; however, I'm using "passwd-file" for the passdb and not a true database.
Here is doveconf -n:
# 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.13 (7b14904) # OS: Linux 4.4.0-34-generic x86_64 Ubuntu 16.04.1 LTS ext4 auth_mechanisms = plain login hostname = mail.domain.test info_log_path = /var/log/dovecot.log log_path = /var/log/dovecot.log mail_location = maildir:/var/mail/vhosts/%d/%n namespace inbox { inbox = yes location = mailbox "Deleted Items" { special_use = \Trash } mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox "Junk E-Mail" { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Items" { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=PLAIN username_format=%u /etc/dovecot/dovecot-users driver = passwd-file } protocols = " imap lmtp pop3" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } } service imap-login { inet_listener imaps { port = 993 ssl = yes } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service pop3-login { inet_listener pop3s { port = 995 ssl = yes } } ssl = required ssl_cert =
Thank you in advance!
On 2016-08-29 21:40, Scott W. Sander wrote:
I am using a postfix + Dovecot server as a test mail server for which some applications in our test environment use as a target to deliver email so that our real endusers don't receive messages from our test servers.
A few of the mailboxes in Dovecot receive hundreds of emails per day.
I'd like to automatically remove all emails in all mailboxes and mailbox folders that were received more than 90 days prior to the received date. As I'm a novice Dovecot administrator, I'm not exactly sure what the best way to accomplish this is, but I've started looking at the "doveadm expunge" command. I figure I could create a cron job that calls this command with the -A switch and that has a search query that finds all emails older than 90 days.I'm aware that I can test my query by using the "doveadm search" command. The problem is that when I do any sort of search query with that command (e.g. "doveadm search -A NEW"), I receive the following error messages:
Error: User listing returned failure doveadm: Error: Failed to iterate through some users
You have to switch your userdb to something else than static, passwd-file for example http://wiki2.dovecot.org/AuthDatabase/PasswdFile
I've searched for help with this error, but most of the guidance I'm seeing refers to making sure that dovecot-sql is configured correctly; however, I'm using "passwd-file" for the passdb and not a true database.
there is the expire plugin http://wiki2.dovecot.org/Plugins/Expire
Here is doveconf -n:
# 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.13 (7b14904) # OS: Linux 4.4.0-34-generic x86_64 Ubuntu 16.04.1 LTS ext4 auth_mechanisms = plain login hostname = mail.domain.test info_log_path = /var/log/dovecot.log log_path = /var/log/dovecot.log mail_location = maildir:/var/mail/vhosts/%d/%n namespace inbox { inbox = yes location = mailbox "Deleted Items" { special_use = \Trash } mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox "Junk E-Mail" { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Items" { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=PLAIN username_format=%u /etc/dovecot/dovecot-users driver = passwd-file } protocols = " imap lmtp pop3" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } } service imap-login { inet_listener imaps { port = 993 ssl = yes } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service pop3-login { inet_listener pop3s { port = 995 ssl = yes } } ssl = required ssl_cert =
Thank you in advance!
-- Christian Kivalo
- 29, hétfő keltezéssel 19.40-kor Scott W. Sander ezt írta:
userdb { args = uid=vmail gid=vmail home=/var/mail/vhosts/%d/%n driver = static }
Use passwd-file driver to get iteration working. The static driver is can not be used for iteration.
-- Üdvözlettel
Stefán Tamás --------------------------------- domain > email > web >>> siker Numex Informatika Kft. Mobil: +36 20 956 0233, Tel: +36 1 205 3915, Fax: +36 1 203 6037 http://numex.hu
Forgive me, I'm a novice. Consider that today I have already set up a working postfix + Dovecot server with userdb using the static driver and passdb using a password file (in the format <username>:{hashing algorithm}<password hash>). The Dovecot server has several mailboxes which are already in use today.
I understand that I'll need to change over to using the passwd-file driver in order to get doveadm mailbox iteration working. That said, can someone please explain how I could migrate over from the static to passwd-file without causing disruption to the existing mailboxes? Do I need to just add some more information to my Dovecot password file (namely the uid, gid, and mailbox home location)? If so, can I still use the "vmail" uid and gid for all mailbox users that I was using with the static driver configuration?
Sorry, when I followed the setup guide I did, I did not realize it would prevent me from being able to use doveadm expunge -A and doveadm search -A.
Thank you!
On Tue, Aug 30, 2016 at 1:07 AM Stefán Tamás tamas@numex.hu wrote:
- 29, hétfő keltezéssel 19.40-kor Scott W. Sander ezt írta:
userdb { args = uid=vmail gid=vmail home=/var/mail/vhosts/%d/%n driver = static }
Use passwd-file driver to get iteration working. The static driver is can not be used for iteration.
-- Üdvözlettel
Stefán Tamás --------------------------------- domain > email > web >>> siker Numex Informatika Kft. Mobil: +36 20 956 0233, Tel: +36 1 205 3915, Fax: +36 1 203 6037 http://numex.hu
You can provide those defaults with
mail_uid=vmail mail_gid=vmail mail_home=/var/mail/vhosts/%d/%u mail_location=Maildir:~/maildir
global settings.
And then you can remove userdb block completely.
Aki
On August 30, 2016 at 7:02 PM "Scott W. Sander" scottwsx96@gmail.com wrote:
Forgive me, I'm a novice. Consider that today I have already set up a working postfix + Dovecot server with userdb using the static driver and passdb using a password file (in the format <username>:{hashing algorithm}<password hash>). The Dovecot server has several mailboxes which are already in use today.
I understand that I'll need to change over to using the passwd-file driver in order to get doveadm mailbox iteration working. That said, can someone please explain how I could migrate over from the static to passwd-file without causing disruption to the existing mailboxes? Do I need to just add some more information to my Dovecot password file (namely the uid, gid, and mailbox home location)? If so, can I still use the "vmail" uid and gid for all mailbox users that I was using with the static driver configuration?
Sorry, when I followed the setup guide I did, I did not realize it would prevent me from being able to use doveadm expunge -A and doveadm search -A.
Thank you!
On Tue, Aug 30, 2016 at 1:07 AM Stefán Tamás tamas@numex.hu wrote:
- 29, hétfő keltezéssel 19.40-kor Scott W. Sander ezt írta:
userdb { args = uid=vmail gid=vmail home=/var/mail/vhosts/%d/%n driver = static }
Use passwd-file driver to get iteration working. The static driver is can not be used for iteration.
-- Üdvözlettel
Stefán Tamás --------------------------------- domain > email > web >>> siker Numex Informatika Kft. Mobil: +36 20 956 0233, Tel: +36 1 205 3915, Fax: +36 1 203 6037 http://numex.hu
I have now set those (with some modifications detailed below) in /etc/dovecont/conf.d/10-mail.conf:
mail_uid=vmail mail_uid=vmail mail_home=/var/mail/vhosts/%d/%n mail_location=maildir:/var/mail/vhosts/%d/$n
I made these changes relative to your suggestion to more closely match my original settings.
After doing that, I modified /etc/dovecot/conf.d/auth-passwdfile.conf.ext and commented out the userdb block. After doing sudo service dovecot reload, I was having the same problem as I was originally when I ran sudo doveadm search -A NEW:
Error: User listing returned failure doveadm: Error: Failed to iterate through some users
I went back in and added the following userdb block to /etc/dovecot/conf.d/auth-passwdfile.conf.ext:
userdb { driver = passwd-file args = /etc/dovecot/dovecot-users }
After doing another sudo service dovecot reload and then trying sudo doveadm search -A NEW, I now get new errors:
doveadm(testuser@domain.test): Info: User no longer exists, skipping doveadm(pfuser1@domain.test): Info: User no longer exists, skipping
So at least I'm getting closer as it does see the two Dovecot mailbox accounts I have in /etc/dovecot/dovecot-users, but it is acting like the user doesn't exist for some reason. Any more ideas? I did not modify the /etc/dovecot/dovecot-users file at all from my previous description. Do I need to modify that to include, at the very least, a uid (vmail), gid (vmail), and userdb_mail locaiton (userdb_mail=maildir:/var/mail/vhosts/%d/%n)?
On Tue, Aug 30, 2016 at 12:33 PM Aki Tuomi aki.tuomi@dovecot.fi wrote:
You can provide those defaults with
mail_uid=vmail mail_gid=vmail mail_home=/var/mail/vhosts/%d/%u mail_location=Maildir:~/maildir
global settings.
And then you can remove userdb block completely.
Aki
On August 30, 2016 at 7:02 PM "Scott W. Sander" scottwsx96@gmail.com wrote:
Forgive me, I'm a novice. Consider that today I have already set up a working postfix + Dovecot server with userdb using the static driver and passdb using a password file (in the format <username>:{hashing algorithm}<password hash>). The Dovecot server has several mailboxes which are already in use today.
I understand that I'll need to change over to using the passwd-file driver in order to get doveadm mailbox iteration working. That said, can someone please explain how I could migrate over from the static to passwd-file without causing disruption to the existing mailboxes? Do I need to just add some more information to my Dovecot password file (namely the uid, gid, and mailbox home location)? If so, can I still use the "vmail" uid and gid for all mailbox users that I was using with the static driver configuration?
Sorry, when I followed the setup guide I did, I did not realize it would prevent me from being able to use doveadm expunge -A and doveadm search -A.
Thank you!
On Tue, Aug 30, 2016 at 1:07 AM Stefán Tamás tamas@numex.hu wrote:
- 29, hétfő keltezéssel 19.40-kor Scott W. Sander ezt írta:
userdb { args = uid=vmail gid=vmail home=/var/mail/vhosts/%d/%n driver = static }
Use passwd-file driver to get iteration working. The static driver is can not be used for iteration.
-- Üdvözlettel
Stefán Tamás --------------------------------- domain > email > web >>> siker Numex Informatika Kft. Mobil: +36 20 956 0233, Tel: +36 1 205 3915, Fax: +36 1 203 6037 http://numex.hu
participants (6)
-
Aki Tuomi
-
Christian Kivalo
-
Edgar Pettijohn
-
Scott W. Sander
-
Sean Greenslade
-
Stefán Tamás