[Dovecot] purge maildir folder after sa-learn
Hello,
I am looking for a command-line utility to automatically purge one Maildir folder. I want to periodically run sa-learn from cron, and after the spam learning folder has been added to the Bayes database, its contents should be deleted. I've searched a bit, but only found a few old utilities or some telnet+expect scripts, which both didn't look too reliable.
Is there any command-line utility to delete all contents of a Maildir folder, or is there any other solution?
Thanks, Florian
Florian Effenberger wrote:
Hello,
I am looking for a command-line utility to automatically purge one Maildir folder. I want to periodically run sa-learn from cron, and after the spam learning folder has been added to the Bayes database, its contents should be deleted. I've searched a bit, but only found a few old utilities or some telnet+expect scripts, which both didn't look too reliable.
Is there any command-line utility to delete all contents of a Maildir folder, or is there any other solution?
Thanks, Florian
I would write a script that cron executes, and have the script do the sa-learn, then delete the file.
I run qmail-toaster, which uses vpopmail for virtual domains/accounts. My maildirs for learning ham/spam are shared, and are named sa-learn/Ham and sa-learn/Spam.
Here's the script that I run in cron.daily to execute sa-learn:
#!/bin/sh ##################################################################### # learn and remove spam and ham in shared folders ##################################################################### # shubes 3/26/08 - created #####################################################################
learndir="/home/vpopmail/domains/shubes.net/sa-learn" hambox=.Ham spambox=.Spam
do_the_learning(){
learnas=$1 maildir=$2
shopt -s extglob
for spamfile in find $maildir/+(cur|new)/* 2>/dev/null; do
sudo -u vpopmail -H sa-learn --$learnas $spamfile
rc=$?
if [ $? != "0" ]; then
echo "sa-learn failed, rc=$rc, spamfile=$spamfile"
exit $rc
fi
rm $spamfile
done
}
do_the_learning ham "$learndir/$hambox" do_the_learning spam "$learndir/$spambox"
exit 0 # end of script
HTH.
-Eric 'shubes'
On Tue, 2010-05-25 at 21:29 +0200, Florian Effenberger wrote:
Hello,
I am looking for a command-line utility to automatically purge one Maildir folder.
How about just:
rm -f /path/to/Maildir/.spam-learn/*
With v2.0 you can also do:
doveadm expunge -u username mailbox spam-learn all
On 25/05/2010 21:10, Timo Sirainen wrote:
On Tue, 2010-05-25 at 21:29 +0200, Florian Effenberger wrote:
Hello,
I am looking for a command-line utility to automatically purge one Maildir folder.
How about just:
rm -f /path/to/Maildir/.spam-learn/*
Surely we need to delete everything in 'new' and 'cur'? :
rm -f /path/to/Maildir/.spam-learn/new/* /path/to/Maildir/.spam-learn/cur/*
Or perhaps, a bit more pipey and less risk of blowing up whilst globbing in the shell:
maildir=/path/to/Maildir/.spam-learn find $maildir/new $maildir/cur -mindepth 1 -maxdepth 1 -type f -print0 | xargs -0 -r rm -f
Regards,
Bill
Hello everyone,
thanks for the fast replies, much appreciated!
2010/5/25 Timo Sirainen <tss@iki.fi>:
I am looking for a command-line utility to automatically purge one Maildir folder.
How about just:
rm -f /path/to/Maildir/.spam-learn/*
With v2.0 you can also do:
doveadm expunge -u username mailbox spam-learn all
I run 1.x, so doeveadm doesn't exist. :-) Can I simply delete the files, don't I destroy any indices or slow down Dovecot operation because of index rebuilding? Sounds like a simple solution I also thought of, but I was hesitant of just deleting the files...
Thanks, Florian
On Wed, 2010-05-26 at 09:56 +0200, Florian Effenberger wrote:
rm -f /path/to/Maildir/.spam-learn/*
Oh, I really ment .spam-learn/cur/* (and maybe .spam-learn/new/* if you make it learn from both and it doesn't move mails to cur/).
Can I simply delete the files, don't I destroy any indices or slow down Dovecot operation because of index rebuilding?
Keep Dovecot's index files around so that they won't get deleted while Dovecot is trying to read/write them. You won't slow down anything by manually deleting maildir files.
Hi Timo,
2010/5/26 Timo Sirainen <tss@iki.fi>:
Oh, I really ment .spam-learn/cur/* (and maybe .spam-learn/new/* if you make it learn from both and it doesn't move mails to cur/).
Can I simply delete the files, don't I destroy any indices or slow down Dovecot operation because of index rebuilding?
Keep Dovecot's index files around so that they won't get deleted while Dovecot is trying to read/write them. You won't slow down anything by manually deleting maildir files.
thanks a lot, seems to work like a charm!
Florian
On Wed, May 26, 2010 at 06:13, Timo Sirainen <tss@iki.fi> wrote:
On Wed, 2010-05-26 at 09:56 +0200, Florian Effenberger wrote:
rm -f /path/to/Maildir/.spam-learn/*
Oh, I really ment .spam-learn/cur/* (and maybe .spam-learn/new/* if you make it learn from both and it doesn't move mails to cur/).
Can I simply delete the files, don't I destroy any indices or slow down Dovecot operation because of index rebuilding?
Keep Dovecot's index files around so that they won't get deleted while Dovecot is trying to read/write them. You won't slow down anything by manually deleting maildir files.
So basically, just moving mail files out (to a learn queue or whatever), from either "new" or "cur" is safe for Dovecot? That would be simpler.
Phil Howard wrote:
On Wed, May 26, 2010 at 06:13, Timo Sirainen <tss@iki.fi> wrote:
On Wed, 2010-05-26 at 09:56 +0200, Florian Effenberger wrote:
rm -f /path/to/Maildir/.spam-learn/* Oh, I really ment .spam-learn/cur/* (and maybe .spam-learn/new/* if you make it learn from both and it doesn't move mails to cur/).
Can I simply delete the files, don't I destroy any indices or slow down Dovecot operation because of index rebuilding? Keep Dovecot's index files around so that they won't get deleted while Dovecot is trying to read/write them. You won't slow down anything by manually deleting maildir files.
So basically, just moving mail files out (to a learn queue or whatever), from either "new" or "cur" is safe for Dovecot? That would be simpler.
That's what I do. I move them either to a (shared) ham or spam folder, and run the script posted earlier, which processes and deletes them.
-- -Eric 'shubes'
On Sat, May 29, 2010 at 12:03, Eric Shubert <ejs@shubes.net> wrote:
That's what I do. I move them either to a (shared) ham or spam folder, and run the script posted earlier, which processes and deletes them.
Thanks. I just wanted to be sure such a move would not effect (corrupt) what Dovecot was doing (e.g. indexes and such). Glad it won't. That makes things simpler.
Florian Effenberger <floeff@gmail.com> wrote:
I am looking for a command-line utility to automatically purge one Maildir folder. I want to periodically run sa-learn from cron, and after the spam learning folder has been added to the Bayes database, its contents should be deleted. I've searched a bit, but only found a few old utilities or some telnet+expect scripts, which both didn't look too reliable.
Is there any command-line utility to delete all contents of a Maildir folder, or is there any other solution?
Have you considered fetching from dovecot (maildir) to mbox file and learning from mbox file? a) fetchmail CAN use .../dovecot/imap wrapper script in "plugin" option to read maildir directly [set environment variable MAIL to maildir:~/Maildir in the wrapper] b) fetchmail CAN use procmail in "mda" option to deliver directly to mbox
-- [pl>en: Andrew] Andrzej Adam Filip : anfi@onet.eu I am a friend of the working man, and I would rather be his friend than be one. -- Clarence Darrow
Hi Andrzej,
2010/5/28 Andrzej Adam Filip <anfi@onet.eu>:
Have you considered fetching from dovecot (maildir) to mbox file and learning from mbox file? a) fetchmail CAN use .../dovecot/imap wrapper script in "plugin" option to read maildir directly [set environment variable MAIL to maildir:~/Maildir in the wrapper] b) fetchmail CAN use procmail in "mda" option to deliver directly to mbox
does fetching to mbox file bring any advantage? It seems that learning out of maildir and rm'ing afterwards works like a charm, so I guess fetchmail would bring more overhead?
Florian
Florian Effenberger <floeff@gmail.com> wrote:
Hi Andrzej,
2010/5/28 Andrzej Adam Filip <anfi@onet.eu>:
Have you considered fetching from dovecot (maildir) to mbox file and learning from mbox file? a) fetchmail CAN use .../dovecot/imap wrapper script in "plugin" option to read maildir directly [set environment variable MAIL to maildir:~/Maildir in the wrapper] b) fetchmail CAN use procmail in "mda" option to deliver directly to mbox
does fetching to mbox file bring any advantage? It seems that learning out of maildir and rm'ing afterwards works like a charm, so I guess fetchmail would bring more overhead?
It will bring more overhead but "some people" value portable solutions. Some people try to avoid locking THEMSELVES into a) specific format of IMAP folder As I understand dovecot "recommends" switching from maildir to dbox b) specific software (specific IMAP server) I do not plan to change dovecot but I do like to keep my options open.
Anyway: instead of scanning all and deleting files in maildir (race risk) you may consider moving files to another file directory/folder (on the same file-system) and learn from it. You may ask Timo How to lock maildir message/file before "moving it out"/"deleting it" by non dovecot software :-)
-- [pl>en: Andrew] Andrzej Adam Filip : anfi@onet.eu Never tell people how to do things. Tell them WHAT to do and they will surprise you with their ingenuity. -- Gen. George S. Patton, Jr.
Hi Andrzej,
2010/5/28 Andrzej Adam Filip <anfi@onet.eu>:
does fetching to mbox file bring any advantage? It seems that learning out of maildir and rm'ing afterwards works like a charm, so I guess fetchmail would bring more overhead?
It will bring more overhead but "some people" value portable solutions. Some people try to avoid locking THEMSELVES into a) specific format of IMAP folder As I understand dovecot "recommends" switching from maildir to dbox b) specific software (specific IMAP server) I do not plan to change dovecot but I do like to keep my options open.
Anyway: instead of scanning all and deleting files in maildir (race risk) you may consider moving files to another file directory/folder (on the same file-system) and learn from it. You may ask Timo How to lock maildir message/file before "moving it out"/"deleting it" by non dovecot software :-)
Hm, now I'm irritated a bit. :-) If I have a dedicated maildir directory (IMAP folder) for spam learn, what can happen if I scan and delete files out of it? The worst thing that can happen is that the IMAP folder gets corrupt, which is not harm, as it's spam anyway. However, from what I've heard, Maildir doesn't have any file locking issues, so there should be no problem?
Florian
Florian Effenberger <floeff@gmail.com> wrote:
Hi Andrzej,
2010/5/28 Andrzej Adam Filip <anfi@onet.eu>:
does fetching to mbox file bring any advantage? It seems that learning out of maildir and rm'ing afterwards works like a charm, so I guess fetchmail would bring more overhead?
It will bring more overhead but "some people" value portable solutions. Some people try to avoid locking THEMSELVES into a) specific format of IMAP folder As I understand dovecot "recommends" switching from maildir to dbox b) specific software (specific IMAP server) I do not plan to change dovecot but I do like to keep my options open.
Anyway: instead of scanning all and deleting files in maildir (race risk) you may consider moving files to another file directory/folder (on the same file-system) and learn from it. You may ask Timo How to lock maildir message/file before "moving it out"/"deleting it" by non dovecot software :-)
Hm, now I'm irritated a bit. :-) If I have a dedicated maildir directory (IMAP folder) for spam learn, what can happen if I scan and delete files out of it? The worst thing that can happen is that the IMAP folder gets corrupt, which is not harm, as it's spam anyway. However, from what I've heard, Maildir doesn't have any file locking issues, so there should be no problem?
Florian
It depends on what you want: a) dirty&ugly but *trivial* "hack" serving your very specific requirement that can "make troubles" after some change in pattern of use b) simple solution
-- [pl>en: Andrew] Andrzej Adam Filip : anfi@onet.eu 10) there is no 10, but it sounded like a nice number :) -- Wichert Akkerman
Hi,
2010/5/30 Andrzej Adam Filip <anfi@onet.eu>:
It depends on what you want: a) dirty&ugly but *trivial* "hack" serving your very specific requirement that can "make troubles" after some change in pattern of use b) simple solution
well, I don't see a real difference. As far as I know, and as Timo said, deleting messages files out of maildir is absolutely fine, so I wouldn't consider it a hack. :-)
Florian
participants (6)
-
Andrzej Adam Filip
-
Eric Shubert
-
Florian Effenberger
-
Phil Howard
-
Timo Sirainen
-
William Blunn