[Dovecot] VERY cool add-on request.
I would like to have the ability to execute a command on a folder based on a user doing something like copying a message to that folder. This could be accomplished with a config file directive.
For example, this would be EXTREMEMLY useful as a spam learning hook. Take a scenario where each user has their own spamassassin bayes database, and you want to give the users the ability to train their own database based on what they think is SPAM. It would be nice to give all the users a folder called Spam, and tell them to just copy/move mail into that folder, and spamassassin will automatically learn about it.
If we had a configuration hook where you could execute on a copy/move, then I could tell Dovecot whenever a user puts a message into the Spam folder, execute 'sa-learn --spam --file %s' on that message. Sure we could just do an external script in a cron job to do it every hour or something, which I'll probably do in the mean time, but often users will just delete messsages after they copy into it. Doing it through IMAP means spamassassin will learn about it immediately.
How cool would that be? It seems like an easy hook to implement. I would like to patch it myself for my own use, but I'm not very proficient in C. I saw someone on the net patched Courier to do the same thing so I know I'm not alone in wanting this. Anyone know how I could patch it?
Can we add that to the feature request list?
~Adam
Adam M. Dunn wrote:
For example, this would be EXTREMEMLY useful as a spam learning hook. Take a scenario where each user has their own spamassassin bayes database, and you want to give the users the ability to train their own database based on what they think is SPAM. It would be nice to give all the users a folder called Spam, and tell them to just copy/move mail into that folder, and spamassassin will automatically learn about it.
Doing so via cron is much mroe efficient as you will only have to initialize SA's database once and learn from multiple messages. And if you make another cronjob that periodically wipes out that mailbox, both you and your users will be happy.
Cheers, -jkt
-- cd /local/pub && more beer > /dev/mouth
Cron is more efficient, yes, but I really like this idea. I use spamprobe and have a cron tab setup daily to do this kind of thing, but having it done instantly would be really useful for many reasons.
Sometimes I get bursts of new spam, I dont want to wait for the cron script, now I have to remote login and run the script manually ... yuck.
I have found it tedious to setup each user's cron scripts ... so I only use spamprobe for some users -- (very SMALL server ... only 10 people on it) ... -- If it was done as a hook to IMAP then I think it would be easier to configure a new user and create a true system wide spamprobe tool.
I can imagine many other uses for this technology ... say Print on Move, Broadcast Forward emails without having to use Forward ... just drag& drop ..., maybe an spoof@ebay.com folder where I just drag my daily spoofs and off they go. I bet creative minds can come up with LOTS of ways this can be used (and abused).
Of course, with any new toy, you have to watch out for abusing it ... but that doesnt always mean its a bad capibility to have ...
----------- QUOTE ------------ Adam M. Dunn wrote:
For example, this would be EXTREMEMLY useful as a spam learning hook. Take a scenario where each user has their own spamassassin bayes database, and you want to give the users the ability to train their own database based on what they think is SPAM. It would be nice to give all the users a folder called Spam, and tell them to just copy/move mail into that folder, and spamassassin will automatically learn about it.
Doing so via cron is much mroe efficient as you will only have to initialize SA's database once and learn from multiple messages. And if you make another cronjob that periodically wipes out that mailbox, both you and your users will be happy.
Cheers, -jkt
Adam M. Dunn wrote: <snip>
How cool would that be? It seems like an easy hook to implement. I would like to patch it myself for my own use, but I'm not very proficient in C. I saw someone on the net patched Courier to do the same thing so I know I'm not alone in wanting this. Anyone know how I could patch it?
Adam, Yup, that's pretty cool. You may be able to accomplish the same thing this way:
new folder learnspam
new folder learnham
- move incoming mail to folder according to spam/ham
on your *nix box setup a crontab to run every 4 hours
- 0 */4 * * * /usr/local/bin/salearn.sh
in salearn.sh, put the following
------>8------>8------>8------>8-------------- #!/bin/bash
DIR="/var/spool/mail/"
Learn() { username="$1" # the users mailbox to scan school="$2" # type of learning to do email="$3" # who to email when done
# loop through all "learn" new, cur and tmp dirs in : /var/spool/mail/$username/
for f in `find $DIR/$username -name '.learn'"$school" -type d`; do
#echo "learning spam in $f"
l1=`sa-learn --$school "$f/new/*"`
l2=`sa-learn --$school "$f/cur/*"`
l3=`sa-learn --$school "$f/tmp/*"`
NOW=`date +%D" "%T`
# creaate an email message
printf "$school learned from account %s\n---------------\nmails in new:%s\nmails in cur:%s\nmails in tmp:%s" "$username" "$l1" "$l2" "$l3" | mailx -s "$school learned for $NOW" "$email"
#echo "cleaning up learn* mail box"
# remove echo and double quotes to actually do anything
echo "/bin/rm -rf $f/new/*"
echo "/bin/rm -rf $f/cur/*"
echo "/bin/rm -rf $f/tmp/*"
done}
# pass the account and... # the type of learning to do - also the directory name so must be either 'ham' or 'spam' # where to send the email # Learn 'testuser' 'spam' 'user1@domain.com' Learn 'testuser' 'ham' 'user1@domain.com'
------>8------>8------>8------>8--------------
On Fri, 2006-03-17 at 13:04 -0600, Adam M. Dunn wrote:
For example, this would be EXTREMEMLY useful as a spam learning hook. Take a scenario where each user has their own spamassassin bayes database, and you want to give the users the ability to train their own database based on what they think is SPAM. It would be nice to give all the users a folder called Spam, and tell them to just copy/move mail into that folder, and spamassassin will automatically learn about it.
You've seen my plugin for dspam, right? :) I've been doing this for, umm, 1.5 years or so now.
http://johannes.sipsolutions.net/Projects - dovecot dspam plugin
johannes
Adam M. Dunn ha scritto:
I would like to have the ability to execute a command on a folder based on a user doing something like copying a message to that folder. This could be accomplished with a config file directive.
You could simply use a normal folder and use dnotify/inotify to trigger the execution of salearn.
In a larger evironment, all users could share a folder with a single dnotify daemon, moving dropped emails to a specified directory and using cron to spawn salearn.
That would be the simpler solution, imho.
Massimo
I've just now been reading about dnotify and inotify, and while if they existed on my system could work for this, I still think direct support would be cleaner and more efficient. D/I notify both require kernel patches not available on all Linux systems (and not at all on non-Linux systems). They require an additional program to be running at all times monitoring the directory (probably one or more per user), and in one case it doesn't even say which file changed so the overhead of scanning the files in the directory is added on ... plus if you have multiple directories you'd have to write a special variant that kept a bunch of directory handles open. Very cumbersome.
While having a hook into dovecot to invoke a process when a particular folder was touched would be very clean. Maybe the "plugin" mechanism could be used to write such a thingy.
Adam M. Dunn ha scritto:
I would like to have the ability to execute a command on a folder based on a user doing something like copying a message to that folder. This could be accomplished with a config file directive.
You could simply use a normal folder and use dnotify/inotify to trigger the execution of salearn.
In a larger evironment, all users could share a folder with a single dnotify daemon, moving dropped emails to a specified directory and using cron to spawn salearn.
That would be the simpler solution, imho.
Massimo
David A. Lee ha scritto:
I've just now been reading about dnotify and inotify, and while if they existed on my system could work for this, I still think direct support would be cleaner and more efficient. D/I notify both require kernel patches not available on all Linux systems (and not at all on non-Linux systems).
Inotify is avalaible as a patch for 2.6. dnotify is included in the kernel, since 2.4.something and does not requires patches. You only need userspace tools (only 1 file).
They require an additional program to be running at all times monitoring the directory (probably one or more per user), and in one case it doesn't even say which file changed so the overhead of scanning the files in the directory is added on ...
It says which file/directory has changed. Try it. It does run all the time but it is triggered by a kernel event. It does not scans files/dirs all the time.
plus if you have multiple directories you'd have to write a special variant that kept a bunch of directory handles open. Very cumbersome.
Not at all. Try it.
While having a hook into dovecot to invoke a process when a particular folder was touched would be very clean. Maybe the "plugin" mechanism could be used to write such a thingy.
I agree that it would be a better solution. I don't agree when you say that dnotify is not a good solution, particulary when you have not tried it!
If timo will not write a plugin or you won't write it, I think dnotify/inotify is a good alternative.
Max
Max "CtRiX" wrote:
David A. Lee ha scritto:
I've just now been reading about dnotify and inotify, and while if they existed on my system could work for this, I still think direct support would be cleaner and more efficient. D/I notify both require kernel patches not available on all Linux systems (and not at all on non-Linux systems).
Inotify is avalaible as a patch for 2.6. dnotify is included in the kernel, since 2.4.something and does not requires patches. You only need userspace tools (only 1 file).
And in fact inotify's been rolled into 2.6.13 and up, no patch needed.
On Sat, 2006-03-18 at 19:19 +0100, Max "CtRiX" wrote:
plus if you have multiple directories you'd have to write a special variant that kept a bunch of directory handles open. Very cumbersome.
Not at all.
This is certainly true for dnotify, one of the reasons why inotify was invented.
johannes
Adam M. Dunn wrote:
For example, this would be EXTREMEMLY useful as a spam learning hook.
Take a look through the archives for the DSpam plugin. It makes any mail moved into a 'magic' folder get flagged for learning as SPAM, and any message moved out of that folder flagged for learning as HAM.
One folder, periodic updates, and (I'm told) it's working quite well.
However, a more generic "magic folder hooks" plugin does sound like a good idea.
-- Curtis Maloney cmaloney@cardgate.net
On Mon, 2006-03-20 at 11:40 +1100, Curtis Maloney wrote:
Adam M. Dunn wrote:
For example, this would be EXTREMEMLY useful as a spam learning hook.
Take a look through the archives for the DSpam plugin. It makes any mail moved into a 'magic' folder get flagged for learning as SPAM, and any message moved out of that folder flagged for learning as HAM.
One folder, periodic updates, and (I'm told) it's working quite well.
Just to clarify one thing: it doesn't do periodic updates, I dropped that idea early due to the added complexity. Updates to the dspam database are during the move, even before the ok response is given to the client.
johannes
participants (8)
-
Adam M. Dunn
-
Curtis Maloney
-
David A. Lee
-
Jan Kundrát
-
Johannes Berg
-
Leeman Strout
-
Max "CtRiX"
-
tblader