[Dovecot] mail messages moving between mailfolders
Hello to all!
I'm using IMAP-based antispam self-learning. I have a folder "spam" in each Maildir in which I place spam-tagged messages during delivery. If antispam filter makes mistake, user can drag'n'drop message to/from "spam" folder. I running perl-script periodicaly and retrain spam-filter (dSpam). Now I must scan all messages, but it's not a good practice (server load depends from mail volume, not user activity) so I want add a "hook" to IMAP-server, i.e. I want server to report me something like "date: file1 -> file2" via any IPC (socket?) or log file while user moves messages across different mail folders. Overload from antispam retraining will be function from user activity with such feature added.
P.S.: Now I'm using courier, but want to try dovecot.
Sorry for bad English. With respect, Taras.
-- С уважением, Савчук Тарас "ЭЛАНТЕК" +7 (495) 589 68 81 +7 (926) 575 22 11 http://www.elantech.ru
taras@elantech.ru пишет:
Hello to all!
I'm using IMAP-based antispam self-learning. I have a folder "spam" in each Maildir in which I place spam-tagged messages during delivery. If antispam filter makes mistake, user can drag'n'drop message to/from "spam" folder. I running perl-script periodicaly and retrain spam-filter (dSpam). Now I must scan all messages, but it's not a good practice (server load depends from mail volume, not user activity) so I want add a "hook" to IMAP-server, i.e. I want server to report me something like "date: file1 -> file2" via any IPC (socket?) or log file while user moves messages across different mail folders. Overload from antispam retraining will be function from user activity with such feature added.
P.S.: Now I'm using courier, but want to try dovecot.
Sorry for bad English. With respect, Taras.
Sorry, I forget to ask question :) I need recomendations from experienced people about described situation. Is it's good practice practice to add "hook" to IMAP server for reaching user-friendly server-side antispam training?
-- С уважением, Савчук Тарас "ЭЛАНТЕК" +7 (495) 589 68 81 +7 (926) 575 22 11 http://www.elantech.ru
On Tue, 2006-05-02 at 02:54 +0400, taras@elantech.ru wrote:
I need recomendations from experienced people about described situation. Is it's good practice practice to add "hook" to IMAP server for reaching user-friendly server-side antispam training?
I do that, but it's much more userfriendly than multiple folders: http://johannes.sipsolutions.net/Projects/dovecot-dspam-integration/
johannes
On Tue, 2006-05-02 at 11:27 +0200, Johannes Berg wrote:
I do that, but it's much more userfriendly than multiple folders:
Actually, I misunderstood what you're doing. But you can change my plugin to do exactly what you were looking for by simply doing some other action than calling dspam when a user moves messages.
johannes
Johannes Berg пишет:
On Tue, 2006-05-02 at 11:27 +0200, Johannes Berg wrote:
I do that, but it's much more userfriendly than multiple folders:
Actually, I misunderstood what you're doing. But you can change my plugin to do exactly what you were looking for by simply doing some other action than calling dspam when a user moves messages.
johannes
I rebuild dspam for making strlen() of spam and not spam tags equivalen each to other. I use only one spam folder, as you, but without any links or such things. Here is script to understand what I'm doing:
---------- BEGIN
use DBI; use File::Find;
$user = "root"; $database = "mail"; $password = "****";
$spam_subfolder = ".SPAM";
$SENDER = "";
$SPAM_TAG = "SpamXXXX"; $HAM_TAG = "Innocent"; $DSPAM_TAG = "X-DSPAM-Result:";
$curfile = "";
$ST = $IT = $SR = $IR = $AM = $AA = 0;
my $DBH = DBI->connect("DBI:mysql:$database", $user, $password) or die "I can't connect! Error: $DBI::errstr\n";
my $sqlm = "SELECT email, homedir, maildir, uid, gid FROM
postfix_users"; my $sthm = $DBH->prepare($sqlm); $sthm->execute; my ($email, $homedir, $maildir, $uid, $gid); $sthm->bind_columns(undef, \($email, $homedir, $maildir, $uid, $gid));
while($sthm->fetch){
# find files in /.spam subfolder
# and retrain dspam with mistake files
my $spamdir = $homedir . "/" . $maildir . $spam_subfolder ;
my $hamdir = $homedir . "/" . $maildir;
$spamdir_cur = $spamdir . "/cur";
$spamdir_new = $spamdir . "/new";
$spamdir_tmp = $spamdir . "/tmp";
$SENDER = $email;
$ST = $IT = $SR = $IR = 0;
# SPAM
print "Training SPAM for user $email in $spamdir_cur...\n";
find(\&wanted_spam, $spamdir_cur);
find(\&wanted_spam, $spamdir_new);
# HAM
print "Training HAM for user $email in $hamdir\n";
find(\&wanted_ham, $hamdir);
$AM = $ST+$IT;
print "Stats for user $SENDER:\n";
print "Total messages : " . $AM . "\n";
print "Spam total: " . $ST . "\n";
print "Innocent total: " . $IT . "\n";
print "HAM -> SPAM: " . $SR . "\n";
print "SPAM -> HAM: " . $IR . "\n";
$AA += $AM;
}
print "Total messages :" . $AA . "\n";
print `date`;
$sthm->finish(); $DBH->disconnect();
#-----------------------------------------------------
sub wanted_spam {
if ( -f $_) {
$ST += 1;
my $yes = 0;
$curfile = $_;
open("FD", "+<", $_) || die "Can't open file $_\n";
flock FD, LOCK_EX || die "Can't lock file $_\n";
while($str = <FD>){
if($str =~ /$DSPAM_TAG/){
$tag = $';
if($tag =~ /$HAM_TAG/){
print "$HAM_TAG found: Rewriting!\n";
print "File: $curfile\n";
print "Dir: $File::Find::dir\n";
seek(FD, - (length($SPAM_TAG) + 2), 1);
print FD " $SPAM_TAG";
seek(FD, -1, 2);
$yes = 1;
$SR += 1;
}else
{
if($tag =~ /" $SPAM_TAG"/){
print "$SPAM_TAG found: OK!\n";
}
}
}
}
flock FD, LOCK_UN || die "Can't RElock file $_\n";
close(FD);
if($yes == 1){
`cat '$curfile'| /usr/local/bin/dspam --user $SENDER
--class=spam --source=error --deliver=`;
}
}
}
sub wanted_ham {
if("$File::Find::dir" ne "$spamdir_cur" && "$File::Find::dir" ne
"$spamdir_new" && "$File::Find::dir" ne "$spamdir_tmp")
{
if ( -f $_) {
$IT += 1;
my $yes = 0;
$curfile = $_;
open("FD", "+<", $_) || die "Can't open file $_\n";
flock FD, LOCK_EX || die "Can't lock file $_\n";
while($str = <FD>){
if($str =~ /$DSPAM_TAG/){
$tag = $';
if($tag =~ /$SPAM_TAG/){
print "$SPAM_TAG found: Rewriting!\n";
print "File: $curfile\n";
print "Dir: $File::Find::dir\n";
seek(FD, - (length($HAM_TAG) + 2), 1);
print FD " $HAM_TAG";
seek(FD, -1, 2);
$yes = 1;
$IR += 1;
}else
{
if($tag =~ /" $HAM_TAG"/){
print "$HAM_TAG found: OK!\n";
}
}
}
}
flock FD, LOCK_UN || die "Can't RElock file $_\n";
close(FD);
if($yes == 1){
`cat '$curfile' | /usr/local/bin/dspam --user $SENDER
--class=innocent --source=error --deliver=`; } } } }
--------- END
I think, that easiest method to don't scan folders is simple log message moving between folders by IMAP server. What do you think about this?
-- С уважением, Савчук Тарас "ЭЛАНТЕК" +7 (495) 589 68 81 +7 (926) 575 22 11 http://www.elantech.ru
On Tue, 2006-05-02 at 15:43 +0400, taras@elantech.ru wrote:
[...]
Yes, I see what you're doing. And as far as I understand you, you want to get rid of having to periodically scan the folders.
I think, that easiest method to don't scan folders is simple log message moving between folders by IMAP server. What do you think about this?
Well my point is that my plugin provides you with an example of how to do this in dovecot. You can change my plugin, rewrite it, ... and have create a log message or hit your mysql database directly :)
johannes
taras@elantech.ru wrote:
Hello to all!
I'm using IMAP-based antispam self-learning. I have a folder "spam" in each Maildir in which I place spam-tagged messages during delivery. If antispam filter makes mistake, user can drag'n'drop message to/from "spam" folder. I running perl-script periodicaly and retrain spam-filter (dSpam). Now I must scan all messages, but it's not a good practice (server load depends from mail volume, not user activity) so I want add a "hook" to IMAP-server, i.e. I want server to report me something like "date: file1 -> file2" via any IPC (socket?) or log file while user moves messages across different mail folders. Overload from antispam retraining will be function from user activity with such feature added.
I've done a similar thing, script filters messages to a "spam" mailbox, users can move/copy messages to "not-a-spam" or "is-a-spam" mailboxes. Script runs several times a day, starts spam learning/unleraning for each message in the respective mailbox and then removes them.
HTH, -jkt
-- cd /local/pub && more beer > /dev/mouth
participants (3)
-
Jan Kundrát
-
Johannes Berg
-
taras@elantech.ru