[Dovecot] Advanced dovecot tricks - spam review/release
Hi,
I'm looking for some advice to do a really advanced trick with Dovecot. I'm not sure if this can be done. I need to describe first.
I have a spam filtering company that does front end spam filtering. (Junk Email Filter) I want to add a system where I store a copy of spam on a server and make it available to the customer to review and maybe resent on false positives.
I know I could do something simple where I deliver all spam to a domain account and make it available to an administrator. Then if it's a false positive they would drag the message to a "resend" folder. I'll have something the checks the folder one a minute to pick up and resend.
However
What would be very cool is delivering the spam to individual accounts. So a user who logs in individually can see their own spam. But the admin for the domain would be able to see all users. Maybe the users would appear as folders?
Then a master account (me) would be able to log in and see all the domains as folders and the users as folders inside the domains?
One thing I can do is deliver the spam to 3 different places so it's visible on all levels.
I'm just wondering if anyone out there has any ideas about that. And I'll need an authentication system.
Hi Marc, i solved this using an automated report for users quarantine. In front of dovecot i have 2 mailscanner boxes that stores spam emails in quarantine and logs them to a database, periodically there is a script that sends an html report to users that recieved spam in the last interval (1h, 4h, 24hs depending on the user preferences) showing a list of time-from-subject of all new items in quarantine. There is also a link to release the email from quarantine and the users recieves it on his inbox. So our users can release emails without bothering anyone. (There is also an admin view where the admin can see all the trafic for the domain).
my 2cents.
regards,
eduardo.
2012/10/3 Marc Perkel marc@perkel.com
Hi,
I'm looking for some advice to do a really advanced trick with Dovecot. I'm not sure if this can be done. I need to describe first.
I have a spam filtering company that does front end spam filtering. (Junk Email Filter) I want to add a system where I store a copy of spam on a server and make it available to the customer to review and maybe resent on false positives.
I know I could do something simple where I deliver all spam to a domain account and make it available to an administrator. Then if it's a false positive they would drag the message to a "resend" folder. I'll have something the checks the folder one a minute to pick up and resend.
However
What would be very cool is delivering the spam to individual accounts. So a user who logs in individually can see their own spam. But the admin for the domain would be able to see all users. Maybe the users would appear as folders?
Then a master account (me) would be able to log in and see all the domains as folders and the users as folders inside the domains?
One thing I can do is deliver the spam to 3 different places so it's visible on all levels.
I'm just wondering if anyone out there has any ideas about that. And I'll need an authentication system.
If you ever figure out how to do this, I've got an excellent name for it: MailWatch
http://sourceforge.net/projects/mailwatch/
steve
On 10/3/2012 3:48 PM, Marc Perkel wrote:
Hi,
I'm looking for some advice to do a really advanced trick with Dovecot. I'm not sure if this can be done. I need to describe first.
I have a spam filtering company that does front end spam filtering. (Junk Email Filter) I want to add a system where I store a copy of spam on a server and make it available to the customer to review and maybe resent on false positives.
I know I could do something simple where I deliver all spam to a domain account and make it available to an administrator. Then if it's a false positive they would drag the message to a "resend" folder. I'll have something the checks the folder one a minute to pick up and resend.
However
What would be very cool is delivering the spam to individual accounts. So a user who logs in individually can see their own spam. But the admin for the domain would be able to see all users. Maybe the users would appear as folders?
Then a master account (me) would be able to log in and see all the domains as folders and the users as folders inside the domains?
One thing I can do is deliver the spam to 3 different places so it's visible on all levels.
I'm just wondering if anyone out there has any ideas about that. And I'll need an authentication system.
Maildir, layout=fs /var/vmail/domain/user/
Spams get delivered there. User has access there. Domain admin has mail_location=/var/vmail/domain, you have mail_location=/var/vmail
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
Marc Perkel marc@perkel.com wrote:
Hi,
I'm looking for some advice to do a really advanced trick with Dovecot. I'm not sure if this can be done. I need to describe first.
I have a spam filtering company that does front end spam filtering. (Junk Email Filter) I want to add a system where I store a copy of spam on a server and make it available to the customer to review and maybe resent on false positives.
I know I could do something simple where I deliver all spam to a domain account and make it available to an administrator. Then if it's a false positive they would drag the message to a "resend" folder. I'll have something the checks the folder one a minute to pick up and resend.
However
What would be very cool is delivering the spam to individual accounts. So a user who logs in individually can see their own spam. But the admin for the domain would be able to see all users. Maybe the users would appear as folders?
Then a master account (me) would be able to log in and see all the domains as folders and the users as folders inside the domains?
One thing I can do is deliver the spam to 3 different places so it's visible on all levels.
I'm just wondering if anyone out there has any ideas about that. And I'll need an authentication system.
I'm a little confused. What about the cur, new, and tmp directories? How does that fit in?
On 10/3/2012 1:04 PM, Timo Sirainen wrote:
Maildir, layout=fs /var/vmail/domain/user/
Spams get delivered there. User has access there. Domain admin has mail_location=/var/vmail/domain, you have mail_location=/var/vmail
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
Marc Perkel marc@perkel.com wrote:
Hi, I'm looking for some advice to do a really advanced trick with Dovecot. I'm not sure if this can be done. I need to describe first. I have a spam filtering company that does front end spam filtering. (Junk Email Filter) I want to add a system where I store a copy of spam on a server and make it available to the customer to review and maybe resent on false positives. I know I could do something simple where I deliver all spam to a domain account and make it available to an administrator. Then if it's a false positive they would drag the message to a "resend" folder. I'll have something the checks the folder one a minute to pick up and resend. However What would be very cool is delivering the spam to individual accounts. So a user who logs in individually can see their own spam. But the admin for the domain would be able to see all users. Maybe the users would appear as folders? Then a master account (me) would be able to log in and see all the domains as folders and the users as folders inside the domains? One thing I can do is deliver the spam to 3 different places so it's visible on all levels. I'm just wondering if anyone out there has any ideas about that. And I'll need an authentication system.
On 4.10.2012, at 2.42, Marc Perkel wrote:
On 10/3/2012 1:04 PM, Timo Sirainen wrote:
Maildir, layout=fs /var/vmail/domain/user/
Spams get delivered there. User has access there. Domain admin has mail_location=/var/vmail/domain, you have mail_location=/var/vmail I'm a little confused. What about the cur, new, and tmp directories? How does that fit in?
users: mail_location = maildir:/var/vmail/%d/%n:LAYOUT=fs
domain admins: mail_location = maildir:/var/vmail/%d:LAYOUT=fs
full admins: mail_location = maildir:/var/vmail:LAYOUT=fs
The cur/new/tmp directories are under the /var/vmail/domain/username/. So the spams get delivered to /var/vmail/domain/username/new/ directory. This is visible to the users as their INBOX. Domain admins see the mails under the "username" mailbox. Full admins see the mails under "domain/username" mailbox.
On 10/3/2012 4:46 PM, Timo Sirainen wrote:
On 4.10.2012, at 2.42, Marc Perkel wrote:
On 10/3/2012 1:04 PM, Timo Sirainen wrote:
Maildir, layout=fs /var/vmail/domain/user/
Spams get delivered there. User has access there. Domain admin has mail_location=/var/vmail/domain, you have mail_location=/var/vmail I'm a little confused. What about the cur, new, and tmp directories? How does that fit in? users: mail_location = maildir:/var/vmail/%d/%n:LAYOUT=fs
domain admins: mail_location = maildir:/var/vmail/%d:LAYOUT=fs
full admins: mail_location = maildir:/var/vmail:LAYOUT=fs
The cur/new/tmp directories are under the /var/vmail/domain/username/. So the spams get delivered to /var/vmail/domain/username/new/ directory. This is visible to the users as their INBOX. Domain admins see the mails under the "username" mailbox. Full admins see the mails under "domain/username" mailbox.
I'm testing it now and the user level works. But the other levels I don't see anything. I am a little brain dead today though. I'll test more
On 10/3/2012 4:46 PM, Timo Sirainen wrote:
On 4.10.2012, at 2.42, Marc Perkel wrote:
On 10/3/2012 1:04 PM, Timo Sirainen wrote:
Maildir, layout=fs /var/vmail/domain/user/
Spams get delivered there. User has access there. Domain admin has mail_location=/var/vmail/domain, you have mail_location=/var/vmail I'm a little confused. What about the cur, new, and tmp directories? How does that fit in? users: mail_location = maildir:/var/vmail/%d/%n:LAYOUT=fs
domain admins: mail_location = maildir:/var/vmail/%d:LAYOUT=fs
full admins: mail_location = maildir:/var/vmail:LAYOUT=fs
The cur/new/tmp directories are under the /var/vmail/domain/username/. So the spams get delivered to /var/vmail/domain/username/new/ directory. This is visible to the users as their INBOX. Domain admins see the mails under the "username" mailbox. Full admins see the mails under "domain/username" mailbox.
Hi Timo,
Thanks for your help. I think I'm close.
This works: mail_location = maildir:/email/%d/%n:LAYOUT=fs
This doesn't: mail_location = maildir:/email/%d:LAYOUT=fs
The email client doesn't see the directories as folders and nothing is visible. I must be missing something.
On 4.10.2012, at 5.28, Marc Perkel wrote:
Thanks for your help. I think I'm close.
This works: mail_location = maildir:/email/%d/%n:LAYOUT=fs
This doesn't: mail_location = maildir:/email/%d:LAYOUT=fs
The email client doesn't see the directories as folders and nothing is visible. I must be missing something.
Dunno. At least this method of testing works:
create test mail:
doveadm -O -o mail=maildir:/tmp/vmail/domain/user mailbox create INBOX touch /tmp/vmail/domain/user/cur/newmail
test that user@domain works:
./imap -O -o mail=maildir:/tmp/vmail/domain/user:LAYOUT=fs a select inbox
- FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
- OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted.
- 1 EXISTS
test that domain works:
./imap -O -o mail=maildir:/tmp/vmail/domain:LAYOUT=fs b list "" *
- LIST (\HasNoChildren) "/" "user"
- LIST (\HasNoChildren) "/" "INBOX" b OK List completed. c select user
- FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
- OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted.
- 1 EXISTS
- 0 RECENT
test that superuser works:
./imap -O -o mail=maildir:/tmp/vmail:LAYOUT=fs d list "" *
- LIST (\Noselect \HasChildren) "/" "domain"
- LIST (\HasNoChildren) "/" "domain/user"
- LIST (\HasNoChildren) "/" "INBOX" d OK List completed. e select domain/user
- FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
- OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted.
- 1 EXISTS
On 5.10.2012, at 10.45, Micha Krause wrote:
./imap -O -o mail=maildir:/tmp/vmail/domain/user:LAYOUT=fs
Wow, thats a really cool way to debug/test mailboxes, is this documented somewhere?
No. The -O, -o, -k and some other options should be put into some new global.inc where it gets included to all doveadm/dovecot/doveconf man pages..
What does -O do, any other interesting options?
All the global settings are:
-O ignores dovecot.conf and just uses the default settings. -o <key>=<value> can be used multiple times to override any setting -k preserves environment variables (which can also be used to override settings, e.g. MAIL=foo) -c <path> changes dovecot.conf path -i <name> changes to dovecot.conf used by the given instance name -L logs directly to destination specified by log_path/info_log_path/debug_log_path, bypassing log process (allowing logging to different location than normally, log process always logs only to one location)
Timo Sirainen wrote:
-i <name> changes to dovecot.conf used by the given instance name
This does not seem to work, at least not with version 2.1.10:
mail01:~# doveadm instance list
path name last used running
/var/run/dovecot dovecot-mailbox 2012-10-05 19:19:33 yes
/var/run/dovecot-director dovecot-director 2012-10-05 19:20:13 yes
mail01:~# doveadm -c /etc/dovecot-director/dovecot-director.conf director status dparthey@example.org Current: 10.129.3.192 (expires 2012-10-07 20:10:25) Hashed: 10.129.3.192 Initial config: 10.129.3.192
mail01:~# doveadm -i dovecot-director director status dparthey@example.org doveadm(root): Fatal: read(/var/run/dovecot/director-admin) failed: Connection reset by peer
Regards Daniel
On 5.10.2012, at 22.48, Daniel Parthey wrote:
Timo Sirainen wrote:
-i <name> changes to dovecot.conf used by the given instance name
This does not seem to work, at least not with version 2.1.10:
On 03. okt. 2012 21:48, Marc Perkel wrote:
Hi,
I'm looking for some advice to do a really advanced trick with Dovecot. I'm not sure if this can be done. I need to describe first.
I have a spam filtering company that does front end spam filtering. (Junk Email Filter) I want to add a system where I store a copy of spam on a server and make it available to the customer to review and maybe resent on false positives.
I know I could do something simple where I deliver all spam to a domain account and make it available to an administrator. Then if it's a false positive they would drag the message to a "resend" folder. I'll have something the checks the folder one a minute to pick up and resend.
However
What would be very cool is delivering the spam to individual accounts. So a user who logs in individually can see their own spam. But the admin for the domain would be able to see all users. Maybe the users would appear as folders?
Then a master account (me) would be able to log in and see all the domains as folders and the users as folders inside the domains?
One thing I can do is deliver the spam to 3 different places so it's visible on all levels.
I'm just wondering if anyone out there has any ideas about that. And I'll need an authentication system.
Check out the dovecot sieve plugin. I use the following default pre-filter for all users:
require ["regex", "fileinto", "imap4flags"];
# Catch mail tagged as Spam, except Spam retrained and delivered to the mailbox if allof (header :regex "X-DSPAM-Result" "^(Spam|Virus|Bl[ao]cklisted)$", not header :contains "X-DSPAM-Reclassified" "Innocent", not header :contains "Received-SPF" "pass .securityfocus.com") {
# Mark as read #setflag "\\Seen"; addflag "$junk"; # Move into the Junk folder fileinto "INBOX.Junk";
# Stop processing here stop; }
Together with the dovecot antispam plugin this makes the beginnings of a very intuitive system. I just click to remove the junk flag on any false positive, and it gets re-delivered to me.
The dovecot lda also supports a switch to deliver to a specific folder I believe. This would be an alternative if you get the spam delivered through a separate channel anyway.
The other part of your requirements could be met by using dovecot public folders, which I have never used myself. Maybe set up so admins can subscribe to the junk-folder of any user they want ? Refiling false positives might get messy for an admin though.
Regards, Håkon.
Am 03.10.2012 21:48, schrieb Marc Perkel:
I'm looking for some advice to do a really advanced trick with Dovecot. I'm not sure if this can be done. I need to describe first.
I have a spam filtering company that does front end spam filtering. (Junk Email Filter) I want to add a system where I store a copy of spam on a server and make it available to the customer to review and maybe resent on false positives.
this is the job of your filter comapny first, anyway , dont use them anymore and use i.e amavis with quarantaine i dont think other cases make sense in real by getting very complicated
Best Regards MfG Robert Schetterer
On 10/3/2012 11:36 PM, Robert Schetterer wrote:
I'm looking for some advice to do a really advanced trick with Dovecot. I'm not sure if this can be done. I need to describe first.
I have a spam filtering company that does front end spam filtering. (Junk Email Filter) I want to add a system where I store a copy of spam on a server and make it available to the customer to review and maybe resent on false positives.
Am 03.10.2012 21:48, schrieb Marc Perkel: this is the job of your filter comapny first, anyway , dont use them anymore and use i.e amavis with quarantaine i dont think other cases make sense in real by getting very complicated
I am the spam filtering company. :)
participants (8)
-
Daniel Parthey
-
Eduardo Casarero
-
Håkon Alstadheim
-
Marc Perkel
-
Micha Krause
-
Robert Schetterer
-
Steve Campbell
-
Timo Sirainen