post-delivery virus scan
I have discovered that many times the virus definitions I use for scanning messages (ClamAV, with the unofficial signatures http://sanesecurity.com/usage/linux-scripts/) are updated some time after my server has received an infected email. It seems the virus creators are trying to race the virus definition creators to see who can deliver first; more than half of the infected messages are found after they’ve been delivered. Great.
To help detect and remove the infected messages after they’ve been delivered to users’ mailboxes, I created a small script that iterates the INBOX and Junk mailbox directories, scans recent messages for viruses, and deletes them if found. The source of my script (run via cron) is here: https://gitlab.koehn.com/snippets/9
Unfortunately Dovecot doesn’t like it if messages are deleted (dbox) out from under it. I tried a doveadm force-resync on the folder containing the messages, but it seems Dovecot is still unhappy. At least on the new version (2.2.26.0) it doesn’t crash; 2.2.25 would panic and coredump when it discovered messages had been deleted.
I’m wondering if there’s a better way to scan recent messages and eradicate them so the Dovecot isn’t upset when it happens. Maybe using doveadm search? Looking for suggestions.
Brad
On 2016-11-09 21:36, Brad Koehn wrote:
I have discovered that many times the virus definitions I use for scanning messages (ClamAV, with the unofficial signatures http://sanesecurity.com/usage/linux-scripts/) are updated some time after my server has received an infected email. It seems the virus creators are trying to race the virus definition creators to see who can deliver first; more than half of the infected messages are found after they’ve been delivered. Great.
To help detect and remove the infected messages after they’ve been delivered to users’ mailboxes, I created a small script that iterates the INBOX and Junk mailbox directories, scans recent messages for viruses, and deletes them if found. The source of my script (run via cron) is here: https://gitlab.koehn.com/snippets/9
Unfortunately Dovecot doesn’t like it if messages are deleted (dbox) out from under it. I tried a doveadm force-resync on the folder containing the messages, but it seems Dovecot is still unhappy. At least on the new version (2.2.26.0) it doesn’t crash; 2.2.25 would panic and coredump when it discovered messages had been deleted.
I’m wondering if there’s a better way to scan recent messages and eradicate them so the Dovecot isn’t upset when it happens. Maybe using doveadm search? Looking for suggestions.
leave an empty message behind with the same name as deleted message ?
-- key ID: 0x4BFEBB31
I’ve decided to try this approach. I’ve updated my script as follows:
#!/bin/bash
# Scan junk folders for messages containing viruses we didn't have definitions # for when the mail was received. Truncate the body of infected messages and # replace the body with a message.
cd /var/mail
for dir in $( find . \( -name Junk -o -name INBOX \) -type d ) ; do files=$( find "$dir" -type f -name u.\* -mtime -14 -print ) for file in $files ; do /usr/local/bin/clamdscan --quiet --fdpass "$file" if [ $? -eq 1 ] ; then sed -i '/^$/,$d' "$file" echo "\r\n\r\n[The body of this message contained a virus and was deleted.]" >> "$file" fi done done
We’ll see if that does the trick.
On Nov 9, 2016, at 6:12 PM, mick crane mick.crane@gmail.com wrote:
On 2016-11-09 21:36, Brad Koehn wrote:
I have discovered that many times the virus definitions I use for scanning messages (ClamAV, with the unofficial signatures http://sanesecurity.com/usage/linux-scripts/) are updated some time after my server has received an infected email. It seems the virus creators are trying to race the virus definition creators to see who can deliver first; more than half of the infected messages are found after they’ve been delivered. Great. To help detect and remove the infected messages after they’ve been delivered to users’ mailboxes, I created a small script that iterates the INBOX and Junk mailbox directories, scans recent messages for viruses, and deletes them if found. The source of my script (run via cron) is here: https://gitlab.koehn.com/snippets/9 Unfortunately Dovecot doesn’t like it if messages are deleted (dbox) out from under it. I tried a doveadm force-resync on the folder containing the messages, but it seems Dovecot is still unhappy. At least on the new version (2.2.26.0) it doesn’t crash; 2.2.25 would panic and coredump when it discovered messages had been deleted. I’m wondering if there’s a better way to scan recent messages and eradicate them so the Dovecot isn’t upset when it happens. Maybe using doveadm search? Looking for suggestions.
leave an empty message behind with the same name as deleted message ?
-- key ID: 0x4BFEBB31
On 09.11.2016 23:36, Brad Koehn wrote:
I have discovered that many times the virus definitions I use for scanning messages (ClamAV, with the unofficial signatures http://sanesecurity.com/usage/linux-scripts/) are updated some time after my server has received an infected email. It seems the virus creators are trying to race the virus definition creators to see who can deliver first; more than half of the infected messages are found after they’ve been delivered. Great.
To help detect and remove the infected messages after they’ve been delivered to users’ mailboxes, I created a small script that iterates the INBOX and Junk mailbox directories, scans recent messages for viruses, and deletes them if found. The source of my script (run via cron) is here: https://gitlab.koehn.com/snippets/9
Unfortunately Dovecot doesn’t like it if messages are deleted (dbox) out from under it. I tried a doveadm force-resync on the folder containing the messages, but it seems Dovecot is still unhappy. At least on the new version (2.2.26.0) it doesn’t crash; 2.2.25 would panic and coredump when it discovered messages had been deleted.
I’m wondering if there’s a better way to scan recent messages and eradicate them so the Dovecot isn’t upset when it happens. Maybe using doveadm search? Looking for suggestions. The removal should if possible be done with the doveadm cli tool or using the doveadm http api.
br, Teemu Huovila
Brad
Op 11/10/2016 om 10:05 AM schreef Teemu Huovila:
On 09.11.2016 23:36, Brad Koehn wrote:
I have discovered that many times the virus definitions I use for scanning messages (ClamAV, with the unofficial signatures http://sanesecurity.com/usage/linux-scripts/) are updated some time after my server has received an infected email. It seems the virus creators are trying to race the virus definition creators to see who can deliver first; more than half of the infected messages are found after they’ve been delivered. Great.
To help detect and remove the infected messages after they’ve been delivered to users’ mailboxes, I created a small script that iterates the INBOX and Junk mailbox directories, scans recent messages for viruses, and deletes them if found. The source of my script (run via cron) is here: https://gitlab.koehn.com/snippets/9
Unfortunately Dovecot doesn’t like it if messages are deleted (dbox) out from under it. I tried a doveadm force-resync on the folder containing the messages, but it seems Dovecot is still unhappy. At least on the new version (2.2.26.0) it doesn’t crash; 2.2.25 would panic and coredump when it discovered messages had been deleted.
I’m wondering if there’s a better way to scan recent messages and eradicate them so the Dovecot isn’t upset when it happens. Maybe using doveadm search? Looking for suggestions. The removal should if possible be done with the doveadm cli tool or using the doveadm http api.
Still, Dovecot should handle external removal of messages gracefully. What exactly happens?
Regards,
Stephan.
On Nov 10, 2016, at 3:38 AM, Stephan Bosch stephan@rename-it.nl wrote:
Op 11/10/2016 om 10:05 AM schreef Teemu Huovila:
On 09.11.2016 23:36, Brad Koehn wrote:
I have discovered that many times the virus definitions I use for scanning messages (ClamAV, with the unofficial signatures http://sanesecurity.com/usage/linux-scripts/) are updated some time after my server has received an infected email. It seems the virus creators are trying to race the virus definition creators to see who can deliver first; more than half of the infected messages are found after they’ve been delivered. Great.
To help detect and remove the infected messages after they’ve been delivered to users’ mailboxes, I created a small script that iterates the INBOX and Junk mailbox directories, scans recent messages for viruses, and deletes them if found. The source of my script (run via cron) is here: https://gitlab.koehn.com/snippets/9
Unfortunately Dovecot doesn’t like it if messages are deleted (dbox) out from under it. I tried a doveadm force-resync on the folder containing the messages, but it seems Dovecot is still unhappy. At least on the new version (2.2.26.0) it doesn’t crash; 2.2.25 would panic and coredump when it discovered messages had been deleted.
I’m wondering if there’s a better way to scan recent messages and eradicate them so the Dovecot isn’t upset when it happens. Maybe using doveadm search? Looking for suggestions. The removal should if possible be done with the doveadm cli tool or using the doveadm http api.
Still, Dovecot should handle external removal of messages gracefully. What exactly happens?
Regards,
Stephan.
On Dovecot 2.2.5: Nov 9 14:32:11 ds postfix/anvil[13298]: statistics: max cache size 2 at Nov 9 14:23:08 Nov 9 14:32:29 ds dovecot: imap(user): Error: Recent flags state corrupted for mailbox Junk Nov 9 14:32:29 ds dovecot: imap(user): Error: /var/mail/user_dbox/mailboxes/Junk/dbox-Mails/dovecot.index reset, view is now inconsistent Nov 9 14:32:29 ds dovecot: imap(user): Panic: Message count decreased Nov 9 14:32:29 ds dovecot: imap(user): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x89cc0) [0x7f0b64641cc0] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x89d9e) [0x7f0b646 41d9e] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f0b645e4165] -> dovecot/imap() [0x42259c] -> dovecot/imap(imap_sync_more+0x104) [0x422f14] -> dovecot/imap() [0x410720] -> do vecot/imap() [0x4108d1] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0x52147) [0x7f0b64917147] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0xe2) [0x7f0b64654992] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x93) [0x7f0b64655d83] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7f0b64654b45] -> /usr/l ocal/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f0b64654cf8] -> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f0b645ea243] -> dovecot/imap(main+0x312) [0x40c612 ] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f0b64214f45] -> dovecot/imap() [0x40c780] Nov 9 14:32:30 ds dovecot: imap(bkc): Fatal: master: service(imap): child 8456 killed with signal 6 (core dumped)
On Dovecot 2.2.6.0: Nov 10 10:35:13 ds dovecot: imap(user): Error: Recent flags state corrupted for mailbox Junk Nov 10 10:35:13 ds dovecot: imap(user): Error: /var/mail/user_dbox/mailboxes/Junk/dbox-Mails/dovecot.index reset, view is now inconsistent Nov 10 10:35:13 ds dovecot: imap(user): IMAP session state is inconsistent, please relogin. in=6212 out=49396
Op 10-11-2016 om 12:25 schreef Brad Koehn:
On Nov 10, 2016, at 3:38 AM, Stephan Bosch stephan@rename-it.nl wrote:
Op 11/10/2016 om 10:05 AM schreef Teemu Huovila:
On 09.11.2016 23:36, Brad Koehn wrote:
I’m wondering if there’s a better way to scan recent messages and eradicate them so the Dovecot isn’t upset when it happens. Maybe using doveadm search? Looking for suggestions. The removal should if possible be done with the doveadm cli tool or using the doveadm http api. Still, Dovecot should handle external removal of messages gracefully. What exactly happens? On Dovecot 2.2.6.0: Nov 10 10:35:13 ds dovecot: imap(user): Error: Recent flags state corrupted for mailbox Junk Nov 10 10:35:13 ds dovecot: imap(user): Error: /var/mail/user_dbox/mailboxes/Junk/dbox-Mails/dovecot.index reset, view is now inconsistent Nov 10 10:35:13 ds dovecot: imap(user): IMAP session state is inconsistent, please relogin. in=6212 out=49396
OK, so at least it doesn't panic anymore in the last release. Also, the mailbox is fixed upon relogin. To prevent the remaining errors from occurring, i.e. to gracefully remove messages, you can use the doveadm expunge command (it has a man page).
Regards,
Stephan.
On Wed, 9 Nov 2016 15:36:33 -0600 Brad Koehn wrote:
[ ... ]
To help detect and remove the infected messages after they’ve been delivered to users’ mailboxes, I created a small script that iterates the INBOX and Junk mailbox directories, scans recent messages for viruses, and deletes them if found. The source of my script (run via cron) is here: https://gitlab.koehn.com/snippets/9
Bad idea. The user may already taken the action needed for infection. And what about legal aspects? In my country (Germany), information suppression would be punishable.
Just my 0.02 €, Frank
Turns out the technical part of your reasoning is correct: MUAs that have downloaded the message don’t get any updates, and hold onto the infected message. No legal ramifications here; it’s my personal server, and it’s in the US. Strange to think that deleting the content of a message would somehow be worse than deleting the content and the headers.
On Nov 10, 2016, at 6:47 AM, Frank Elsner frank@moltke28.b.shuttle.de wrote:
On Wed, 9 Nov 2016 15:36:33 -0600 Brad Koehn wrote:
[ ... ]
To help detect and remove the infected messages after they’ve been delivered to users’ mailboxes, I created a small script that iterates the INBOX and Junk mailbox directories, scans recent messages for viruses, and deletes them if found. The source of my script (run via cron) is here: https://gitlab.koehn.com/snippets/9
Bad idea. The user may already taken the action needed for infection. And what about legal aspects? In my country (Germany), information suppression would be punishable.
Just my 0.02 €, Frank
10.11.2016 16:47, Frank Elsner пишет:
On Wed, 9 Nov 2016 15:36:33 -0600 Brad Koehn wrote:
[ ... ]
To help detect and remove the infected messages after they’ve been delivered to users’ mailboxes, I created a small script that iterates the INBOX and Junk mailbox directories, scans recent messages for viruses, and deletes them if found. The source of my script (run via cron) is here: https://gitlab.koehn.com/snippets/9 Bad idea. The user may already taken the action needed for infection. And what about legal aspects? Is it legal to redistribute malware in Germany? :-D In my country (Germany), information suppression would be punishable.
I guess main problem here is that this is not on access scan, i.e. even if virus can be already detected by newer virus database, it can be accessed by user before it rescanned.
participants (6)
-
Brad Koehn
-
Dmitry Melekhov
-
Frank Elsner
-
mick crane
-
Stephan Bosch
-
Teemu Huovila