post-delivery virus scan

[Brad Koehn]

> On Nov 10, 2016, at 3:38 AM, Stephan Bosch <stephan at rename-it.nl> wrote:
> 
> Op 11/10/2016 om 10:05 AM schreef Teemu Huovila:
>> 
>> On 09.11.2016 23:36, Brad Koehn wrote:
>>> I have discovered that many times the virus definitions I use for scanning messages (ClamAV, with the unofficial signatures http://sanesecurity.com/usage/linux-scripts/) are updated some time after my server has received an infected email. It seems the virus creators are trying to race the virus definition creators to see who can deliver first; more than half of the infected messages are found after they’ve been delivered. Great. 
>>> 
>>> To help detect and remove the infected messages after they’ve been delivered to users’ mailboxes, I created a small script that iterates the INBOX and Junk mailbox directories, scans recent messages for viruses, and deletes them if found. The source of my script (run via cron) is here: https://gitlab.koehn.com/snippets/9
>>> 
>>> Unfortunately Dovecot doesn’t like it if messages are deleted (dbox) out from under it. I tried a doveadm force-resync on the folder containing the messages, but it seems Dovecot is still unhappy. At least on the new version (2.2.26.0) it doesn’t crash; 2.2.25 would panic and coredump when it discovered messages had been deleted. 
>>> 
>>> I’m wondering if there’s a better way to scan recent messages and eradicate them so the Dovecot isn’t upset when it happens. Maybe using doveadm search? Looking for suggestions. 
>> The removal should if possible be done with the doveadm cli tool or using the doveadm http api.
> 
> Still, Dovecot should handle external removal of messages gracefully.
> What exactly happens?
> 
> Regards,
> 
> Stephan.

On Dovecot 2.2.5:
Nov  9 14:32:11 ds postfix/anvil[13298]: statistics: max cache size 2 at Nov  9 14:23:08
Nov  9 14:32:29 ds dovecot: imap(user): Error: Recent flags state corrupted for mailbox Junk
Nov  9 14:32:29 ds dovecot: imap(user): Error: /var/mail/user_dbox/mailboxes/Junk/dbox-Mails/dovecot.index reset, view is now inconsistent
Nov  9 14:32:29 ds dovecot: imap(user): Panic: Message count decreased
Nov  9 14:32:29 ds dovecot: imap(user): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x89cc0) [0x7f0b64641cc0] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x89d9e) [0x7f0b646
41d9e] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f0b645e4165] -> dovecot/imap() [0x42259c] -> dovecot/imap(imap_sync_more+0x104) [0x422f14] -> dovecot/imap() [0x410720] -> do
vecot/imap() [0x4108d1] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0x52147) [0x7f0b64917147] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0xe2) [0x7f0b64654992]
-> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x93) [0x7f0b64655d83] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7f0b64654b45] -> /usr/l
ocal/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f0b64654cf8] -> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f0b645ea243] -> dovecot/imap(main+0x312) [0x40c612
] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f0b64214f45] -> dovecot/imap() [0x40c780]
Nov  9 14:32:30 ds dovecot: imap(bkc): Fatal: master: service(imap): child 8456 killed with signal 6 (core dumped)


On Dovecot 2.2.6.0:
Nov 10 10:35:13 ds dovecot: imap(user): Error: Recent flags state corrupted for mailbox Junk
Nov 10 10:35:13 ds dovecot: imap(user): Error: /var/mail/user_dbox/mailboxes/Junk/dbox-Mails/dovecot.index reset, view is now inconsistent
Nov 10 10:35:13 ds dovecot: imap(user): IMAP session state is inconsistent, please relogin. in=6212 out=49396