fix SIS attachment errors

[Webert de Souza Lima]

Hi, I've sent some e-mails about this before but since there was no answers
I'll write it differently, with different information.

I'm using SIS (Single Instance Attachment Storage).
For any reason that is not relevant now, many attachments are missing and
the messages can't be fetched:

Error:
read(attachments-connector(zlib(/dovecot/mdbox/bar.example/foo/storage/m.1)))
failed:
read(/dovecot/attach/bar.example/23/ae/23aed008c1f32f048afd38d9aae68c5aeae2d17a9170e28c60c75a02ec199ef4e7079cd92988ad857bd6e12cd24cdd7619bd29f26edeec842a6911bb14a86944-fb0b6a214dfa63573c1f00009331bd36[base64:19
b/l]) failed:
open(/dovecot/attach/bar.example/23/ae/23aed008c1f32f048afd38d9aae68c5aeae2d17a9170e28c60c75a02ec199ef4e7079cd92988ad857bd6e12cd24cdd7619bd29f26edeec842a6911bb14a86944-fb0b6a214dfa63573c1f00009331bd36)
failed: No such file or directory

in this specific case, the /dovecot/attach/bar.example/23/ae/ director
doesn't exist.
In other cases, just one file is missing so I would assume the hardlink
could be recreated and it would work.

If I create the missing file (with touch or whatever), I get the following
errors:
Error:
read(/dovecot/attach/bar.example/23/ae/23aed008c1f32f048afd38d9aae68c5aeae2d17a9170e28c60c75a02ec199ef4e7079cd92988ad857bd6e12cd24cdd7619bd29f26edeec842a6911bb14a86944-fb0b6a214dfa63573c1f00009331bd36[base64:19
b/l]) failed: Stream is smaller than expected (0 < 483065)
Error:
read(attachments-connector(zlib(/dovecot/mdbox/bar.example/foo/storage/m.1)))
failed:
read(/dovecot/attach/bar.example/23/ae/23aed008c1f32f048afd38d9aae68c5aeae2d17a9170e28c60c75a02ec199ef4e7079cd92988ad857bd6e12cd24cdd7619bd29f26edeec842a6911bb14a86944-fb0b6a214dfa63573c1f00009331bd36[base64:19
b/l]) failed: Stream is smaller than expected (0 < 483065)
Error: fetch(body) failed for box=INBOX uid=15: BUG: Unknown internal error

If I try to fill the file with the amount of bytes it complains about with
the following command:

$ dd if=/dev/zero
of=/dovecot/attach/bar.example/23/ae/23aed008c1f32f048afd38d9aae68c5aeae2d17a9170e28c60c75a02ec199ef4e7079cd92988ad857bd6e12cd24cdd7619bd29f26edeec842a6911bb14a86944-fb0b6a214dfa63573c1f00009331bd36
bs=1 count=483065

then I get the following error:

Error:
read(/dovecot/attach/bar.example/23/ae/23aed008c1f32f048afd38d9aae68c5aeae2d17a9170e28c60c75a02ec199ef4e7079cd92988ad857bd6e12cd24cdd7619bd29f26edeec842a6911bb14a86944-fb0b6a214dfa63573c1f00009331bd36[base64:19
b/l]) failed: Stream is larger than expected (483928 > 483065, eof=0)
Error:
read(attachments-connector(zlib(/srv/dovecot/mdbox/bar.example/foo/storage/m.1)))
failed:
read(//dovecot/attach/bar.example/23/ae/23aed008c1f32f048afd38d9aae68c5aeae2d17a9170e28c60c75a02ec199ef4e7079cd92988ad857bd6e12cd24cdd7619bd29f26edeec842a6911bb14a86944-fb0b6a214dfa63573c1f00009331bd36[base64:19
b/l]) failed: Stream is larger than expected (483928 > 483065, eof=0)
Error: fetch(body) failed for box=INBOX uid=15: BUG: Unknown internal error

Based on this I have a few questions:
1. Is there a way, or a tool to scan all mailboxes to get all the messages
that have compromised attachments?

2. is there a way to "fix" the missing files (even if creating fake files
or removing the attachments information from the messages)

3. What I need is to migrate these boxes using doveadm backup/sync, which
fails when these errors occur. Is is possible to ignore them or is there
another tool that would do it?

Thank you.

Webert Lima
Belo Horizonte, Brasil