fix SIS attachment errors

[Webert de Souza Lima]

To whom it may interest;

With the help of Aki Tuomi I've found a way to remove such errors and move
forward, in a way that could be automated.
As this might be a problem to others and there seems to be no discussion
about it, i'll share it with you.

What I did, essentially, was to write a shell script that do the following,
per user:

- read all the mailboxes with `doveadm fetch -u $username text all` and
redirect errors to a file
- identify all missing attachments' paths from the file created previously
and try to create a hardlink to it. Any file with the same hash (before
`-`) is good.
- identify all mailboxes and uids from messages there are still broken (the
same error file created before should have this information) and fetch
them, and save them elsewhere.
- after fetching and saving, expunge such messages.
- use doveadm save to put the messages back. They'll be without the
attachments but also without any errors.

There are some gotchas to do the above, and to automate that, so I'll be
happy to help if anyone needs.

Thank you.

On Wed, Oct 5, 2016 at 3:59 PM Webert de Souza Lima <webert.boss at gmail.com>
wrote:

Hi, I've sent some e-mails about this before but since there was no answers
I'll write it differently, with different information.

I'm using SIS (Single Instance Attachment Storage).
For any reason that is not relevant now, many attachments are missing and
the messages can't be fetched:

Error:
read(attachments-connector(zlib(/dovecot/mdbox/bar.example/foo/storage/m.1)))
failed:
read(/dovecot/attach/bar.example/23/ae/23aed008c1f32f048afd38d9aae68c5aeae2d17a9170e28c60c75a02ec199ef4e7079cd92988ad857bd6e12cd24cdd7619bd29f26edeec842a6911bb14a86944-fb0b6a214dfa63573c1f00009331bd36[base64:19
b/l]) failed:
open(/dovecot/attach/bar.example/23/ae/23aed008c1f32f048afd38d9aae68c5aeae2d17a9170e28c60c75a02ec199ef4e7079cd92988ad857bd6e12cd24cdd7619bd29f26edeec842a6911bb14a86944-fb0b6a214dfa63573c1f00009331bd36)
failed: No such file or directory

in this specific case, the /dovecot/attach/bar.example/23/ae/ director
doesn't exist.
In other cases, just one file is missing so I would assume the hardlink
could be recreated and it would work.

If I create the missing file (with touch or whatever), I get the following
errors:
Error:
read(/dovecot/attach/bar.example/23/ae/23aed008c1f32f048afd38d9aae68c5aeae2d17a9170e28c60c75a02ec199ef4e7079cd92988ad857bd6e12cd24cdd7619bd29f26edeec842a6911bb14a86944-fb0b6a214dfa63573c1f00009331bd36[base64:19
b/l]) failed: Stream is smaller than expected (0 < 483065)
Error:
read(attachments-connector(zlib(/dovecot/mdbox/bar.example/foo/storage/m.1)))
failed:
read(/dovecot/attach/bar.example/23/ae/23aed008c1f32f048afd38d9aae68c5aeae2d17a9170e28c60c75a02ec199ef4e7079cd92988ad857bd6e12cd24cdd7619bd29f26edeec842a6911bb14a86944-fb0b6a214dfa63573c1f00009331bd36[base64:19
b/l]) failed: Stream is smaller than expected (0 < 483065)
Error: fetch(body) failed for box=INBOX uid=15: BUG: Unknown internal error

If I try to fill the file with the amount of bytes it complains about with
the following command:

$ dd if=/dev/zero
of=/dovecot/attach/bar.example/23/ae/23aed008c1f32f048afd38d9aae68c5aeae2d17a9170e28c60c75a02ec199ef4e7079cd92988ad857bd6e12cd24cdd7619bd29f26edeec842a6911bb14a86944-fb0b6a214dfa63573c1f00009331bd36
bs=1 count=483065

then I get the following error:

Error:
read(/dovecot/attach/bar.example/23/ae/23aed008c1f32f048afd38d9aae68c5aeae2d17a9170e28c60c75a02ec199ef4e7079cd92988ad857bd6e12cd24cdd7619bd29f26edeec842a6911bb14a86944-fb0b6a214dfa63573c1f00009331bd36[base64:19
b/l]) failed: Stream is larger than expected (483928 > 483065, eof=0)
Error:
read(attachments-connector(zlib(/srv/dovecot/mdbox/bar.example/foo/storage/m.1)))
failed:
read(//dovecot/attach/bar.example/23/ae/23aed008c1f32f048afd38d9aae68c5aeae2d17a9170e28c60c75a02ec199ef4e7079cd92988ad857bd6e12cd24cdd7619bd29f26edeec842a6911bb14a86944-fb0b6a214dfa63573c1f00009331bd36[base64:19
b/l]) failed: Stream is larger than expected (483928 > 483065, eof=0)
Error: fetch(body) failed for box=INBOX uid=15: BUG: Unknown internal error

Based on this I have a few questions:
1. Is there a way, or a tool to scan all mailboxes to get all the messages
that have compromised attachments?

2. is there a way to "fix" the missing files (even if creating fake files
or removing the attachments information from the messages)

3. What I need is to migrate these boxes using doveadm backup/sync, which
fails when these errors occur. Is is possible to ignore them or is there
another tool that would do it?

Thank you.

Webert Lima
Belo Horizonte, Brasil