Dear Aki,
I switched to "gz" now, since "zstd" also gave some errors on
writing to files.
What kind of errors?
I dont know if "xz" compression or "zstd" shreddered my MDBOX
Files, but I lost 4 days of mail. (a couple of thousand mails).
After restoring the backup (what was made after switching to
version 2.3.11.3) I still have some broken mdfiles, but not too
many.
Interestingly always in the "Sent" Mailbox on a number of Users.
I just can not go back to 20.08.2020 before I updated to 2.3.11.3 -
too many emails would have been lost.
So - the current status is 2.3.11.3,
with "gz" compression.
I force-synced and re-indexed all
the mdbox files (250 GB), but still have some broken - please
how can I fix those ?
Note that force-resync doesn't read through the mails to verify whether they are readable. It just verifies that the indexes and metadata is ok. The only way to verify that all mails are readable is to actually try to read all of their text (e.g. doveadm fetch -u user text mailbox Sent > /dev/null).
You stated :
(In theory you could leave the existing mails xz-compressed, but best
would be to re-compress everything via dsync so old mails can be read
when we eventually remove xz support.)
what is the optimal way to do that, expecially not loosing the
index (?) for outlook/thunderbird, for third party tools that rely
on some index/hash (dunno how exactly that works) ?
From your previous mail:
sudo doveadm backup -D -u "${mailbox_username}" "mdbox:/home/vmail/virtualmailboxes/${mailbox_username}_backup"
sudo service dovecot stop
sudo mv "/home/vmail/virtualmailboxes/${mailbox_username}" "/home/vmail/virtualmailboxes/${mailbox_username}_original"
sudo mv "/home/vmail/virtualmailboxes/${mailbox_username}_backup" "/home/vmail/virtualmailboxes/${mailbox_username}"
sudo service dovecot start
The problem is that in your config you have:
mail_location = mdbox:/home/vmail/virtualmailboxes/%n:DIRNAME=dbox-Mails-nocollision-random-KOKxNmMJkEBeCitBhFwS
You need to preserve the DIRNAME. So:
sudo doveadm backup -D -u "${mailbox_username}" "mdbox:/home/vmail/virtualmailboxes/${mailbox_username}_backup:DIRNAME=dbox-Mails-nocollision-random-KOKxNmMJkEBeCitBhFwS"
sudo service dovecot stop
sudo mv "/home/vmail/virtualmailboxes/${mailbox_username}" "/home/vmail/virtualmailboxes/${mailbox_username}_original"
sudo mv "/home/vmail/virtualmailboxes/${mailbox_username}_backup" "/home/vmail/virtualmailboxes/${mailbox_username}"
sudo service dovecot start
However, the dsync will likely fail as well because it can't read some of the mails. So you'll need to fix those in any case first. That's a bit trickier:
Sep 3 08:33:25 lxc-imap dovecot:
imap(mpaul)<48684><2/9E5mKuAezAqKjk>: Error: Mailbox
Sent: UID=2171:
read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119))
failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119)
failed: Broken pipe (FETCH BODY[])
For example here the best fix would be to try to preserve the mail as best as possible:
a) Preserve as much of the text as possible and expunge the broken mail:
doveadm fetch -u mpaul text mailbox Sent uid 2171 > msg.broken
doveadm expunge -u mpaul mailbox Sent uid 2171
b) You could also see if the issue is that Dovecot just can't read a properly compressed email, or if the issue was that it wrote broken emails. You can extract the raw compressed mail with:
doveadm -o mail_plugins=virtual fetch -u mpaul text mailbox Sent uid 2171 | tail -n +2 > msg.broken
Also this way you can see if the broken mail is actually xz or zstd or zlib. It would be nice to know if there are any zstd or zlib compressed mails that have problems. We did a lot of stress testing with zstd and also with xz, but haven't been able to reproduce any problems. It's also strange that it says taht the error is "Broken pipe" - that doesn't indicate that the mail is corrupted but that there is something more strange going on. So perhaps you don't actually have any mails written as corrupted, but Dovecot is just somehow having trouble reading the mails.