[Dovecot] Some questions about mdbox

[Timo Sirainen]

On 15.2.2011, at 22.16, Maria Arrea wrote:

> *1º How should I restore from backup with mdbox & separate indexes?*
> 
> I have read in dovecot list archive about restoring the full user mailbox to a temp location and then run something like:
> 
> /doveadm import mdbox:/tmp/restored_mailbox/ "" savedsince 2011-01-01 / In my schema indexes and mail stores are in separated paths. What (and where!) should I copy to the temp location and how do I restore it?

The import parameter works exactly the same as mail_location. So if you for example restore user's files to:

/tmp/backup/mdbox/storage/m.*
/tmp/backup/indexes/dovecot.index*

You can use "doveadm import mdbox:/tmp/backup/mdbox:INDEX=/tmp/backup/indexes".

> *2º Disaster recovery: indexes & mail store mismatch*
> 
> When creating backups, indexes and mail store are in different paths. My backup takes several hours, so indexes are copied BEFORE mail stores. Is the backup consistent?

Well, this isn't ideal.. Better would probably be if indexes were backed up after. Now it's possible that when restoring a mail there are some mails that aren't in index yet. You could run "doveadm force-resync" to make sure they're all added.

> What would happens if I lose indexes? Can I still recover the messages? Reading the doc it seems I would lose all the message flags and all the messages would be in INBOX, is this right?

doveadm force-resync makes all messages visible, even if all flags are lost. The messages will show up where they were saved to the first time (INBOX and Sent mainly, also if Sieve fileinto was used then those too). So mainly if user/client copied/moved messages around, those changes would be lost.

Anyway, since you have backups and Dovecot also generates dovecot.index.backup files, you should always have some versions of them available. And mdbox index rebuilding tries really hard to use those files to avoid losing changes.

> *3º Performance tuning with mdbox*
> 
> I have read in dovecot archive about using mdbox_rotate_size & mdbox_rotate_interval to balance between a lot of smaill files (like Maildir) or really big files (a "variant" of mbox). I have tested with a big vaule of mdbox_rotate_size (100 MB) and is a complete disaster for backup:if a user delete a message, then a nightly cronjob purges the mailbox (refcount=0) a new mdbox .m file is created!. So deleting a 1 k message from the mailbox of an user is a new 99 MB file to backup. Any hint about this?

Well, that's how it is intended to work.. Typically users would delete only recent mails, so if you rotate files daily, then deletions would happen only from the file that hasn't yet even been backed up. Of course, with daily rotations you rarely get even close to 100 MB files.

> *5º Recover a message with refcount=0*
> 
> Is there any way to recover a message with refcount=0 without using a backup? I am thinking about the following plan:
> 
> /1º a user gets her mail via pop3 (or delete a message via imap and empties trash)
> 2º the message is still stored in my mail server, but has refcount=0
> 3º nightly backup is performed
> 4º nightly cronjob purges the mailbox (deleting messaged with refcount=0)/
> 
> Using this schema the backup is bigger, but If I can recover files with refcount=0 from backup we can give our users a safeguard against accidentaly deleted pop3 mail.

There is no clean way to do this currently. An ugly way would be to delete dovecot.map.index* files and rebuild index. The rebuilding sees that there are some mails that don't exist in any mailbox indexes, and assigns them to the mailbox where they initially were saved to. No (flag) changes are lost by doing this.