[Dovecot] backing up maildir dovecot files

[Steffen Kaiser]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 13 Feb 2013, Reindl Harald wrote:

>>> Thanks, if I did:
>>>
>>> rsync --force --delete-after -tPrlHpogEAXz /folder/ /backups/newmailstore/
>>>
>>> could I then tar up the newmailstore folder with something like tar
>>> zcf without messing anything up?
>>
>> keep in mind, that the backup of the Maildir with rsync is no 100% consistent snapshot, because of the filename
>> renames; otherwise: yes
>
> who would rename them and why?

"/folder/" would be a Maildir on a production mail server. If rsync runs 
(without help from other funtionality, such as LVM or ZFS snapshots) and 
at the same time someone accesses the Maildir and:

a) "see"s a message, which lets Dovecot rename the message file from 
***/new/ to  ***/cur

b) tags a message with a keyword, which lets Dovecot rename the file to 
have some lower-case letter

c) untags a keyword -> remove that letter from the filename

d) changes Deleted, Read, Answered status --> add/remove an upper-case 
letter from filename

rsync might have cached a message with an old filename no longer 
physically present on disk. You will see a "XYZ vanished" message and 
that particular message is not backuped, but removed from backup. Some 
scripts use rsync in a loop in assumption, that no Maildir is accessed 
that often, that you get a clean run of rsync eventually.

Maybe, if you tag a message with a new keyword, rsync already copied 
dovecot-keywords without the new keyword, but copy the message, when its 
filename has the keyword-letter.

In fact, if I want to make 100% sure I get an exact copy, I do this:

rsync /from /to
rsync /from /to
mv /from /from_locked
sleep 1
rsync /from_locked /to
mv /from_locked /from

"/from" is the base directory of the Maildir. In my environment that 
causes tempfails on delivery and internal server errors on IMAP/POP3 
access. But the time between the two mv's is very small, because the main 
differences are handled with the first two rsync's. As you seen, I copy 
one user after another, which breaks hardlinks between users, but keeps 
memory footprint of rsync low and as well as the downtime.

I do make my usual daily backups without the loop and the mv's, because it 
is very seldom I get the "vanished" messages. I suppose, because the 
script runs two hours after midnight.

- -- 
Steffen Kaiser
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQEVAwUBURuQIF3r2wJMiz2NAQKgjgf8CtyeO6pONFU038x8O2wggcntdvGyeg8Z
Uq8KR0QFVg1n0HNDaa33OYN4IxSHX8zNvElf+wc0ejQ3NUOPVDl2mdm4iEihyOYv
Veb/p2iK671Nrs8nB7USwx7OE9vY8IYoB/ZSXrXGWowqOqRQIcJWHAfZ9Ewj3Rg/
iRGMUNCn7UfEDfWl+F5yWpdp/+3xJGxoWeWaegW/yfTzlJ5nKffS/SAfJlUm7zuV
u31JL4fjk25uGG7alzrCxOq0z4A3PvcpGag2nkfIRbrLLmo4Wzr+09Bd2zqmSn74
/PaHlxFS6a2uy7ugqdd5kxfiZHnOS2/d6JgV428I8qS0CWugEPuFbg==
=mw5e
-----END PGP SIGNATURE-----