[Dovecot] backup mdbox best strategy

[claus]

Am 2013-12-07 10:04, schrieb Robert Schetterer:
> Am 07.12.2013 08:53, schrieb Claus:
>> Thanks for your suggestion to use maildir, but
>> i'm sure the key problem are the large Number of small files.
>> the backup goes first to diskstorage and afterwards to LTO4 Tapes.
>> We backup with EMC-Networker. The backuptime of large files (some GB)
>> are normal.
>> 
>> I wonder if there is no one who operates a large Mailsystem with mdbox
>> and has suggestions to backup this?
> 
> you are able to backup and restore mdbox with dsync ( never tested ),
> but its not like with maildir simply i.e rsync the mailbox or one only 
> mail
> so you may find some combined strategy using both formats depending to
> your kind of storage, as people said i wouldnt recommand backup to tape
> in a direct way

Thanks Robert!
As i wrote the backup goes first to diskstorage and than to tape.


> 
> why not simply do some testing with only one mdbox mailbox , should be
> easy with dsync
> 
>> 
>> 
>> Claus
>> 
>> 
>> 
>> Am 06.12.2013 11:51, schrieb Nick Edwards:
>>> Claus,
>>> Robert's suggestion is good, we backup maildir, with 5.9TB in 2 hours
>>> to SATA , we also run a backup of that backup, so we have two extra
>>> copies (this is on top of NAS snapshots)
>>> tape is just too slow for anything serious these days.
>>> 
>>> N ik
>>> 
>>> 
>>> On 12/5/13, Claus <claus.r at bayern-mail.de> wrote:
>>>> Am 02.12.2013 20:37, schrieb Robert Schetterer:
>>>>> Am 02.12.2013 18:47, schrieb Claus:
>>>>>> Hello,
>>>>>> 
>>>>>> i have to backup (tape library) a mailsystem with about 300.000
>>>>>> Mailboxes on 2 backends. Summary of all mailboxes are 2 TByte.
>>>>>> The mailstore is mdbox.
>>>>>> Is it save to do a simple filesystem backup (full and incremental)
>>>>>> with
>>>>>> backupsoftware?
>>>>> hm i wouldnt do that with mdbox, cause it does strongly depends to
>>>>> index, but if you use dsync you could convert to maildir at backup,
>>>>> maildir should be fine to store on tape, but with 2 TB this is only
>>>>> good
>>>>> for total desaster archive, i guess it would take long time to 
>>>>> restore
>>>>> specially single mailboxes in acceptable time, from tape
>>>>> Better use cheap sata storage raids for frequent backups, and do 
>>>>> tape
>>>>> snapshots sometimes.
>>>>> 
>>>>>> What is the prefered strategy to do a backup for desaster recovery
>>>>>> (mailsystem crash) and restoring single usermailboxes?
>>>>> dsync should do fine
>>>>> 
>>>>> http://wiki2.dovecot.org/Tools/Dsync
>>>>> 
>>>>> but there is no universal answer to this, perhaps filesystem 
>>>>> snapshots
>>>>> strategy might better, it depened on your general setup , 
>>>>> filesystems ,
>>>>> mailstore etc.
>>>>> 
>>>>> wait for others report their solutions, and find the one fits best 
>>>>> at
>>>>> your place
>>>>> 
>>>>>> Regards,
>>>>>> Claus
>>>>> 
>>>>> Best Regards
>>>>> MfG Robert Schetterer
>>>>> 
>>>> thanks for your answer,
>>>> the main reason we plan to use mdbox is to reduce backuptime.
>>>> Now we are using Maildir on cyrus and the (full)-backup takes about 
>>>> 20
>>>> hours on one backend storing 150 GB of Mails in about 3 million 
>>>> files.
>>>> Because of that we have many backends, so we can parallelize the 
>>>> backup
>>>> prozess.
>>>> 
>>>> greets, Claus
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>> 
> 
> 
> 
> Best Regards
> MfG Robert Schetterer