Doveadm backup...

Sun Mar 27 20:09:45 UTC 2022

>>>>> "Stephane" == Stephane Magnier <steph.mag220 at netcourrier.com> writes:

Sorry, I deleted your most recent email post before I could reply.
But why don't you just do 'imapsync' instead from your production
dovecot box to some other backup system?  Otherwise I'd probably work
to setup dovecot's own replication but only have it go one way.

For example, I've got a VPS out in the cloud for my email, and I
should probably back it up to my home system using replication, but it
would be strictly primary->secondary.  I wouldn't be trying to run two
primaries replicating between each other.

Imapsync would be an improvement over rsync because it works within
dovecot, so you'd get a more consistent view, but maybe not quite as
upto date.  But how important is your email if you worry about losing
20 minutes worth of it?  If it's that critical, then you should be
investing in a more robust setup.

John

Stephane> I've seen in a previous post, that the fact to do an RSYNC
Stephane> might break the  index.. So, I've heard that this is not
Stephane> recommended. that's the reason why I decided to find a way
Stephane> to do a "clean" backup and be able to come back online if
Stephane> needed.

Stephane> So, do you use an RSYNC and in case of restoring the mailbox, do you do a  simple

Stephane>  doveadm index -u USERx INBOX

Stephane> And that's it ? works fine ?

Stephane> 2) I will try the Backup , or Sync.. locally.. Effectively, I don't know where the problem comes from..I have effectively an NFS Mount
Stephane>  for the mailboxes and a VM for the Email server and that could be another another point of failure :-(

Stephane> Thanks for sharing.. I am a bit in a rush... I realized that my backup maybe not correct.. and I prefer not to discover it while running into trouble..

Stephane> On 2/28/22 19:03, Ben Burk wrote:

Stephane>     I'm not sure what you are attempting to do here. It looks like you just ran a doveadm backup
Stephane>     and the process completed for userx with a warning that the remote system (your nfs mount)
Stephane>     lost a particular mailbox (possibly your indexes changed or a mailbox was deleted). From the
Stephane>     logs you pasted it appears the process completed normally.

Stephane>     I personally do not use dovecot's backup or replication processes. If I needed to I would use
Stephane>     its replication process to sync active data between multiple systems, but I have no need for
Stephane>     this as of yet. Personally I chose to create offsite backups using rsync a long time ago, as
Stephane>     rebuilding a mailbox (reindexing) is very simple.

Stephane>     Try running            doveadm mailbox status -u userx guid '*'    as the mailbox
Stephane>     administrator and see if you can find that GUID, 7e05c335174bf1608f0a02004eac7fb4. Also, see
Stephane>     if the backup you've written to nfs has the GUID.

Stephane>     On 2/27/22 23:33, Stephane Magnier wrote:

Stephane>         Well no ..I thought that dsync was for synchro " realtime for 2 different places ?

Stephane>         Having no 2 machines in parallel ( Just a single machine ) , I thought that a backup at
Stephane>         regular interval  would be enough ?

Stephane>         So, a simple backup should be done by dsync finally ?

Stephane>         Do you recommend finally NOT to do a backup ( Doveadm backup ) but a replication process ?
Stephane>         ( https://wiki.dovecot.org/Replication  ) ?

Stephane>         On 2/28/22 06:24, Ben Burk wrote:

Stephane>             Did you try running dsync?

Stephane>             On 2/27/22 23:15, Stephane Magnier wrote:

Stephane>                 HI,

Stephane>                 Any idea ? Any clue ?

Stephane>                 On 2/25/22 21:50, Stephane Magnier wrote:

Stephane>                     Hi

Stephane>                     I've recently tried to use the Dovecadm backup to backup the emails..  with
Stephane>                     the following syntax

Stephane>                     doveadm -Dv backup -u userx maildir:/mnt/nfs-backup/userx

Stephane>                     Sounds to be OK with few emails... Some of them got a lot of emails and one f
Stephane>                     them got an error and stop !

Stephane>                         dsync(userx): Debug: brain S: Import Trash: Import change type=expunge
Stephane>                         GUID=1725fa475d774ee19cb98dfb6737b4f1 UID=24891 hdr_hash= result=GUIDs
Stephane>                         match
Stephane>                         dsync(userx): Debug: brain S: Import Trash: Import change type=expunge
Stephane>                         GUID=916ed110b4b1522868be6194f1ae36ff UID=24892 hdr_hash= result=GUIDs
Stephane>                         match
Stephane>                         dsync(userx): Debug: brain S: Import Trash: Import change type=expunge
Stephane>                         GUID=eb8d75c530a7b02fc26b494d9006c91b UID=24893 hdr_hash= result=GUIDs
Stephane>                         match
Stephane>                         dsync(userx): Debug: brain S: Import Trash: Import change type=expunge
Stephane>                         GUID=aee9155875c34861fd6500f1f2f51a26 UID=24894 hdr_hash= result=GUIDs
Stephane>                         match
Stephane>                         dsync(userx): Debug: brain S: Import Trash: Last common UID=24894. Delayed
Stephane>                         expunges=
Stephane>                         dsync(userx): Debug: brain S: Import Trash: Saved UIDs:
Stephane>                         dsync(userx): Debug: brain S: Import Trash: Finish update: min_next_uid=
Stephane>                         24895 min_first_recent_uid=24895 min_highest_modseq=35344
Stephane>                         min_highest_pvt_modseq=0
Stephane>                         dsync(userx): Debug: /mnt/nfs-backup/userx/.Trash/dovecot.index.cache:
Stephane>                         Compressed, file_seq changed 1645803588 -> 1645803589, size=32, max_uid=
Stephane>                         24894
Stephane>                         dsync(userx): Warning: Mailbox changes caused a desync. You may want to
Stephane>                         run dsync again: Remote lost mailbox GUID 7e05c335174bf1608f0a02004eac7fb4
Stephane>                         (maybe it was just deleted?)
Stephane>                         dsync(userx): Debug: auth-master: conn unix:/run/dovecot/auth-userdb:
Stephane>                         Disconnected: Connection closed (fd=10)

Stephane>                     I empty the trash... exactly the same problem...

Stephane>                     Any idea why this ??

Stephane>                     Thanks and regards,