Assistance with doveadm backup...

SH Development

21 Feb 2019 21 Feb '19

6:11 a.m.

I am having trouble locating examples of how to use doveadm backup. All the examples I see are for sync. I simply want to create a backup to a network volume of the email server's vmail folders. The goal here is to have a reasonably current backup should the main drive on the email server go south.

We currently authenticate our users from a mysql database. User’s mailboxes are stored as domainname/username/Maildir

I assume what I will wind up on the network volume is a duplicate directory structure as the vmail folder on the email server?

Can someone help get me started here?

Jeff

Show replies by date

＠lbutlr

21 Feb 21 Feb

6:45 a.m.

On Feb 20, 2019, at 21:11, SH Development via dovecot dovecot@dovecot.org wrote:

...

Can someone help get me started here?

Does this help?

https://unix.stackexchange.com/questions/489331/what-is-the-correct-way-to-u...

-- This is my signature. There are many like it, but this one is mine.

SH Development

7:04 a.m.

Not really. The suggestions in the post seemed to be:

Tar the whole directory, which I have tried in the past, but a 130G directory takes over 18 hours to tar.
It was also suggested to rsync the directory, but the question was brought up, and not answered, about whether it was advisable to copy live mail, thus the need for doveadm sync/backup.

Based on what I’ve read about doveadm backup, this is what I need, I am just having a hard time formulating the syntax for the command.

Jeff

...

On Feb 20, 2019, at 10:45 PM, @lbutlr via dovecot dovecot@dovecot.org wrote:

On Feb 20, 2019, at 21:11, SH Development via dovecot mailto:dovecot@dovecot.org> wrote:

...
Can someone help get me started here?

Does this help?

https://unix.stackexchange.com/questions/489331/what-is-the-correct-way-to-u... https://unix.stackexchange.com/questions/489331/what-is-the-correct-way-to-u...

-- This is my signature. There are many like it, but this one is mine.

Sami Ketola

9:51 a.m.

...

On 21 Feb 2019, at 7.04, SH Development via dovecot dovecot@dovecot.org wrote:

Not really. The suggestions in the post seemed to be:

Tar the whole directory, which I have tried in the past, but a 130G directory takes over 18 hours to tar.

It was also suggested to rsync the directory, but the question was brought up, and not answered, about whether it was advisable to copy live mail, thus the need for doveadm sync/backup.

Based on what I’ve read about doveadm backup, this is what I need, I am just having a hard time formulating the syntax for the command.

root@ketola ~]# mkdir -p /backup/sami root@ketola ~]# chown vmail:vmail /backup/sami root@ketola ~]# doveadm backup -u sami Maildir:/backup/sami <snip> root@ketola ~]# du -hs /backup/sami 211M /backup/sami

does this help?

Sami

Joseph Tam

11:50 p.m.

On Wed, 20 Feb 2019, SH Development wrote:

...

To: Joan Moreau via dovecot dovecot@dovecot.org

Jean did take over the list for a while when developing his FTS backend, so you can be forgiven that he actually runs this list.

...

It was also suggested to rsync the directory, but the question was brought up, and not answered, about whether it was advisable to copy live mail, thus the need for doveadm sync/backup.

You're right that consistency is a problem if you rsync a large amount of changing data. (If I remember correctly, mdbox is especially sensitive to content/index mismatches.)

One way this can be done is via filesystem magic i.e. filesystems with copy-on-write that allow you to take a moment in time snapshot of your entire filesystem e.g. LVM, ZFS and others. You can then rsync the snapshot, but some of these filesystem also support methods to export snapshots to a remote filesystem.

Joseph Tam jtam.home@gmail.com

Odhiambo Washington

22 Feb 22 Feb

10:46 a.m.

On Thu, 21 Feb 2019 at 07:11, SH Development via dovecot < dovecot@dovecot.org> wrote:

...

I am having trouble locating examples of how to use doveadm backup. All the examples I see are for sync. I simply want to create a backup to a network volume of the email server's vmail folders. The goal here is to have a reasonably current backup should the main drive on the email server go south.

We currently authenticate our users from a mysql database. User’s mailboxes are stored as domainname/username/Maildir

I assume what I will wind up on the network volume is a duplicate directory structure as the vmail folder on the email server?

Can someone help get me started here?

Jeff

In my previous life as a SysAdmin, I solved this kind of problem easily using the MTA. Exim is my MTA of choice and has a facility to do concurrent delivery to two storage locations using shadow_transport. I am guessing your MTA is Postfix (or maybe Sendmail). Try and see if they have such a capability or switch to Exim.

-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)

SH Development

10 Mar 10 Mar

12:40 a.m.

Okay, apparently I’m just a complete idiot. Why doesn’t this work?

doveadm -Dv backup -u user Maildir:/mnt/maelstrombackups/vmailbackup

With user being the actual user name.

No backup is created, it just shows me the usage text.

Jeff

...

On Feb 20, 2019, at 10:11 PM, SH Development listaccount@starionline.com wrote:

I am having trouble locating examples of how to use doveadm backup. All the examples I see are for sync. I simply want to create a backup to a network volume of the email server's vmail folders. The goal here is to have a reasonably current backup should the main drive on the email server go south.

We currently authenticate our users from a mysql database. User’s mailboxes are stored as domainname/username/Maildir

I assume what I will wind up on the network volume is a duplicate directory structure as the vmail folder on the email server?

Can someone help get me started here?

Jeff

Aki Tuomi

8:07 a.m.

SH Development

10:45 a.m.

V 2.0.9

Jeff

...

On Mar 10, 2019, at 12:07 AM, Aki Tuomi via dovecot mailto:dovecot@dovecot.org> wrote:

What version are you using?

Aki

...
On 10 March 2019 00:40 SH Development via dovecot < dovecot@dovecot.org mailto:dovecot@dovecot.org> wrote:

Okay, apparently I’m just a complete idiot. Why doesn’t this work?

doveadm -Dv backup -u user Maildir:/mnt/maelstrombackups/vmailbackup

With user being the actual user name.

No backup is created, it just shows me the usage text.

Jeff

...
On Feb 20, 2019, at 10:11 PM, SH Development < listaccount@starionline.com mailto:listaccount@starionline.com> wrote:

I am having trouble locating examples of how to use doveadm backup. All the examples I see are for sync. I simply want to create a backup to a network volume of the email server's vmail folders. The goal here is to have a reasonably current backup should the main drive on the email server go south.

We currently authenticate our users from a mysql database. User’s mailboxes are stored as domainname/username/Maildir

I assume what I will wind up on the network volume is a duplicate directory structure as the vmail folder on the email server?

Can someone help get me started here?

Jeff

Aki Tuomi

Aki Tuomi

11:12 a.m.

SH Development

7:09 p.m.

Okay then. Is there a document on how to do that in place? That was one of the reasons I was trying to do a backup first.

Jeff

...

On Mar 10, 2019, at 4:12 AM, Aki Tuomi via dovecot mailto:dovecot@dovecot.org> wrote:

You should upgrade 2.2 first.

Aki

...
On 10 March 2019 10:45 SH Development via dovecot mailto:dovecot@dovecot.org> wrote:

V 2.0.9

Jeff

...
On Mar 10, 2019, at 12:07 AM, Aki Tuomi via dovecot < dovecot@dovecot.org mailto:dovecot@dovecot.org> wrote: What version are you using?

Aki

...
On 10 March 2019 00:40 SH Development via dovecot < dovecot@dovecot.org mailto:dovecot@dovecot.org> wrote:

Okay, apparently I’m just a complete idiot. Why doesn’t this work?

doveadm -Dv backup -u user Maildir:/mnt/maelstrombackups/vmailbackup

With user being the actual user name.

No backup is created, it just shows me the usage text.

Jeff

...
On Feb 20, 2019, at 10:11 PM, SH Development < listaccount@starionline.com mailto:listaccount@starionline.com> wrote:

I am having trouble locating examples of how to use doveadm backup. All the examples I see are for sync. I simply want to create a backup to a network volume of the email server's vmail folders. The goal here is to have a reasonably current backup should the main drive on the email server go south.

We currently authenticate our users from a mysql database. User’s mailboxes are stored as domainname/username/Maildir

I assume what I will wind up on the network volume is a duplicate directory structure as the vmail folder on the email server?

Can someone help get me started here?

Jeff

Aki Tuomi

Aki Tuomi

SH Development

11 Mar 11 Mar

3:39 a.m.

Ok, so for lack of any other information, am I to conclude that the doveadm backup command is broken in 2.0.9? I have been through the wiki for it back and forth and cannot make it work.

Yes, I need to upgrade, there is no question, but I won’t do that without a maildir backup. I would prefer to use the built-in dovecot mechanisms for that than something else.

Jeff

...

On Mar 10, 2019, at 12:09 PM, SH Development via dovecot mailto:dovecot@dovecot.org> wrote:

Okay then. Is there a document on how to do that in place? That was one of the reasons I was trying to do a backup first.

Jeff

...
On Mar 10, 2019, at 4:12 AM, Aki Tuomi via dovecot mailto:dovecot@dovecot.org> wrote:

You should upgrade 2.2 first.

Aki

...
On 10 March 2019 10:45 SH Development via dovecot mailto:dovecot@dovecot.org> wrote:

V 2.0.9

Jeff

...
On Mar 10, 2019, at 12:07 AM, Aki Tuomi via dovecot < dovecot@dovecot.org mailto:dovecot@dovecot.org> wrote: What version are you using?

Aki

...
On 10 March 2019 00:40 SH Development via dovecot < dovecot@dovecot.org mailto:dovecot@dovecot.org> wrote:

Okay, apparently I’m just a complete idiot. Why doesn’t this work?

doveadm -Dv backup -u user Maildir:/mnt/maelstrombackups/vmailbackup

With user being the actual user name.

No backup is created, it just shows me the usage text.

Jeff

...
On Feb 20, 2019, at 10:11 PM, SH Development < listaccount@starionline.com mailto:listaccount@starionline.com> wrote:

I am having trouble locating examples of how to use doveadm backup. All the examples I see are for sync. I simply want to create a backup to a network volume of the email server's vmail folders. The goal here is to have a reasonably current backup should the main drive on the email server go south.

We currently authenticate our users from a mysql database. User’s mailboxes are stored as domainname/username/Maildir

I assume what I will wind up on the network volume is a duplicate directory structure as the vmail folder on the email server?

Can someone help get me started here?

Jeff

Aki Tuomi

Aki Tuomi

Aki Tuomi

7:07 a.m.

Natu

11:11 a.m.

Well for the purpose of upgrading your server, you could do the following:

Rsync your maildir directories to a backup directory
repeat rsync again so most of your changed files have been synced
shut down your dovecot and smtp servers
rsync one more time - should be relatively fast
restart your dovecot and smtp servers

Otherwise, just schedule downtime and use whatever backup method you prefer. You will probably want to backup and then leave dovecot and smtp server down until you've completed the dovecot upgrade anyway, so you may as well schedule downtime for the entire process and do the dovecot upgrade between steps 4 and 5 above.

Natu

On 3/10/19 10:07 PM, Aki Tuomi via dovecot wrote:

...

I'd say the correct term is not broken but missing. backup was implemented in later version.

With maildir keeping a copy of the content is sufficient backup.

Aki

...
On 11 March 2019 03:39 SH Development via dovecot dovecot@dovecot.org wrote:

Ok, so for lack of any other information, am I to conclude that the doveadm backup command is broken in 2.0.9? I have been through the wiki for it back and forth and cannot make it work.

Yes, I need to upgrade, there is no question, but I won’t do that without a maildir backup. I would prefer to use the built-in dovecot mechanisms for that than something else.

Jeff

...
On Mar 10, 2019, at 12:09 PM, SH Development via dovecot < dovecot@dovecot.org mailto:dovecot@dovecot.org> wrote: Okay then. Is there a document on how to do that in place? That was one of the reasons I was trying to do a backup first.

Jeff

...
On Mar 10, 2019, at 4:12 AM, Aki Tuomi via dovecot < dovecot@dovecot.org mailto:dovecot@dovecot.org> wrote: You should upgrade 2.2 first.

Aki

...
On 10 March 2019 10:45 SH Development via dovecot < dovecot@dovecot.org mailto:dovecot@dovecot.org> wrote:

V 2.0.9

Jeff

...
On Mar 10, 2019, at 12:07 AM, Aki Tuomi via dovecot < dovecot@dovecot.org mailto:dovecot@dovecot.org> wrote: What version are you using?

Aki > On 10 March 2019 00:40 SH Development via dovecot < > dovecot@dovecot.org mailto:dovecot@dovecot.org> wrote: > > > Okay, apparently I’m just a complete idiot. Why doesn’t this work? > > doveadm -Dv backup -u user > Maildir:/mnt/maelstrombackups/vmailbackup > > With user being the actual user name. > > No backup is created, it just shows me the usage text. > > Jeff > >> On Feb 20, 2019, at 10:11 PM, SH Development < >> listaccount@starionline.com >> mailto:listaccount@starionline.com> wrote: >> >> I am having trouble locating examples of how to use doveadm >> backup. All the examples I see are for sync. I simply want to >> create a backup to a network volume of the email server's vmail >> folders. The goal here is to have a reasonably current backup >> should the main drive on the email server go south. >> >> We currently authenticate our users from a mysql database. >> User’s mailboxes are stored as domainname/username/Maildir >> >> I assume what I will wind up on the network volume is a >> duplicate directory structure as the vmail folder on the email >> server? >> >> Can someone help get me started here? >> >> Jeff

Aki Tuomi

Aki Tuomi

Aki Tuomi

2085

Age (days ago)

2103

Last active (days ago)

List overview

13 comments

7 participants

participants (7)

＠lbutlr
Aki Tuomi
Joseph Tam
Natu
Odhiambo Washington
Sami Ketola
SH Development