[Dovecot] Dovecot dsync mail replication issues

[Reuben Farrelly]

Hi,

I'm trying to set up some dsync based replication between two hosts on 
my network.  The current topology is a single server running Postfix 
with a single dovecot installation with a Maildir per user (only 4 users 
including myself).  No NFS, just local system users on ext4.  I am only 
using system users, ie no virtual users.  I am using dovecot deliver to 
deliver mails into the Maildir's.  This - being a very standard Maildir 
installation -  all works just fine.

I'm running dovecot-2.1.5 (release versions) x86_64 with Gentoo on both 
hosts and using key based ssh to transport the data using the root login.

What I would like to do is extend the design so as to replicate the 
Maildirs across a second machine and in the future be able to connect 
via IMAP into either machine to sync mail.  Two way dsync would be 
rather cool because I could then deliver via SMTP to either system and 
have changes automagically propagate - but initially just simple 
replication would be a good start.

I was hoping that dsync would allow me to do this, but I've run into 
quite a number of problems while getting this to work.

http://dovecot.org/list/dovecot/2012-March/064243.html
...was very useful and I've based my config on that.

Initially I've tried to sync up the user Maildirs, and this has more or 
less worked:

doveadm sync -u lyn remote:root at dustbowl.reub.net

This succeeds without error on the initial sync.

However if I try to run the re-sync again (such as a use case of if the 
sync of another Maildir takes 4 hours so I want to resync up the earlier 
ones again) I end up with a mysteriously named INBOX folder in both the 
source and destination Maildirs:

drwx------  5 lyn lyn  4096 Apr 30 19:32 
.INBOX_7a86a62d465a974fb92f00003b258734

It has the basic structure of a Maildir but is empty in terms of mails:

drwx------  2 lyn lyn 4096 Apr 30 19:32 cur
-rw-------  1 lyn lyn  220 Apr 30 19:32 dovecot.index.log
-rw-------  1 lyn lyn   51 Apr 30 19:32 dovecot-uidlist
-rw-------  1 lyn lyn    0 Apr 30 19:32 maildirfolder
drwx------  2 lyn lyn 4096 Apr 30 19:32 new
drwx------  2 lyn lyn 4096 Apr 30 19:32 tmp

First question:  why is this random named directory being created in the 
origin Maildir?  Shouldn't the replication be more or less read-only in 
the origin Maildir?

Second question:  If I re-attempt a doveadm sync a second time I get 
this error:

tornado Maildir # doveadm sync -u lyn remote:root at dustbowl.reub.net
dsync-local(lyn): Error: Can't rename mailbox 
INBOX_7a86a62d465a974fb92f00003b258734 to INBOX: Target mailbox already 
exists
dsync-local(lyn): Error: Can't rename mailbox INBOX to 
INBOX_eb15f30ea563be4b703200002bd68bb1: Renaming INBOX isn't supported.
tornado Maildir #

It's not clear if the second attempt has failed or succeeded, and it's a 
bit odd that it errors out on a directory that the dovecot sync process 
itself has created.

Third question:  Upon starting Dovecot lots of ugliness is logged in the 
mail log:

Apr 30 19:44:59 tornado dovecot: master: Dovecot v2.1.5 starting up 
(core dumps disabled)
Apr 30 19:44:59 tornado dovecot: doveadm(mozsync): Error: user mozsync: 
Initialization failed: Namespace '': 
mkdir(/var/www/xxx/server-full/Maildir) failed: Permission denied 
(euid=1016(mozsync) egid=1016(<unknown>) missing +w perm: 
/var/www/xxx/server-full, dir owned by 0:0 mode=0755)
Apr 30 19:44:59 tornado dovecot: doveadm(mozsync): Error: sync: User 
init failed
Apr 30 19:44:59 tornado dovecot: doveadm(mozsync): Warning: I/O leak: 
0x414190 (line 102, fd 16)
Apr 30 19:44:59 tornado dovecot: dsync-local(cisco): Error: remote: 
doveadm(cisco): Fatal: User doesn't exist
Apr 30 19:44:59 tornado dovecot: dsync-local(cisco): Error: read() from 
worker server failed: EOF

Users mozsync and cisco are not valid mail users and it's not 
appropriate that Dovecot tries to create a Maildir for either of them. 
The users are system unprivileged users only, and do not ever send or 
receive mail.

And - I/O leak? ;)

Also, user cisco is local to one box only, and does not exist (nor does 
it need to) on the remote host.  So any complaints about this user are 
invalid and dovecot needs to ignore replication for this user anyway.


Fourth question, upon starting dovecot again, mail.log then spews 
several hundred of these messages:

Apr 30 19:45:06 tornado dovecot: dsync-local(reuben): Error: msg-get 
failed: box=Trash uid=114863 
guid=1335382569.M98089P29952.tornado,S=6479,W=6625

Before aborting entirely with:

Apr 30 19:45:09 tornado dovecot: imap: Server shutting down. in=328 out=2042

It seems to me that a a few of those problems logged could be solved by 
being able to specify which system users to synchronise, rather than 
Dovecot making a blind assumption that all users actually have valid 
Maildirs that need to be created, and all need to be sync'd between two 
hosts.

Subsequent delivery based sync'ing fails silently (pending more 
investigation) but I'd like to try and fix the above listed problems 
first before worrying about the later stages of the sync not working.

Has anyone else run into these problems or found better ways of making 
this type of setup all work?

Thanks,
Reuben