[Dovecot] dsync migration questions

[Michael Smith (DF)]

We are currently in the process of replacing one of our customer mail systems with a dovecot solution.  However, one of the sticking points right now is how to get the old mail to the new system.

On the dovecot side, we are using mdbox storage.  On the old system we are using qpopper/mbox mailboxes in the following setup.  All user inboxes are in /mnt/mail/mail_spool.  Only select, privileged, users have mail folders in /mnt/mail/mail_spool/FOLDERS/user/.  I know, insane setup, but it's almost 18 years legacy now.  Like this...
/mnt/mail/mail_spool/user1.mbox
/mnt/mail/mail_spool/user2.mbox
/mnt/mail/mail_spool/user3.mbox
/mnt/mail/mail_spool/user4.mbox
/mnt/mail/mail_spool/FOLDERS/user2/folder1.mbox
/mnt/mail/mail_spool/FOLDERS/user2/folder2.mbox

Also, the UIDs between the old system and the new dovecot solution are different.  User1 may have uid 2134 on the old system, but have uid 193847 on the new system.

We've tried the following.  Note: User Anakin has a single message already in the new inbox.

mkdir -p /mnt/mail/mail_spool/.migrate/
ln /mnt/mail/mail_spool/anakin /mnt/mail/mail_spool/.migrate/inbox
chmod 644 /mnt/mail/mail_spool/.migrate/inbox
dsync -D -v -R -u anakin at domain.com backup mbox:/mnt/mail/mail_spool/.migrate/
...
chmod 600 /mnt/mail/mail_spool/.migrate/inbox
/bin/rm -rf /mnt/mail/mail_spool/.migrate

dsync(root): Debug: Loading modules from directory: /usr/lib64/dovecot
dsync(root): Debug: Module loaded: /usr/lib64/dovecot/lib10_quota_plugin.so
dsync(root): Debug: Module loaded: /usr/lib64/dovecot/lib20_zlib_plugin.so
dsync(root): Debug: auth input: anakin uid=1000001228 gid=1000001228 home=/var/spool/mail/CC/1000001228 quota_rule=*:storage=0:messages=0
dsync(root): Debug: Added userdb setting: plugin/quota_rule=*:storage=0:messages=0
dsync(root): Debug: Effective uid=1000001228, gid=1000001228, home=/var/spool/mail/CC/1000001228
dsync(root): Debug: Quota root: name=User quota backend=dict args=:file:/var/spool/mail/CC/1000001228/dovecot.quota
dsync(root): Debug: Quota rule: root=User quota mailbox=* bytes=0 messages=0
dsync(root): Debug: dict quota: user=anakin, uri=file:/var/spool/mail/CC/1000001228/dovecot.quota, noenforcing=0
dsync(root): Debug: fs: root=/var/spool/mail/CC/1000001228/mail, index=/var/spool/mail/CC/1000001228/index, control=, inbox=
dsync(root): Debug: Namespace : Using permissions from /var/spool/mail/CC/1000001228/mail: mode=0700 gid=-1
dsync(root): Debug: Effective uid=1000001228, gid=1000001228, home=/var/spool/mail/CC/1000001228
dsync(root): Debug: Quota root: name=User quota backend=dict args=:file:/var/spool/mail/CC/1000001228/dovecot.quota
dsync(root): Debug: Quota rule: root=User quota mailbox=* bytes=0 messages=0
dsync(root): Debug: dict quota: user=anakin, uri=file:/var/spool/mail/CC/1000001228/dovecot.quota, noenforcing=0
dsync(root): Debug: mbox: INBOX defaulted to /mnt/test/mail_spool/.migrate/inbox
dsync(root): Debug: fs: root=/mnt/test/mail_spool/.migrate, index=, control=, inbox=/mnt/test/mail_spool/.migrate/inbox
dsync(anakin): Debug: Namespace : Using permissions from /mnt/test/mail_spool/.migrate: mode=0777 gid=-1
dsync(anakin): Error: Can't delete mailbox INBOX: INBOX can't be deleted.
dsync(anakin): Debug: Namespace : Permission lookup failed from /var/spool/mail/CC/1000001228/mail/mailboxes/INBOX_b6845831821c42517c490000a72406a4
dsync(anakin): Debug: Namespace : Using permissions from /var/spool/mail/CC/1000001228/mail: mode=0700 gid=-1
dsync(anakin): Debug: Namespace : Permission lookup failed from /var/spool/mail/CC/1000001228/mail/mailboxes/INBOX_b6845831821c42517c490000a72406a4
dsync(anakin): Debug: Namespace : Using permissions from /var/spool/mail/CC/1000001228/mail: mode=0700 gid=-1
dsync(anakin): Info: INBOX: only in dest
dsync(anakin): Info: INBOX: only in source
dsync(anakin): Error: Trying to open a non-listed mailbox with guid=814b6332541c425169490000a72406a4
dsync(anakin): Error: msg iteration failed: Couldn't open mailbox 814b6332541c425169490000a72406a4
dsync(anakin): Error: Trying to open a non-listed mailbox with guid=814b6332541c425169490000a72406a4


Nothing was migrated.  In some other combinations, trying to reproduce them again, it does migrate the old mail, but first destroys the existing mail in the new mailbox.

Our goal is to cut mail delivery over to the new dovecot system (which will result in existing emails before migration), and then migrate the old mail *while* maintaining message UIDs.  This is the easiest way to ensure no "lost" email with the migration.

What are we doing wrong, and how do we achieve our goals?

Thanks in advance.

--
Michael Smith