Hello,
Upfront note: I know very little about dovecot and IMAP lingo / terms of art, I'll try to be clear and complete but I may be missing important bits.
I have a VPS backup with mailboxes in the maildir dovecot format:
$ ls some/folder/backup/mail/domain/\* cur dovecot-uidlist dovecot-uidvalidity.5c0637b7 dovecot.index.cache dovecot.mailbox.log subscriptions dovecot-keywords dovecot-uidvalidity dovecot.index dovecot.index.log new tmp
Separately and independently, I also have an existing IMAP account I do not personally manage. By "account" I mean that if I connect to the IMAP server, LOGIN, then LIST I see a pretty standard set of folders:
(\HasNoChildren) "/" INBOX (\HasNoChildren \Archive) "/" Archive (\HasNoChildren \Drafts) "/" Drafts (\HasNoChildren \XNotes) "/" Notes (\HasNoChildren \Sent) "/" Sent (\HasNoChildren \Junk) "/" Spam (\HasNoChildren \Trash) "/" Trash
I would like to copy / upload / … the content of the backup mailboxes to the IMAP account, ideally without losing the metadata.
I asked about this on IRC and user "cmouse" provided me with information
about the imapc storage (plugin?), a sample "doveadmin import" command,
and a sample dovecot configuration which I expanded to this (for simplicity,
the original sample command used a lot of -c and I figured I could just
shove them into the configuration file for readability):
$ dovecot -n # 2.3.16 (): /opt/local/etc/dovecot/dovecot.conf # OS: Darwin 15.6.0 x86_64 auth_verbose = yes imapc_host = <server> imapc_password = # hidden, use -P to show it imapc_port = 993 imapc_ssl = imaps imapc_user = <user> mail_location = imapc: # did not setup a "vmail" user or group as I just want to move the contents mail_uid = <current_user> mail_gid = staff namespace { inbox = yes location = prefix = separator = / } ssl = no verbose_proctitle = yes verbose_ssl = yes
That did not work as I guess "import" wants to create all the mailboxes, and these already exist:
$ doveadm -Dv import maildir:some/folder/backup/mail/domain/\* '.' all […] Debug: Mailbox Archive: Mailbox opened because: import Debug: Mailbox spam: Mailbox opened because: import Debug: Mailbox Junk: Mailbox opened because: import Debug: Mailbox Trash: Mailbox opened because: import Debug: Mailbox Sent: Mailbox opened because: import Debug: Mailbox Drafts: Mailbox opened because: import Debug: Mailbox INBOX.Spam: Mailbox opened because: import Debug: Mailbox INBOX.Junk: Mailbox opened because: import Debug: Mailbox INBOX.Drafts: Mailbox opened because: import Debug: Mailbox INBOX: Mailbox opened because: import Debug: Namespace : Using permissions from : mode=0700 gid=default Error: Couldn't create mailbox ./INBOX: Mailbox already exists
I tried messing around with the options but didn't achieve anything, the
only change which did anything visible was to replace '.' with '.INBOX'
which resulted in import creating .IMPORT/IMPORT and exiting with:
Error: Couldn't create mailbox INBOX/INBOX: Invalid mailbox name
which… fair enough.
Looking around I found doveadm sync which seemed promising, so far I
arrived not far:
$ doveadm -Dv sync -f maildir:some/folder/backup/mail/domain/\*
this ultimately fails as well:
brain S: Local mailbox tree: INBOX guid=63ace42cc352935925190000691e7660 uid_validity=1315910895 uid_next=71 subs=no last_change=0 last_subs=0 brain M: Local mailbox tree: Trash guid=7f5af7ba291b2df1a11d573bdb55d7e9 uid_validity=1632758505 uid_next=6 subs=yes last_change=0 last_subs=0 brain S: Local mailbox tree: INBOX/Drafts guid=ebb8a22812fe675b34390000691e7660 uid_validity=1533541906 uid_next=2 subs=yes last_change=0 last_subs=0 brain M: Local mailbox tree: Spam guid=59c97b78ad43f83b1f91e47936e7e7d1 uid_validity=1632758508 uid_next=20 subs=yes last_change=0 last_subs=0 brain S: Local mailbox tree: INBOX/Junk guid=62ace42cc352935925190000691e7660 uid_validity=1490450196 uid_next=2 subs=yes last_change=0 last_subs=0 brain M: Local mailbox tree: Sent guid=bfb2e03fdce327671e82bf173b1ccb8b uid_validity=1632758506 uid_next=1 subs=yes last_change=0 last_subs=0 brain S: Local mailbox tree: INBOX/Spam guid=64ace42cc352935925190000691e7660 uid_validity=1315911183 uid_next=1 subs=yes last_change=0 last_subs=0 brain M: Local mailbox tree: Drafts guid=e0187b65e763143666d22094cedfe6a4 uid_validity=1632758507 uid_next=1 subs=yes last_change=0 last_subs=0 brain S: Local mailbox tree: Drafts guid=2fda3e14a611e860c43e2700d32ae34d uid_validity=1625821607 uid_next=1 subs=no last_change=0 last_subs=0 brain M: Local mailbox tree: Archive guid=51a58e1d1d5c491aac71488647b20224 uid_validity=1632758509 uid_next=1 subs=yes last_change=0 last_subs=0 brain S: Local mailbox tree: Sent guid=30da3e14a611e860c43e2700d32ae34d uid_validity=1625821608 uid_next=1 subs=no last_change=0 last_subs=0 brain M: Local mailbox tree: INBOX guid=c92f64f79f0d1ed01e6d5b314f04886c uid_validity=1632758504 uid_next=13 subs=yes last_change=0 last_subs=0 brain S: Local mailbox tree: Trash guid=31da3e14a611e860c43e2700d32ae34d uid_validity=1625821609 uid_next=1 subs=no last_change=0 last_subs=0 brain S: Local mailbox tree: Junk guid=32da3e14a611e860c43e2700d32ae34d uid_validity=1625821610 uid_next=1 subs=no last_change=0 last_subs=0 brain S: Local mailbox tree: spam guid=33da3e14a611e860c43e2700d32ae34d uid_validity=1625821611 uid_next=1 subs=no last_change=0 last_subs=0 brain M: Remote mailbox tree: INBOX guid=63ace42cc352935925190000691e7660 uid_validity=1315910895 uid_next=71 subs=no last_change=0 last_subs=0 brain M: Remote mailbox tree: INBOX/Drafts guid=ebb8a22812fe675b34390000691e7660 uid_validity=1533541906 uid_next=2 subs=yes last_change=0 last_subs=0 brain M: Remote mailbox tree: INBOX/Junk guid=62ace42cc352935925190000691e7660 uid_validity=1490450196 uid_next=2 subs=yes last_change=0 last_subs=0 brain M: Remote mailbox tree: INBOX/Spam guid=64ace42cc352935925190000691e7660 uid_validity=1315911183 uid_next=1 subs=yes last_change=0 last_subs=0 brain M: Remote mailbox tree: Drafts guid=2fda3e14a611e860c43e2700d32ae34d uid_validity=1625821607 uid_next=1 subs=no last_change=0 last_subs=0 brain M: Remote mailbox tree: Sent guid=30da3e14a611e860c43e2700d32ae34d uid_validity=1625821608 uid_next=1 subs=no last_change=0 last_subs=0 brain M: Remote mailbox tree: Trash guid=31da3e14a611e860c43e2700d32ae34d uid_validity=1625821609 uid_next=1 subs=no last_change=0 last_subs=0 brain M: Remote mailbox tree: Junk guid=32da3e14a611e860c43e2700d32ae34d uid_validity=1625821610 uid_next=1 subs=no last_change=0 last_subs=0 brain M: Remote mailbox tree: spam guid=33da3e14a611e860c43e2700d32ae34d uid_validity=1625821611 uid_next=1 subs=no last_change=0 last_subs=0 brain S: Local mailbox tree: Archive guid=4d10f3271f03f7603b232100d32ae34d uid_validity=1533541907 uid_next=1 subs=yes last_change=0 last_subs=0 brain M: Remote mailbox tree: Archive guid=4d10f3271f03f7603b232100d32ae34d uid_validity=1533541907 uid_next=1 subs=yes last_change=0 last_subs=0 brain M: Mailbox Archive: local=51a58e1d1d5c491aac71488647b20224/0/1, remote=4d10f3271f03f7603b232100d32ae34d/0/1: GUIDs conflict - will be merged later brain M: Mailbox Drafts: local=e0187b65e763143666d22094cedfe6a4/0/1, remote=2fda3e14a611e860c43e2700d32ae34d/0/1: GUIDs conflict - will be merged later brain M: Mailbox INBOX: local=c92f64f79f0d1ed01e6d5b314f04886c/0/1, remote=63ace42cc352935925190000691e7660/0/1: GUIDs conflict - will be merged later brain M: Mailbox INBOX/Drafts: local=00000000000000000000000000000000/0/0, remote=ebb8a22812fe675b34390000691e7660/0/1: mailbox not selectable yet brain M: Mailbox INBOX/Junk: local=00000000000000000000000000000000/0/0, remote=62ace42cc352935925190000691e7660/0/1: mailbox not selectable yet brain M: Mailbox INBOX/Spam: local=00000000000000000000000000000000/0/0, remote=64ace42cc352935925190000691e7660/0/1: mailbox not selectable yet brain M: Mailbox Junk: local=00000000000000000000000000000000/0/0, remote=32da3e14a611e860c43e2700d32ae34d/0/1: mailbox not selectable yet brain M: Mailbox Sent: local=bfb2e03fdce327671e82bf173b1ccb8b/0/1, remote=30da3e14a611e860c43e2700d32ae34d/0/1: GUIDs conflict - will be merged later brain M: Mailbox Spam: local=59c97b78ad43f83b1f91e47936e7e7d1/0/1, remote=00000000000000000000000000000000/0/0: mailbox not selectable yet brain M: Mailbox Trash: local=7f5af7ba291b2df1a11d573bdb55d7e9/0/1, remote=31da3e14a611e860c43e2700d32ae34d/0/1: GUIDs conflict - will be merged later brain M: Mailbox forwebmaster: local=4602f79bfcc91efb337823d522c170b9/0/1, remote=00000000000000000000000000000000/0/0: mailbox not selectable yet brain M: Mailbox spam: local=00000000000000000000000000000000/0/0, remote=33da3e14a611e860c43e2700d32ae34d/0/1: mailbox not selectable yet Namespace : Using permissions from : mode=0700 gid=default Can't create mailbox Archive: Mailbox already exists brain S: Remote mailbox tree: Trash guid=7f5af7ba291b2df1a11d573bdb55d7e9 uid_validity=1632758505 uid_next=6 subs=yes last_change=0 last_subs=0 brain S: Remote mailbox tree: Spam guid=59c97b78ad43f83b1f91e47936e7e7d1 uid_validity=1632758508 uid_next=20 subs=yes last_change=0 last_subs=0 brain S: Remote mailbox tree: Sent guid=bfb2e03fdce327671e82bf173b1ccb8b uid_validity=1632758506 uid_next=1 subs=yes last_change=0 last_subs=0 brain S: Remote mailbox tree: Drafts guid=e0187b65e763143666d22094cedfe6a4 uid_validity=1632758507 uid_next=1 subs=yes last_change=0 last_subs=0 brain S: Remote mailbox tree: Archive guid=51a58e1d1d5c491aac71488647b20224 uid_validity=1632758509 uid_next=1 subs=yes last_change=0 last_subs=0 brain S: Remote mailbox tree: INBOX guid=c92f64f79f0d1ed01e6d5b314f04886c uid_validity=1632758504 uid_next=13 subs=yes last_change=0 last_subs=0
So it looks like it can see both sides of the sync, sees that they diverge (because they were created independently so the folders of the mailboxes have the same names but different GUIDs), then it tries to create the "missing" folders (by guid) which I expect fails because there's already a folder with that name? Something like that?
Anyway that's where I'm stuck so far, I've got no idea whether sync is
even the right direction but I don't really know what to try.
NOTES: the maildir I'm working on right now has 41 messages in it:
$ ls some/folder/backup/mail/domain/\*/cur | wc -l 41
though the uidlist has 45 lines (which is 3 more than I'd have expected):
$ wc -l some/folder/backup/mail/domain/\*/dovecot-uidlist 45 some/folder/backup/mail/domain/\*/dovecot-uidlist
Xavier
I don't have any direct answer to your question, but wanted to make sure you were aware of these tools, which are built to safely merge two IMAP mailboxes together with no loss. They might provide an alternative strategy.
Imapsync: https://github.com/imapsync/imapsync
Offlineimap: https://github.com/OfflineIMAP/offlineimap
P.
+1000 for IMAPSync. I was never able to get doveadm sync/backup to work despite a lot of tinkering attempting to do so. Never. I could get imapsync to work in <10m and it works incredibly well. I can't say enough good things about imapsync (And enough bad things about doveadm sync - I imagine it must work for some people, but how to do so is seemingly like witchcraft with secret incantations, potions and arcane hidden knowledge.) Save yourself the agony, and try imapsync *first,* not second. :)
I don't have any direct answer to your question, but wanted to make sure you were aware of these tools, which are built to safely merge two IMAP mailboxes together with no loss. They might provide an alternative strategy.
Imapsync: https://github.com/imapsync/imapsync
Offlineimap: https://github.com/OfflineIMAP/offlineimap
P.
Only want to reconfirm the +1000 for IMAPSync.
Used it a few years ago to sync all the mailboxes from dbmail to dovecot. And it worked flawlessly and without any issues.
Give it a first go :)
Am 04.10.2021 um 04:22 schrieb Gregory Sloop:
+1000 for IMAPSync.
I was never able to get doveadm sync/backup to work despite a lot of tinkering attempting to do so. Never.
I could get imapsync to work in <10m and it works incredibly well.
I can't say enough good things about imapsync
(And enough bad things about doveadm sync - I imagine it must work for some people, but how to do so is seemingly like witchcraft with secret incantations, potions and arcane hidden knowledge.)
Save yourself the agony, and try imapsync *first,* not second. :)
I don't have any direct answer to your question, but wanted to make sure you were aware of these tools, which are built to safely merge two IMAP mailboxes together with no loss. They might provide an alternative strategy. Imapsync: https://github.com/imapsync/imapsync <https://github.com/imapsync/imapsync> Offlineimap: https://github.com/OfflineIMAP/offlineimap <https://github.com/OfflineIMAP/offlineimap> P.
participants (4)
-
Admin Beckspaced
-
Gregory Sloop
-
Plutocrat
-
Xavier Morel