Exporting a mailbox over IMAP.
Xavier Morel
lists at masklinn.net
Sun Oct 3 15:30:53 EEST 2021
Hello,
Upfront note: I know very little about dovecot and IMAP lingo / terms of
art, I'll try to be clear and complete but I may be missing important
bits.
I have a VPS backup with mailboxes in the maildir dovecot format:
$ ls some/folder/backup/mail/domain/\*
cur dovecot-uidlist dovecot-uidvalidity.5c0637b7 dovecot.index.cache dovecot.mailbox.log subscriptions
dovecot-keywords dovecot-uidvalidity dovecot.index dovecot.index.log new tmp
Separately and independently, I also have an existing IMAP account I do
not personally manage. By "account" I mean that if I connect to the IMAP
server, LOGIN, then LIST I see a pretty standard set of folders:
(\HasNoChildren) "/" INBOX
(\HasNoChildren \Archive) "/" Archive
(\HasNoChildren \Drafts) "/" Drafts
(\HasNoChildren \XNotes) "/" Notes
(\HasNoChildren \Sent) "/" Sent
(\HasNoChildren \Junk) "/" Spam
(\HasNoChildren \Trash) "/" Trash
I would like to copy / upload / … the content of the backup mailboxes to
the IMAP account, ideally without losing the metadata.
I asked about this on IRC and user "cmouse" provided me with information
about the imapc storage (plugin?), a sample "doveadmin import" command,
and a sample dovecot configuration which I expanded to this (for simplicity,
the original sample command used a lot of `-c` and I figured I could just
shove them into the configuration file for readability):
$ dovecot -n
# 2.3.16 (): /opt/local/etc/dovecot/dovecot.conf
# OS: Darwin 15.6.0 x86_64
auth_verbose = yes
imapc_host = <server>
imapc_password = # hidden, use -P to show it
imapc_port = 993
imapc_ssl = imaps
imapc_user = <user>
mail_location = imapc:
# did not setup a "vmail" user or group as I just want to move the contents
mail_uid = <current_user>
mail_gid = staff
namespace {
inbox = yes
location =
prefix =
separator = /
}
ssl = no
verbose_proctitle = yes
verbose_ssl = yes
That did not work as I guess "import" wants to create all the
mailboxes, and these already exist:
$ doveadm -Dv import maildir:some/folder/backup/mail/domain/\* '.' all
[…]
Debug: Mailbox Archive: Mailbox opened because: import
Debug: Mailbox spam: Mailbox opened because: import
Debug: Mailbox Junk: Mailbox opened because: import
Debug: Mailbox Trash: Mailbox opened because: import
Debug: Mailbox Sent: Mailbox opened because: import
Debug: Mailbox Drafts: Mailbox opened because: import
Debug: Mailbox INBOX.Spam: Mailbox opened because: import
Debug: Mailbox INBOX.Junk: Mailbox opened because: import
Debug: Mailbox INBOX.Drafts: Mailbox opened because: import
Debug: Mailbox INBOX: Mailbox opened because: import
Debug: Namespace : Using permissions from : mode=0700 gid=default
Error: Couldn't create mailbox ./INBOX: Mailbox already exists
I tried messing around with the options but didn't achieve anything, the
only change which did anything visible was to replace '.' with '.INBOX'
which resulted in import creating `.IMPORT/IMPORT` and exiting with:
Error: Couldn't create mailbox INBOX/INBOX: Invalid mailbox name
which… fair enough.
Looking around I found `doveadm sync` which seemed promising, so far I
arrived not far:
$ doveadm -Dv sync -f maildir:some/folder/backup/mail/domain/\*
this ultimately fails as well:
brain S: Local mailbox tree: INBOX guid=63ace42cc352935925190000691e7660 uid_validity=1315910895 uid_next=71 subs=no last_change=0 last_subs=0
brain M: Local mailbox tree: Trash guid=7f5af7ba291b2df1a11d573bdb55d7e9 uid_validity=1632758505 uid_next=6 subs=yes last_change=0 last_subs=0
brain S: Local mailbox tree: INBOX/Drafts guid=ebb8a22812fe675b34390000691e7660 uid_validity=1533541906 uid_next=2 subs=yes last_change=0 last_subs=0
brain M: Local mailbox tree: Spam guid=59c97b78ad43f83b1f91e47936e7e7d1 uid_validity=1632758508 uid_next=20 subs=yes last_change=0 last_subs=0
brain S: Local mailbox tree: INBOX/Junk guid=62ace42cc352935925190000691e7660 uid_validity=1490450196 uid_next=2 subs=yes last_change=0 last_subs=0
brain M: Local mailbox tree: Sent guid=bfb2e03fdce327671e82bf173b1ccb8b uid_validity=1632758506 uid_next=1 subs=yes last_change=0 last_subs=0
brain S: Local mailbox tree: INBOX/Spam guid=64ace42cc352935925190000691e7660 uid_validity=1315911183 uid_next=1 subs=yes last_change=0 last_subs=0
brain M: Local mailbox tree: Drafts guid=e0187b65e763143666d22094cedfe6a4 uid_validity=1632758507 uid_next=1 subs=yes last_change=0 last_subs=0
brain S: Local mailbox tree: Drafts guid=2fda3e14a611e860c43e2700d32ae34d uid_validity=1625821607 uid_next=1 subs=no last_change=0 last_subs=0
brain M: Local mailbox tree: Archive guid=51a58e1d1d5c491aac71488647b20224 uid_validity=1632758509 uid_next=1 subs=yes last_change=0 last_subs=0
brain S: Local mailbox tree: Sent guid=30da3e14a611e860c43e2700d32ae34d uid_validity=1625821608 uid_next=1 subs=no last_change=0 last_subs=0
brain M: Local mailbox tree: INBOX guid=c92f64f79f0d1ed01e6d5b314f04886c uid_validity=1632758504 uid_next=13 subs=yes last_change=0 last_subs=0
brain S: Local mailbox tree: Trash guid=31da3e14a611e860c43e2700d32ae34d uid_validity=1625821609 uid_next=1 subs=no last_change=0 last_subs=0
brain S: Local mailbox tree: Junk guid=32da3e14a611e860c43e2700d32ae34d uid_validity=1625821610 uid_next=1 subs=no last_change=0 last_subs=0
brain S: Local mailbox tree: spam guid=33da3e14a611e860c43e2700d32ae34d uid_validity=1625821611 uid_next=1 subs=no last_change=0 last_subs=0
brain M: Remote mailbox tree: INBOX guid=63ace42cc352935925190000691e7660 uid_validity=1315910895 uid_next=71 subs=no last_change=0 last_subs=0
brain M: Remote mailbox tree: INBOX/Drafts guid=ebb8a22812fe675b34390000691e7660 uid_validity=1533541906 uid_next=2 subs=yes last_change=0 last_subs=0
brain M: Remote mailbox tree: INBOX/Junk guid=62ace42cc352935925190000691e7660 uid_validity=1490450196 uid_next=2 subs=yes last_change=0 last_subs=0
brain M: Remote mailbox tree: INBOX/Spam guid=64ace42cc352935925190000691e7660 uid_validity=1315911183 uid_next=1 subs=yes last_change=0 last_subs=0
brain M: Remote mailbox tree: Drafts guid=2fda3e14a611e860c43e2700d32ae34d uid_validity=1625821607 uid_next=1 subs=no last_change=0 last_subs=0
brain M: Remote mailbox tree: Sent guid=30da3e14a611e860c43e2700d32ae34d uid_validity=1625821608 uid_next=1 subs=no last_change=0 last_subs=0
brain M: Remote mailbox tree: Trash guid=31da3e14a611e860c43e2700d32ae34d uid_validity=1625821609 uid_next=1 subs=no last_change=0 last_subs=0
brain M: Remote mailbox tree: Junk guid=32da3e14a611e860c43e2700d32ae34d uid_validity=1625821610 uid_next=1 subs=no last_change=0 last_subs=0
brain M: Remote mailbox tree: spam guid=33da3e14a611e860c43e2700d32ae34d uid_validity=1625821611 uid_next=1 subs=no last_change=0 last_subs=0
brain S: Local mailbox tree: Archive guid=4d10f3271f03f7603b232100d32ae34d uid_validity=1533541907 uid_next=1 subs=yes last_change=0 last_subs=0
brain M: Remote mailbox tree: Archive guid=4d10f3271f03f7603b232100d32ae34d uid_validity=1533541907 uid_next=1 subs=yes last_change=0 last_subs=0
brain M: Mailbox Archive: local=51a58e1d1d5c491aac71488647b20224/0/1, remote=4d10f3271f03f7603b232100d32ae34d/0/1: GUIDs conflict - will be merged later
brain M: Mailbox Drafts: local=e0187b65e763143666d22094cedfe6a4/0/1, remote=2fda3e14a611e860c43e2700d32ae34d/0/1: GUIDs conflict - will be merged later
brain M: Mailbox INBOX: local=c92f64f79f0d1ed01e6d5b314f04886c/0/1, remote=63ace42cc352935925190000691e7660/0/1: GUIDs conflict - will be merged later
brain M: Mailbox INBOX/Drafts: local=00000000000000000000000000000000/0/0, remote=ebb8a22812fe675b34390000691e7660/0/1: mailbox not selectable yet
brain M: Mailbox INBOX/Junk: local=00000000000000000000000000000000/0/0, remote=62ace42cc352935925190000691e7660/0/1: mailbox not selectable yet
brain M: Mailbox INBOX/Spam: local=00000000000000000000000000000000/0/0, remote=64ace42cc352935925190000691e7660/0/1: mailbox not selectable yet
brain M: Mailbox Junk: local=00000000000000000000000000000000/0/0, remote=32da3e14a611e860c43e2700d32ae34d/0/1: mailbox not selectable yet
brain M: Mailbox Sent: local=bfb2e03fdce327671e82bf173b1ccb8b/0/1, remote=30da3e14a611e860c43e2700d32ae34d/0/1: GUIDs conflict - will be merged later
brain M: Mailbox Spam: local=59c97b78ad43f83b1f91e47936e7e7d1/0/1, remote=00000000000000000000000000000000/0/0: mailbox not selectable yet
brain M: Mailbox Trash: local=7f5af7ba291b2df1a11d573bdb55d7e9/0/1, remote=31da3e14a611e860c43e2700d32ae34d/0/1: GUIDs conflict - will be merged later
brain M: Mailbox forwebmaster: local=4602f79bfcc91efb337823d522c170b9/0/1, remote=00000000000000000000000000000000/0/0: mailbox not selectable yet
brain M: Mailbox spam: local=00000000000000000000000000000000/0/0, remote=33da3e14a611e860c43e2700d32ae34d/0/1: mailbox not selectable yet
Namespace : Using permissions from : mode=0700 gid=default
Can't create mailbox Archive: Mailbox already exists
brain S: Remote mailbox tree: Trash guid=7f5af7ba291b2df1a11d573bdb55d7e9 uid_validity=1632758505 uid_next=6 subs=yes last_change=0 last_subs=0
brain S: Remote mailbox tree: Spam guid=59c97b78ad43f83b1f91e47936e7e7d1 uid_validity=1632758508 uid_next=20 subs=yes last_change=0 last_subs=0
brain S: Remote mailbox tree: Sent guid=bfb2e03fdce327671e82bf173b1ccb8b uid_validity=1632758506 uid_next=1 subs=yes last_change=0 last_subs=0
brain S: Remote mailbox tree: Drafts guid=e0187b65e763143666d22094cedfe6a4 uid_validity=1632758507 uid_next=1 subs=yes last_change=0 last_subs=0
brain S: Remote mailbox tree: Archive guid=51a58e1d1d5c491aac71488647b20224 uid_validity=1632758509 uid_next=1 subs=yes last_change=0 last_subs=0
brain S: Remote mailbox tree: INBOX guid=c92f64f79f0d1ed01e6d5b314f04886c uid_validity=1632758504 uid_next=13 subs=yes last_change=0 last_subs=0
So it looks like it can see both sides of the sync, sees that they
diverge (because they were created independently so the folders of the
mailboxes have the same names but different GUIDs), then it tries to
create the "missing" folders (by guid) which I expect fails because
there's already a folder with that name? Something like that?
Anyway that's where I'm stuck so far, I've got no idea whether `sync` is
even the right direction but I don't really know what to try.
NOTES: the maildir I'm working on right now has 41 messages in it:
$ ls some/folder/backup/mail/domain/\*/cur | wc -l
41
though the uidlist has 45 lines (which is 3 more than I'd have expected):
$ wc -l some/folder/backup/mail/domain/\*/dovecot-uidlist
45 some/folder/backup/mail/domain/\*/dovecot-uidlist
Xavier
More information about the dovecot
mailing list