[Dovecot] Migrating existing local Maildir structure to dovecot server Maildir
Hello,
I'm about to migrate a number of existing users who have been using POP3 with emails stored locally in KMail's Maildir format into an IMAP solution based on Dovecot.
Will it be possible to simply move all the KDE Maildir folders from the local PCs to the Maildir folder on the IMAP server running Dovecot and then have the emails accessible on the server, or will this most likely mess up the Dovecot index files?
If this causes index file trouble, is there any way to force a re-indexing of all these files, or is there another recommended way of migrating up to 7 years of email communication from local hosts to the IMAP server, for later putting them in folders?
Thanks in advance!
(PS. I noticed a few of the howto's on how to migrate to dovecot, but they only focus on moving from mbox to dovecot or similar, and not migrating existing Maildir emails to a new dovecot server... Any such pointers would be greatly appreciated :-))
-y1
On Friday 13 November 2009 15:00:09 Halim Issa wrote:
Hello,
I'm about to migrate a number of existing users who have been using POP3 with emails stored locally in KMail's Maildir format into an IMAP solution based on Dovecot.
Will it be possible to simply move all the KDE Maildir folders from the local PCs to the Maildir folder on the IMAP server running Dovecot and then have the emails accessible on the server, or will this most likely mess up the Dovecot index files?
If this causes index file trouble, is there any way to force a re-indexing of all these files, or is there another recommended way of migrating up to 7 years of email communication from local hosts to the IMAP server, for later putting them in folders?
I don't think the migration of the Maildir structure itself is the problem. Dovecot has no problem reading it, no matter where it came from, as long as it is a proper Maildir structure. Existing indexes should not be affected.
The indexes that KMail writes, however, will probably not migrate well or at all. I do not know anything about the index format KMail uses but converting it to something Dovecot understands and accepts seems troublesome and difficult to me.
This means that you can copy all existing, local Maildirs over to the server (you can even put them in subfolders in existing accounts) and Dovecot will read them just fine. But it also means that Dovecot will very probably present them to the user as new messages, never seen before.
But do yourself a favour and test any procedure you plan on implementing properly and extensively before actually implementing it. Otherwise it could end up messy and irreparable.
HTH
Andreas
Andreas Ntaflos Vienna, Austria
GPG Fingerprint: 6234 2E8E 5C81 C6CB E5EC 7E65 397C E2A8 090C A9B4
On 16/11/2009, Andreas Ntaflos daff@dword.org wrote:
On Friday 13 November 2009 15:00:09 Halim Issa wrote:
Hello,
I'm about to migrate a number of existing users who have been using POP3 with emails stored locally in KMail's Maildir format into an IMAP solution based on Dovecot.
Will it be possible to simply move all the KDE Maildir folders from the local PCs to the Maildir folder on the IMAP server running Dovecot and then have the emails accessible on the server, or will this most likely mess up the Dovecot index files?
If this causes index file trouble, is there any way to force a re-indexing of all these files, or is there another recommended way of migrating up to 7 years of email communication from local hosts to the IMAP server, for later putting them in folders?
I don't think the migration of the Maildir structure itself is the problem. Dovecot has no problem reading it, no matter where it came from, as long as it is a proper Maildir structure. Existing indexes should not be affected.
The indexes that KMail writes, however, will probably not migrate well or at all. I do not know anything about the index format KMail uses but converting it to something Dovecot understands and accepts seems troublesome and difficult to me.
This means that you can copy all existing, local Maildirs over to the server (you can even put them in subfolders in existing accounts) and Dovecot will read them just fine. But it also means that Dovecot will very probably present them to the user as new messages, never seen before.
But do yourself a favour and test any procedure you plan on implementing properly and extensively before actually implementing it. Otherwise it could end up messy and irreparable.
Thanks much for your thorough answer! I'll make sure to test extensively.
One thing came to mind - would it be possible to use Dovecot and Disconnected IMAP itself for the transfer, and perhaps thus keep the indexes? If I move the messages from the local mailbox to the IMAP mailbox from within Kmail it should perhaps keep the status?
Also - is there any documentation anywhere (or tips on better google search terms) on what is stored in these indexes? I suppose apart from Read/New status, it also contains whether or not there have been replies to the mail and on what date the email was replied to?
Thanks again!
On Monday 16 November 2009 12:35:35 Halim Issa wrote:
On 16/11/2009, Andreas Ntaflos daff@dword.org wrote:
On Friday 13 November 2009 15:00:09 Halim Issa wrote:
Hello,
I'm about to migrate a number of existing users who have been using POP3 with emails stored locally in KMail's Maildir format into an IMAP solution based on Dovecot.
Will it be possible to simply move all the KDE Maildir folders from the local PCs to the Maildir folder on the IMAP server running Dovecot and then have the emails accessible on the server, or will this most likely mess up the Dovecot index files?
If this causes index file trouble, is there any way to force a re-indexing of all these files, or is there another recommended way of migrating up to 7 years of email communication from local hosts to the IMAP server, for later putting them in folders?
I don't think the migration of the Maildir structure itself is the problem. Dovecot has no problem reading it, no matter where it came from, as long as it is a proper Maildir structure. Existing indexes should not be affected.
The indexes that KMail writes, however, will probably not migrate well or at all. I do not know anything about the index format KMail uses but converting it to something Dovecot understands and accepts seems troublesome and difficult to me.
This means that you can copy all existing, local Maildirs over to the server (you can even put them in subfolders in existing accounts) and Dovecot will read them just fine. But it also means that Dovecot will very probably present them to the user as new messages, never seen before.
But do yourself a favour and test any procedure you plan on implementing properly and extensively before actually implementing it. Otherwise it could end up messy and irreparable.
Thanks much for your thorough answer! I'll make sure to test extensively.
One thing came to mind - would it be possible to use Dovecot and Disconnected IMAP itself for the transfer, and perhaps thus keep the indexes? If I move the messages from the local mailbox to the IMAP mailbox from within Kmail it should perhaps keep the status?
Interesting idea, but I don't know how well or even if KMail handles moving messages from one account to another. In any case it seems worth a try.
Also - is there any documentation anywhere (or tips on better google search terms) on what is stored in these indexes? I suppose apart from Read/New status, it also contains whether or not there have been replies to the mail and on what date the email was replied to?
If you are talking about the indexes KMail keeps I suppose you should try one of the KDE-PIM-related mailing lists. For Dovecot's index files you should ask Timo or browse the wiki, maybe there is some documentation about it somewhere. I myself don't know anything about the format of any of the index files, sorry.
Andreas
Andreas Ntaflos Vienna, Austria
GPG Fingerprint: 6234 2E8E 5C81 C6CB E5EC 7E65 397C E2A8 090C A9B4
On Mon, 2009-11-16 at 12:35 +0100, Halim Issa wrote:
One thing came to mind - would it be possible to use Dovecot and Disconnected IMAP itself for the transfer, and perhaps thus keep the indexes? If I move the messages from the local mailbox to the IMAP mailbox from within Kmail it should perhaps keep the status?
Also - is there any documentation anywhere (or tips on better google search terms) on what is stored in these indexes? I suppose apart from Read/New status, it also contains whether or not there have been replies to the mail and on what date the email was replied to?
I doubt you need to worry about the indexes. The messages' flag state should be stored in the maildir filenames, and I'd hope that kmail does that too.
Timo Sirainen tss@iki.fi wrote:
On Mon, 2009-11-16 at 12:35 +0100, Halim Issa wrote:
One thing came to mind - would it be possible to use Dovecot and Disconnected IMAP itself for the transfer, and perhaps thus keep the indexes? If I move the messages from the local mailbox to the IMAP mailbox from within Kmail it should perhaps keep the status?
Also - is there any documentation anywhere (or tips on better google search terms) on what is stored in these indexes? I suppose apart from Read/New status, it also contains whether or not there have been replies to the mail and on what date the email was replied to?
I doubt you need to worry about the indexes. The messages' flag state should be stored in the maildir filenames, and I'd hope that kmail does that too.
How low on your list of priorities would land idea of utility "make-mailstore-tidy"? :-)
P.S. I have "personal mail store" migrated from cyrus (by my own *ugly* migration script) via maildir to dbox. I would like to convert all file names to more "uniform" format before next migration I sure I will see in a few years perspective :-)
-- [pl>en: Andrew] Andrzej Adam Filip : anfi@onet.eu Not that I have anything much against redundancy. But I said that already. -- Larry Wall in 199702271735.JAA04048@wall.org
On Mon, 2009-11-16 at 19:19 +0100, Andrzej Adam Filip wrote:
I doubt you need to worry about the indexes. The messages' flag state should be stored in the maildir filenames, and I'd hope that kmail does that too.
How low on your list of priorities would land idea of utility "make-mailstore-tidy"? :-)
P.S. I have "personal mail store" migrated from cyrus (by my own *ugly* migration script) via maildir to dbox. I would like to convert all file names to more "uniform" format before next migration I sure I will see in a few years perspective :-)
What exactly do you think such utility should do? What filenames?
Timo Sirainen tss@iki.fi wrote:
On Mon, 2009-11-16 at 19:19 +0100, Andrzej Adam Filip wrote:
I doubt you need to worry about the indexes. The messages' flag state should be stored in the maildir filenames, and I'd hope that kmail does that too.
How low on your list of priorities would land idea of utility "make-mailstore-tidy"? :-)
P.S. I have "personal mail store" migrated from cyrus (by my own *ugly* migration script) via maildir to dbox. I would like to convert all file names to more "uniform" format before next migration I sure I will see in a few years perspective :-)
What exactly do you think such utility should do? What filenames?
Let me explain it "by example":
I use dbox mailbox. In dbox-Mails folder I have message files named:
- u.99 [dbox created file name]
- 1231249011.P20963Q0M722109.shevek:2,Sa [dovecot maildir created&modified name]
- 1.:2,Sa [cyrus name, modified by dovecot maildir] [ I have used my own cyrus to maildir conversion scripts]
"Inconsistent" conventions for names of "email message file" is not a very big deal after one conversion but it may lead (IMHO) to very ugly chaos after a few conversions (after a 5-10 years of persistent use).
-- [pl>en: Andrew] Andrzej Adam Filip : anfi@onet.eu "Show me a good loser, and I'll show you a loser." -- Vince Lombardi, football coach
On Mon, 2009-11-23 at 22:14 +0100, Andrzej Adam Filip wrote:
I use dbox mailbox. In dbox-Mails folder I have message files named:
- u.99 [dbox created file name]
- 1231249011.P20963Q0M722109.shevek:2,Sa [dovecot maildir created&modified name]
- 1.:2,Sa [cyrus name, modified by dovecot maildir] [ I have used my own cyrus to maildir conversion scripts]
Oh, you're talking about the maildir-dbox hybrid thing. That was probably a worse idea than I originally realized. v2.0 can no longer read it, so you need to get it converted to dbox-only-files before upgrading to it. Now, how exactly to do that .. I suppose I'll have to add some setting or some utility to v1.2.
Timo Sirainen tss@iki.fi wrote:
On Mon, 2009-11-23 at 22:14 +0100, Andrzej Adam Filip wrote:
I use dbox mailbox. In dbox-Mails folder I have message files named:
- u.99 [dbox created file name]
- 1231249011.P20963Q0M722109.shevek:2,Sa [dovecot maildir created&modified name]
- 1.:2,Sa [cyrus name, modified by dovecot maildir] [ I have used my own cyrus to maildir conversion scripts]
Oh, you're talking about the maildir-dbox hybrid thing. That was probably a worse idea than I originally realized. v2.0 can no longer read it, so you need to get it converted to dbox-only-files before upgrading to it. Now, how exactly to do that .. I suppose I'll have to add some setting or some utility to v1.2.
One more thing: I would be nice (from my perspective) to keep "message file" in "unix mailbox box file" format (with one or many messages in the file).
Current "message files" in dbox mailbox contain some "binary" header and tail - it makes some "very simple" scripting not so simple.
-- [pl>en: Andrew] Andrzej Adam Filip : anfi@onet.eu < sam> /.ing an issue is like asking an infinite number of monkeys for advice -- in #debian-devel
On Mon, 2009-11-23 at 22:41 +0100, Andrzej Adam Filip wrote:
One more thing: I would be nice (from my perspective) to keep "message file" in "unix mailbox box file" format (with one or many messages in the file).
That's not going to happen.
Current "message files" in dbox mailbox contain some "binary" header and tail - it makes some "very simple" scripting not so simple.
There's going to be a new doveadm utility in Dovecot v2.0. Once it gets more features, you can hopefully do all the scripting you want with it.
participants (4)
-
Andreas Ntaflos
-
Andrzej Adam Filip
-
Halim Issa
-
Timo Sirainen