Recommended tool for migrating IMAP servers
Hi Friends, I would like to ask you a suggestion: I need to migrate a imap server to a new one and then dismiss the old one. Reading from relative Dovecot documentation page (https://wiki.dovecot.org/Migration), more tools are shown:
UW-IMAP's mailutil, imapsync, YippieMove and Larch.
The each mail servers are Linux based, one of this (mine) is Dovecot. Based on your experience which of these tools would be preferable to use?
Thank you very much
Davide
Hi,
I vouch for imapsync. Have used it in the past with quite a big amount of emails.
cheers.
x0p
Hi Friends, I would like to ask you a suggestion: I need to migrate a imap server to a new one and then dismiss the old one. Reading from relative Dovecot documentation page (https://wiki.dovecot.org/Migration), more tools are shown:
UW-IMAP's mailutil, imapsync, YippieMove and Larch.
The each mail servers are Linux based, one of this (mine) is Dovecot. Based on your experience which of these tools would be preferable to use?
Thank you very much
Davide
Imapsync for sure. Have used it for both IMAP to IMAP and IMAP to Exchange migrations. Works great.
On Dec 3, 2017, at 2:08 PM, x9p dovecot@x9p.org wrote:
Hi,
I vouch for imapsync. Have used it in the past with quite a big amount of emails.
cheers.
x0p
Hi Friends, I would like to ask you a suggestion: I need to migrate a imap server to a new one and then dismiss the old one. Reading from relative Dovecot documentation page (https://wiki.dovecot.org/Migration), more tools are shown:
UW-IMAP's mailutil, imapsync, YippieMove and Larch.
The each mail servers are Linux based, one of this (mine) is Dovecot. Based on your experience which of these tools would be preferable to use?
Thank you very much
Davide
Also if you have fs access on both servers, and you are using maildir, plain rsync works just as well.
Aki
On 04.12.2017 00:17, Harondel J. Sibble wrote:
Imapsync for sure. Have used it for both IMAP to IMAP and IMAP to Exchange migrations. Works great.
On Dec 3, 2017, at 2:08 PM, x9p dovecot@x9p.org wrote:
Hi,
I vouch for imapsync. Have used it in the past with quite a big amount of emails.
cheers.
x0p
Hi Friends, I would like to ask you a suggestion: I need to migrate a imap server to a new one and then dismiss the old one. Reading from relative Dovecot documentation page (https://wiki.dovecot.org/Migration), more tools are shown:
UW-IMAP's mailutil, imapsync, YippieMove and Larch.
The each mail servers are Linux based, one of this (mine) is Dovecot. Based on your experience which of these tools would be preferable to use?
Thank you very much
Davide
On 3 Dec 2017, at 23.23, Davide Marchi danjde@msw.it wrote:
Hi Friends, I would like to ask you a suggestion: I need to migrate a imap server to a new one and then dismiss the old one. Reading from relative Dovecot documentation page (https://wiki.dovecot.org/Migration), more tools are shown:
UW-IMAP's mailutil, imapsync, YippieMove and Larch.
The each mail servers are Linux based, one of this (mine) is Dovecot. Based on your experience which of these tools would be preferable to use?
If you want to preserve IMAP UID:s and possibly also POP3 UIDL:s then dovecot internal dsync is the only tool that can do it.
With every other tool you will face end users needing to invalidate their local caches and redownloading all headers if not also all mail bodies.
Sami
On Mon, Dec 4, 2017 at 9:16 AM, Sami Ketola sami.ketola@dovecot.fi wrote:
With every other tool you will face end users needing to invalidate their local caches and redownloading all headers if not also all mail bodies.
Sami
I don't think so. Been using imapsync for large scale migrations from external servers to our dovecot setup. Users don't even see it when the key is switched (DNS changes). Go for it.
Regards,
Webert Lima DevOps Engineer at MAV Tecnologia *Belo Horizonte - Brasil*
On 4 Dec 2017, at 19.59, Webert de Souza Lima webert.boss@gmail.com wrote:
On Mon, Dec 4, 2017 at 9:16 AM, Sami Ketola sami.ketola@dovecot.fi wrote:
With every other tool you will face end users needing to invalidate their local caches and redownloading all headers if not also all mail bodies.
Sami
I don't think so. Been using imapsync for large scale migrations from external servers to our dovecot setup. Users don't even see it when the key is switched (DNS changes). Go for it.
You are wrong. There is no way to assign IMAP UID:s over IMAP protocol. It simply does not support it. With imapsync there is absolutely no way to preserve them and you will face problems with IMAP UID:s not matching the cached mail anymore.
Trust us. We have run multiple migrations at scale of 10+ million users.
Sami
On Tue, Dec 5, 2017 at 4:16 AM, Sami Ketola sami.ketola@dovecot.fi wrote:
On Mon, Dec 4, 2017 at 9:16 AM, Sami Ketola sami.ketola@dovecot.fi wrote:
With every other tool you will face end users needing to invalidate their local caches and redownloading all headers if not also all mail bodies.
Sami
On 4 Dec 2017, at 19.59, Webert de Souza Lima webert.boss@gmail.com
wrote:
I don't think so. Been using imapsync for large scale migrations from external servers to our dovecot setup. Users don't even see it when the
key
is switched (DNS changes). Go for it.
You are wrong. There is no way to assign IMAP UID:s over IMAP protocol. It simply does not support it. With imapsync there is absolutely no way to preserve them and you will face problems with IMAP UID:s not matching the cached mail anymore.
Trust us. We have run multiple migrations at scale of 10+ million users.
Sami
Sorry, I might be wrong about cache invalidation, indeed. What I'm sure is that users will hardly notice any server change. We've never had user complaints about mailboxes resyncing or anything like that after imapsync'ing to a new server, that's why I'd recommend using imapsync with no worries.
It's easy enough to test this on a single user account first and see how is imap client's behavior after the DNS change.
Regards,
Webert Lima DevOps Engineer at MAV Tecnologia *Belo Horizonte - Brasil*
On 5 Dec 2017, at 12.46, Webert de Souza Lima webert.boss@gmail.com wrote:
Sorry, I might be wrong about cache invalidation, indeed. What I'm sure is that users will hardly notice any server change. We've never had user complaints about mailboxes resyncing or anything like that after imapsync'ing to a new server, that's why I'd recommend using imapsync with no worries.
It's easy enough to test this on a single user account first and see how is imap client's behavior after the DNS change.
Can't really do DNS change to test with one user as DNS change affects all users. What we usually do is put layer of dovecot proxies in front of the legacy and destination platform and then have database with information if user has been migrated already and forward the connection to legacy or dovecot platform based on that info. That enables us to migrate user-by-user.
And both users and mail server admins do notice if there is IMAP UID or POP3 UIDL changes. Imagine what happens when millions of users suddenly start to redownload terabytes of data.
Sami
On Tue, Dec 5, 2017 at 8:54 AM, Sami Ketola sami.ketola@dovecot.fi wrote:
Can't really do DNS change to test with one user as DNS change affects all users.
Yes, of course. For test purposes one could justchange the system's host file and simulate a DNS change, after imapsync is complete I meant a simple test just to see what happens at the client size, to see what the user itself could notice.
On Tue, Dec 5, 2017 at 8:54 AM, Sami Ketola sami.ketola@dovecot.fi wrote:
What we usually do is put layer of dovecot proxies in front of the legacy and destination platform and then have database with information if user has been migrated already and forward the connection to legacy or dovecot platform based on that info. That enables us to migrate user-by-user.
Cool, we do that too :D. We might have learned that from you even. In that way users can use our Webmail from day 1 when migration starts.
On Tue, Dec 5, 2017 at 8:54 AM, Sami Ketola sami.ketola@dovecot.fi wrote:
And both users and mail server admins do notice if there is IMAP UID or POP3 UIDL changes. Imagine what happens when millions of users suddenly start to redownload terabytes of data.
Sami
That would just kill my cluster. Happened once due to NFS sharing hang. Terrible day to be alive.
Regards,
Webert Lima DevOps Engineer at MAV Tecnologia *Belo Horizonte - Brasil*
Il 2017-12-05 07:16 Sami Ketola ha scritto: [..]
Trust us. We have run multiple migrations at scale of 10+ million users.
Sami
However, it seems that Imapsync has license issues and in fact it's not included in the Debian repositories. Is it to be used anyway or should be avoid?
Many thanks again
Davide
On 11 Dec 2017, at 12.32, Davide Marchi danjde@msw.it wrote:
Il 2017-12-05 07:16 Sami Ketola ha scritto: [..]
Trust us. We have run multiple migrations at scale of 10+ million users. Sami
However, it seems that Imapsync has license issues and in fact it's not included in the Debian repositories. Is it to be used anyway or should be avoid?
We run all our migrations using Dovecot internal dsync. Usually using imapc connector to connect to legacy platform.
Wqmi
On Mon, December 11, 2017 9:32 am, Davide Marchi wrote:
However, it seems that Imapsync has license issues and in fact it's not included in the Debian repositories. Is it to be used anyway or should be avoid?
I do not believe imapsync has license issues. Its written in perl and its hosted on github. You can pay for support if you want. and disable stats uploaded to their servers, via command line.
Many thanks again
Davide
cheers.
-- x9p | PGP : 0x03B50AF5EA4C8D80 / 5135 92C1 AD36 5293 2BDF DDCC 0DFA 74AE 1524 E7EE
participants (6)
-
Aki Tuomi
-
Davide Marchi
-
Harondel J. Sibble
-
Sami Ketola
-
Webert de Souza Lima
-
x9p