I’d like to migrate to a new server. I only need to transfer the mail store, have already done the users. I’ve seen different recommendations:
IMAP-Sync Rsync Doveadmin backup and dsync
What are the pros and cons of each? What would be best for me? I’m thinking that Doveadmin backup is the way to go.
Both old and new servers are on same network. New server is running Dovecot 2.3.2.1, old server is 2.2.32. macOS X.
Does anyone have any advice or examples?
Thanks, James.
You probably can do this with rsync, if you do not need to convert mail storage format. Safest way would be to stop both servers, do rsync, start both servers.
If your users have already written to the new server, you will have to use doveadm sync, otherwise data loss will occur.
Aki
On 04 September 2018 at 10:20 James Brown <jlbrown@bordo.com.au> wrote:
I’d like to migrate to a new server. I only need to transfer the mail store, have already done the users. I’ve seen different recommendations:
IMAP-Sync Rsync Doveadmin backup and dsync
What are the pros and cons of each? What would be best for me? I’m thinking that Doveadmin backup is the way to go.
Both old and new servers are on same network. New server is running Dovecot 2.3.2.1, old server is 2.2.32. macOS X.
Does anyone have any advice or examples?
Thanks, James.
On 4 Sep 2018, at 10.20, James Brown <jlbrown@bordo.com.au> wrote:
I’d like to migrate to a new server. I only need to transfer the mail store, have already done the users. I’ve seen different recommendations:
IMAP-Sync Rsync Doveadmin backup and dsync
What are the pros and cons of each? What would be best for me? I’m thinking that Doveadmin backup is the way to go.
Both old and new servers are on same network. New server is running Dovecot 2.3.2.1, old server is 2.2.32. macOS X.
Does anyone have any advice or examples?
It depends. Are your users already receiving mails to the new server? If yes, then the only way is to use "doveadm sync -1" or data will be lost. Even then data may be lost as UID numbers might already be used in the new server. If your users are not receiving mails to the new server it still depends. Is the mail storage format same in both servers? If yes, then it's probably fastest to use rsync. If not, then you need to use doveadm backup.
imapsync always loses data.
Sami
Thanks Sam
Users not using new mail server yet.
Both old and new using maildir.
Sounds like rsync is the way to go.
Thanks for your help.
James.
Sent from my iPhone XI
On 4 Sep 2018, at 5:41 pm, Sami Ketola <sami.ketola@dovecot.fi> wrote:
On 4 Sep 2018, at 10.20, James Brown <jlbrown@bordo.com.au> wrote:
I’d like to migrate to a new server. I only need to transfer the mail store, have already done the users. I’ve seen different recommendations:
IMAP-Sync Rsync Doveadmin backup and dsync
What are the pros and cons of each? What would be best for me? I’m thinking that Doveadmin backup is the way to go.
Both old and new servers are on same network. New server is running Dovecot 2.3.2.1, old server is 2.2.32. macOS X.
Does anyone have any advice or examples?
It depends. Are your users already receiving mails to the new server? If yes, then the only way is to use "doveadm sync -1" or data will be lost. Even then data may be lost as UID numbers might already be used in the new server. If your users are not receiving mails to the new server it still depends. Is the mail storage format same in both servers? If yes, then it's probably fastest to use rsync. If not, then you need to use doveadm backup.
imapsync always loses data.
Sami
If you have lots of data, you can make an initial pass with rsync whilst the old server is running and another rsync pass (with --delete, use dry run to test) when you've shut down the old server to sync the changes since the initial pass.
Good luck! Reio
On 04/09/2018 11:40, James Brown wrote:
Thanks Sam
Users not using new mail server yet.
Both old and new using maildir.
Sounds like rsync is the way to go.
Thanks for your help.
James.
Sent from my iPhone XI
On 4 Sep 2018, at 5:41 pm, Sami Ketola <sami.ketola@dovecot.fi> wrote:
On 4 Sep 2018, at 10.20, James Brown <jlbrown@bordo.com.au> wrote:
I’d like to migrate to a new server. I only need to transfer the mail store, have already done the users. I’ve seen different recommendations:
IMAP-Sync Rsync Doveadmin backup and dsync
What are the pros and cons of each? What would be best for me? I’m thinking that Doveadmin backup is the way to go.
Both old and new servers are on same network. New server is running Dovecot 2.3.2.1, old server is 2.2.32. macOS X.
Does anyone have any advice or examples?
It depends. Are your users already receiving mails to the new server? If yes, then the only way is to use "doveadm sync -1" or data will be lost. Even then data may be lost as UID numbers might already be used in the new server. If your users are not receiving mails to the new server it still depends. Is the mail storage format same in both servers? If yes, then it's probably fastest to use rsync. If not, then you need to use doveadm backup.
imapsync always loses data.
Sami
-- Tervitades Reio Remma
MR Stuudio 25 aastat
*MR Stuudio OÜ* Tondi 17b, 11316, Tallinn Tel +372 650 4808 Mob +372 56 22 00 33 reio@mrstuudio.ee www.mrstuudio.ee
How do you test the new server?
(Sorry, this seems like it must be obvious. The server I set up is the first one handling this domain, so testing was easy in that it didn't have history.)
Jeff Abrahamson +33 6 24 40 01 57 +44 7920 594 255
http://p27.eu/jeff/ http://transport-nantes.com/
On 04/09/18 10:49, Reio Remma wrote:
If you have lots of data, you can make an initial pass with rsync whilst the old server is running and another rsync pass (with --delete, use dry run to test) when you've shut down the old server to sync the changes since the initial pass.
Good luck! Reio
On 04/09/2018 11:40, James Brown wrote:
Thanks Sam
Users not using new mail server yet.
Both old and new using maildir.
Sounds like rsync is the way to go.
Thanks for your help.
James.
Sent from my iPhone XI
On 4 Sep 2018, at 5:41 pm, Sami Ketola <sami.ketola@dovecot.fi> wrote:
On 4 Sep 2018, at 10.20, James Brown <jlbrown@bordo.com.au> wrote:
I’d like to migrate to a new server. I only need to transfer the mail store, have already done the users. I’ve seen different recommendations:
IMAP-Sync Rsync Doveadmin backup and dsync
What are the pros and cons of each? What would be best for me? I’m thinking that Doveadmin backup is the way to go.
Both old and new servers are on same network. New server is running Dovecot 2.3.2.1, old server is 2.2.32. macOS X.
Does anyone have any advice or examples? It depends. Are your users already receiving mails to the new server? If yes, then the only way is to use "doveadm sync -1" or data will be lost. Even then data may be lost as UID numbers might already be used in the new server. If your users are not receiving mails to the new server it still depends. Is the mail storage format same in both servers? If yes, then it's probably fastest to use rsync. If not, then you need to use doveadm backup.
imapsync always loses data.
Sami
-- Tervitades Reio Remma
MR Stuudio 25 aastat
*MR Stuudio OÜ* Tondi 17b, 11316, Tallinn Tel +372 650 4808 Mob +372 56 22 00 33 reio@mrstuudio.ee www.mrstuudio.ee
--
Jeff Abrahamson +33 6 24 40 01 57 +44 7920 594 255
Thanks everyone for you help and suggestions. I think I’ll use rsync.
I had a try today, and all looked fine. I created the mail store directory with only root access on the new machine. Unfortunately that means I can’t transfer using this login (no root login over ssh).
I’ll have to change permissions, do the transfer, the lock it down afterwards.
Thanks again for your help.
James.
Sent from my iPad Pro
Am 04.09.2018 um 09:41 schrieb Sami Ketola:
imapsync always loses data
never saw this, be carefull by anounce such myths
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG, 80333 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
On 4 Sep 2018, at 17.47, Robert Schetterer <rs@sys4.de> wrote:
Am 04.09.2018 um 09:41 schrieb Sami Ketola:
imapsync always loses data
never saw this, be carefull by anounce such myths
It is a fact. Imapsync works over IMAP only and IMAP protocol does not even support transferring all data. At most at least UID numbering will be lost and end users need to invalidate their local caches.
Sami
Am 04.09.2018 um 16:52 schrieb Sami Ketola:
On 4 Sep 2018, at 17.47, Robert Schetterer <rs@sys4.de> wrote:
Am 04.09.2018 um 09:41 schrieb Sami Ketola:
imapsync always loses data
never saw this, be carefull by anounce such myths
It is a fact. Imapsync works over IMAP only and IMAP protocol does not even support transferring all data. At most at least UID numbering will be lost and end users need to invalidate their local caches.
but using "looses data" might others think it also may fail with imap, so be more detailed next time
Sami
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG, 80333 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
On 4 Sep 2018, at 18.00, Robert Schetterer <rs@sys4.de> wrote:
Am 04.09.2018 um 16:52 schrieb Sami Ketola:
On 4 Sep 2018, at 17.47, Robert Schetterer <rs@sys4.de> wrote:
Am 04.09.2018 um 09:41 schrieb Sami Ketola:
imapsync always loses data
never saw this, be carefull by anounce such myths
It is a fact. Imapsync works over IMAP only and IMAP protocol does not even support transferring all data. At most at least UID numbering will be lost and end users need to invalidate their local caches.
but using "looses data" might others think it also may fail with imap, so be more detailed next time
UID number is also data that is saved on the IMAP backend. If that is lost then it's "lost data".
Sami
Am 04.09.2018 um 17:18 schrieb Sami Ketola:
On 4 Sep 2018, at 18.00, Robert Schetterer <rs@sys4.de> wrote:
Am 04.09.2018 um 16:52 schrieb Sami Ketola:
On 4 Sep 2018, at 17.47, Robert Schetterer <rs@sys4.de> wrote:
Am 04.09.2018 um 09:41 schrieb Sami Ketola:
imapsync always loses data
never saw this, be carefull by anounce such myths
It is a fact. Imapsync works over IMAP only and IMAP protocol does not even support transferring all data. At most at least UID numbering will be lost and end users need to invalidate their local caches.
but using "looses data" might others think it also may fail with imap, so be more detailed next time
UID number is also data that is saved on the IMAP backend. If that is lost then it's "lost data".
Sami
Sorry i migrated terrabytes of mail with imapsync and never had a problem, it works as designed, also with maildir rsync did a good job, what never worked as it should was dsync ,cause of bugs ,that may changed now
so this is my answer to topic
"Best way to move mail from one server to another"
after all there is no "best way", there are a few ways choose the one which fits best to your needs goals and starting points ( kind of servers etc )
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG, 80333 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Quoting Robert Schetterer <rs@sys4.de>:
Am 04.09.2018 um 17:18 schrieb Sami Ketola:
On 4 Sep 2018, at 18.00, Robert Schetterer <rs@sys4.de> wrote:
Am 04.09.2018 um 16:52 schrieb Sami Ketola:
On 4 Sep 2018, at 17.47, Robert Schetterer <rs@sys4.de> wrote:
Am 04.09.2018 um 09:41 schrieb Sami Ketola:
imapsync always loses data
never saw this, be carefull by anounce such myths
It is a fact. Imapsync works over IMAP only and IMAP protocol
does not even support transferring all data. At most at least UID
numbering will be lost and end users need to invalidate their
local caches.but using "looses data" might others think it also may fail with imap, so be more detailed next time
UID number is also data that is saved on the IMAP backend. If that
is lost then it's "lost data".Sami
Sorry i migrated terrabytes of mail with imapsync and never had a problem, it works as designed, also with maildir rsync did a good job, what never worked as it should was dsync ,cause of bugs ,that may changed now
so this is my answer to topic
"Best way to move mail from one server to another"
after all there is no "best way", there are a few ways choose the one which fits best to your needs goals and starting points ( kind of servers etc )
Seems like there's an easy solution to the UID problem
Quoting Robert Schetterer <rs@sys4.de>:
Am 04.09.2018 um 17:18 schrieb Sami Ketola:
On 4 Sep 2018, at 18.00, Robert Schetterer <rs@sys4.de> wrote:
Am 04.09.2018 um 16:52 schrieb Sami Ketola:
On 4 Sep 2018, at 17.47, Robert Schetterer <rs@sys4.de> wrote:
Am 04.09.2018 um 09:41 schrieb Sami Ketola:
imapsync always loses data
never saw this, be carefull by anounce such myths
It is a fact. Imapsync works over IMAP only and IMAP protocol
does not even support transferring all data. At most at least UID
numbering will be lost and end users need to invalidate their
local caches.but using "looses data" might others think it also may fail with imap, so be more detailed next time
UID number is also data that is saved on the IMAP backend. If that
is lost then it's "lost data".Sami
Sorry i migrated terrabytes of mail with imapsync and never had a problem, it works as designed, also with maildir rsync did a good job, what never worked as it should was dsync ,cause of bugs ,that may changed now
so this is my answer to topic
"Best way to move mail from one server to another"
https://imapsync.lamiral.info/FAQ.d/FAQ.Duplicates.txt
Seems to use UIDs so that 'data' isn't lost.
On 4 Sep 2018, at 18.45, Rick Romero <rick@havokmon.com> wrote: https://imapsync.lamiral.info/FAQ.d/FAQ.Duplicates.txt
Seems to use UIDs so that 'data' isn't lost.
No it does not. This is different thing. This is about managing duplicates on multiple syncs. Imapsync seems to keep track of UID -> MAIL mapping for server A and UID -> MAIL mapping for server B so that on multiple runs it does not sync the same messages again. However it does not use the same UID -> MAIL for server B as is originally on server A. And that is the problem.
Sami
Quoting Sami Ketola <sami.ketola@dovecot.fi>:
On 4 Sep 2018, at 18.45, Rick Romero <rick@havokmon.com> wrote:
https://imapsync.lamiral.info/FAQ.d/FAQ.Duplicates.txtSeems to use UIDs so that 'data' isn't lost.
No it does not. This is different thing. This is about managing
duplicates on multiple syncs. Imapsync seems to keep track of UID
-> MAIL mapping for server A and UID -> MAIL mapping for server B so
that on multiple runs it does not sync the same messages again.
However it does not use the same UID -> MAIL for server B as is
originally on server A. And that is the problem.
That's not what you said, you said there was a client sync issue.
That's definitely an issue (especially if you run Mac Mail).
Ignoring duplicates on multiples runs is a non-issue, even with the
2011 version. Recording UIDs is not necessary for that.
Rick
On 4 Sep 2018, at 19.32, Rick Romero <rick@havokmon.com> wrote:
No it does not. This is different thing. This is about managing duplicates on multiple syncs. Imapsync seems to keep track of UID -> MAIL mapping for server A and UID -> MAIL mapping for server B so that on multiple runs it does not sync the same messages again. However it does not use the same UID -> MAIL for server B as is originally on server A. And that is the problem. That's not what you said, you said there was a client sync issue. That's definitely an issue (especially if you run Mac Mail).
Ignoring duplicates on multiples runs is a non-issue, even with the 2011 version. Recording UIDs is not necessary for that.
Nope. I said that with imapsync there is no way to preserve the same UID for a mail on the destination. Data is lost.
If the UID -> MAIL pairs are not kept then client caches need to be invalidated and redownloaded.
Sami
Sami is correct. imapsync loses data.
Source -> imapsync -> destination : the mailboxes are different to an IMAP client Source -> dsync -> destination (running Dovecot w/doveadm): the mailboxes are the same
You may not care that your users w/100,000 message mailboxes that have been cached on their client now have to invalidate that entire cache (and most likely re-download all of those messages).
So there is zero argument that, due to IMAP metadata loss, this is something that doesn't happen if dsync (or rsync) is used.
michael
On September 4, 2018 at 9:45 AM Rick Romero <rick@havokmon.com> wrote:
Quoting Robert Schetterer <rs@sys4.de mailto:rs@sys4.de >: > >Am 04.09.2018 um 17:18 schrieb Sami Ketola: > > >> > > >On 4 Sep 2018, at 18.00, Robert Schetterer <rs@sys4.de mailto:rs@sys4.de > wrote: Am 04.09.2018 um 16:52 schrieb Sami Ketola: > > > > >> > > > > >On 4 Sep 2018, at 17.47, Robert Schetterer <rs@sys4.de mailto:rs@sys4.de > wrote: Am 04.09.2018 um 09:41 schrieb Sami Ketola: > > > > > > >> imapsync always loses data > > > > > > > > never saw this, be carefull by anounce such myths
> > > > > It is a fact. Imapsync works over IMAP only and IMAP protocol does not even support transferring all data. At most at least UID numbering will be lost and end users need to invalidate their local caches.> > > > but using "looses data" might others think it also may fail with imap,so be more detailed next time > > > UID number is also data that is saved on the IMAP backend. If that is lost then it's "lost data".Sami > > Sorry i migrated terrabytes of mail with imapsync and never had aproblem, it works as designed, also with maildir rsync did a good job, what never worked as it should was dsync ,cause of bugs ,that may changed now so this is my answer to topic "Best way to move mail from one server to another" >https://imapsync.lamiral.info/FAQ.d/FAQ.Duplicates.txt Seems to use UIDs so that 'data' isn't lost.
On 4 Sep 2018, at 18.38, Robert Schetterer <rs@sys4.de> wrote:
Sorry i migrated terrabytes of mail with imapsync and never had a problem, it works as designed, also with maildir rsync did a good job, what never worked as it should was dsync ,cause of bugs ,that may changed now
I would like to very much hear about the problems you had with dsync as we have successfully migrated tens of petabytes of mails with it successfully while preserving the UID numbers. Imapsync would have been totally out of question as it is absolutely crucial in those migrations to keep the UID -> MAIL pairs matching legacy server. And usually also POP3 UIDLs. If not keeping the data the servers would have literally melted under load when switching over then of thousands of users forcing them to redownload headers of even mail bodies for millions of mails.
Sami
On 9/4/18 12:27 PM, Sami Ketola wrote:
On 4 Sep 2018, at 18.38, Robert Schetterer <rs@sys4.de> wrote:
Sorry i migrated terrabytes of mail with imapsync and never had a problem, it works as designed, also with maildir rsync did a good job, what never worked as it should was dsync ,cause of bugs ,that may changed now
I would like to very much hear about the problems you had with dsync as we have successfully migrated tens of petabytes of mails with it successfully while preserving the UID numbers. Imapsync would have been totally out of question as it is absolutely crucial in those migrations to keep the UID -> MAIL pairs matching legacy server. And usually also POP3 UIDLs. If not keeping the data the servers would have literally melted under load when switching over then of thousands of users forcing them to redownload headers of even mail bodies for millions of mails.
Sami
why does not rsync work?
Ruben
-- So many immigrant groups have swept through our town that Brooklyn, like Atlantis, reaches mythological proportions in the mind of the world - RI Safir 1998 http://www.mrbrklyn.com DRM is THEFT - We are the STAKEHOLDERS - RI Safir 2002
http://www.nylxs.com - Leadership Development in Free Software http://www.brooklyn-living.com
Being so tracked is for FARM ANIMALS and and extermination camps, but incompatible with living as a free human being. -RI Safir 2013
On 4 Sep 2018, at 21.24, Ruben Safir <ruben@mrbrklyn.com> wrote:
On 9/4/18 12:27 PM, Sami Ketola wrote:
On 4 Sep 2018, at 18.38, Robert Schetterer <rs@sys4.de> wrote:
Sorry i migrated terrabytes of mail with imapsync and never had a problem, it works as designed, also with maildir rsync did a good job, what never worked as it should was dsync ,cause of bugs ,that may changed now
I would like to very much hear about the problems you had with dsync as we have successfully migrated tens of petabytes of mails with it successfully while preserving the UID numbers. Imapsync would have been totally out of question as it is absolutely crucial in those migrations to keep the UID -> MAIL pairs matching legacy server. And usually also POP3 UIDLs. If not keeping the data the servers would have literally melted under load when switching over then of thousands of users forcing them to redownload headers of even mail bodies for millions of mails.
Sami
why does not rsync work?
Nobody said it does not work. But it can be used only if the mail storage format does not change.
Sami
Am 04.09.2018 um 18:27 schrieb Sami Ketola:
On 4 Sep 2018, at 18.38, Robert Schetterer <rs@sys4.de> wrote:
Sorry i migrated terrabytes of mail with imapsync and never had a problem, it works as designed, also with maildir rsync did a good job, what never worked as it should was dsync ,cause of bugs ,that may changed now
I would like to very much hear about the problems you had with dsync as we have successfully migrated tens of petabytes of mails with it successfully while preserving the UID numbers. Imapsync would have been totally out of question as it is absolutely crucial in those migrations to keep the UID -> MAIL pairs matching legacy server. And usually also POP3 UIDLs. If not keeping the data the servers would have literally melted under load when switching over then of thousands of users forcing them to redownload headers of even mail bodies for millions of mails.
Sami
I think you should know the dove buglist, i used dovecot since the beginning and believe me dsync had a lot of problems, as i stated i hope this had changed now. I didnt used/tested it again the last years cause i had no big migration job to do, we ever tried to use it that days cause youre right we would have prefer not broke pop3 stuff at migration. But it never worked, so imapsync was the best way to go, also having courier, cyrus migrations with format/namespace change etc.
Imapsync works as designed and migration might not be ideal at some points but it has no real stoppers if migration is well prepared and organized.
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG, 80333 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
On 04 September 2018 at 23:20 Robert Schetterer <rs@sys4.de> wrote:
Am 04.09.2018 um 18:27 schrieb Sami Ketola:
On 4 Sep 2018, at 18.38, Robert Schetterer <rs@sys4.de> wrote:
Sorry i migrated terrabytes of mail with imapsync and never had a problem, it works as designed, also with maildir rsync did a good job, what never worked as it should was dsync ,cause of bugs ,that may changed now
I would like to very much hear about the problems you had with dsync as we have successfully migrated tens of petabytes of mails with it successfully while preserving the UID numbers. Imapsync would have been totally out of question as it is absolutely crucial in those migrations to keep the UID -> MAIL pairs matching legacy server. And usually also POP3 UIDLs. If not keeping the data the servers would have literally melted under load when switching over then of thousands of users forcing them to redownload headers of even mail bodies for millions of mails.
Sami
I think you should know the dove buglist, i used dovecot since the beginning and believe me dsync had a lot of problems, as i stated i hope this had changed now. I didnt used/tested it again the last years cause i had no big migration job to do, we ever tried to use it that days cause youre right we would have prefer not broke pop3 stuff at migration. But it never worked, so imapsync was the best way to go, also having courier, cyrus migrations with format/namespace change etc.
Imapsync works as designed and migration might not be ideal at some points but it has no real stoppers if migration is well prepared and organized.
Best Regards MfG Robert Schetterer
There has been progress with dsync in the past years, and it can do courier and cyrus migrations with format/namespace changes. "Few years ago" is a long time.
Aki
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG, 80333 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Am 05.09.2018 um 07:37 schrieb Aki Tuomi:
On 04 September 2018 at 23:20 Robert Schetterer <rs@sys4.de> wrote:
Am 04.09.2018 um 18:27 schrieb Sami Ketola:
On 4 Sep 2018, at 18.38, Robert Schetterer <rs@sys4.de> wrote:
Sorry i migrated terrabytes of mail with imapsync and never had a problem, it works as designed, also with maildir rsync did a good job, what never worked as it should was dsync ,cause of bugs ,that may changed now
I would like to very much hear about the problems you had with dsync as we have successfully migrated tens of petabytes of mails with it successfully while preserving the UID numbers. Imapsync would have been totally out of question as it is absolutely crucial in those migrations to keep the UID -> MAIL pairs matching legacy server. And usually also POP3 UIDLs. If not keeping the data the servers would have literally melted under load when switching over then of thousands of users forcing them to redownload headers of even mail bodies for millions of mails.
Sami
I think you should know the dove buglist, i used dovecot since the beginning and believe me dsync had a lot of problems, as i stated i hope this had changed now. I didnt used/tested it again the last years cause i had no big migration job to do, we ever tried to use it that days cause youre right we would have prefer not broke pop3 stuff at migration. But it never worked, so imapsync was the best way to go, also having courier, cyrus migrations with format/namespace change etc.
Imapsync works as designed and migration might not be ideal at some points but it has no real stoppers if migration is well prepared and organized.
Best Regards MfG Robert Schetterer
There has been progress with dsync in the past years, and it can do courier and cyrus migrations with format/namespace changes. "Few years ago" is a long time.
we will give a retry end of this year and report
Aki
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG, 80333 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
-- [*] sys4 AG
https://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG,80333 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer, Wolfgang Stief Aufsichtsratsvorsitzender: Florian Kirstein
Hi Robert.
Am 06.09.18 um 08:50 schrieb Robert Schetterer:
Am 05.09.2018 um 07:37 schrieb Aki Tuomi:
On 04 September 2018 at 23:20 Robert Schetterer <rs@sys4.de> wrote:
Am 04.09.2018 um 18:27 schrieb Sami Ketola:
On 4 Sep 2018, at 18.38, Robert Schetterer <rs@sys4.de> wrote:
Sorry i migrated terrabytes of mail with imapsync and never had a problem, it works as designed, also with maildir rsync did a good job, what never worked as it should was dsync ,cause of bugs ,that may changed now
I would like to very much hear about the problems you had with dsync as we have successfully migrated tens of petabytes of mails with it successfully while preserving the UID numbers. Imapsync would have been totally out of question as it is absolutely crucial in those migrations to keep the UID -> MAIL pairs matching legacy server. And usually also POP3 UIDLs. If not keeping the data the servers would have literally melted under load when switching over then of thousands of users forcing them to redownload headers of even mail bodies for millions of mails.
Sami
I think you should know the dove buglist, i used dovecot since the beginning and believe me dsync had a lot of problems, as i stated i hope this had changed now. I didnt used/tested it again the last years cause i had no big migration job to do, we ever tried to use it that days cause youre right we would have prefer not broke pop3 stuff at migration. But it never worked, so imapsync was the best way to go, also having courier, cyrus migrations with format/namespace change etc.
Imapsync works as designed and migration might not be ideal at some points but it has no real stoppers if migration is well prepared and organized.
Best Regards MfG Robert Schetterer
There has been progress with dsync in the past years, and it can do courier and cyrus migrations with format/namespace changes. "Few years ago" is a long time.
we will give a retry end of this year and report
What's your opinion now for doveadm backup/sync ...?
Aki
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG, 80333 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Finally have some time to review list emails...
On Tue Sep 04 2018 03:41:50 GMT-0400 (Eastern Standard Time), Sami Ketola <sami.ketola@dovecot.fi> wrote:
imapsync always loses data.
Hi Sami,
Can you expand on this?
I used ImapSync to migrate from Dovecot to Office365 a couple of years ago, and didn't notice any issues with it at all.
On 26/09/2018 16:52, Tanstaafl wrote:
Finally have some time to review list emails...
On Tue Sep 04 2018 03:41:50 GMT-0400 (Eastern Standard Time), Sami Ketola <sami.ketola@dovecot.fi> wrote:
imapsync always loses data. Hi Sami,
Can you expand on this?
I used ImapSync to migrate from Dovecot to Office365 a couple of years ago, and didn't notice any issues with it at all.
Keep reading, the topic was discussed at length. :D
Good luck, Reio
Communigate Pro has a nice Migration/MoveIMAPMail utility. As Communigate is free to use for up to five accounts, I think it is legal to install it and use the MoveIMAPMail utility.
I have used it once to migrate mail from Communigate to dovecot.
-- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/
Never mind, should have waited and read the entire thread...
On Wed Sep 26 2018 09:52:26 GMT-0400 (Eastern Standard Time), Tanstaafl <tanstaafl@libertytrek.org> wrote:
Finally have some time to review list emails...
On Tue Sep 04 2018 03:41:50 GMT-0400 (Eastern Standard Time), Sami Ketola <sami.ketola@dovecot.fi> wrote:
imapsync always loses data.
Hi Sami,
Can you expand on this?
I used ImapSync to migrate from Dovecot to Office365 a couple of years ago, and didn't notice any issues with it at all.
I used (had to) imapsync two weeks ago to move Mails from dovecote 2.1 to 2.2, as doveadm failed (different versions issue).
Worked as expected. But all mails 60GB got resynced (summed up to approx. 250GB due to several accounts being used across multiple clients).
Tried both - without and with uid option. With latter one I did not have any duplicate mails being skipped; resync however seemed to be unavoidable.
-M
Am 26.09.2018 um 16:18 schrieb Tanstaafl <tanstaafl@libertytrek.org>:
Never mind, should have waited and read the entire thread...
On Wed Sep 26 2018 09:52:26 GMT-0400 (Eastern Standard Time), Tanstaafl <tanstaafl@libertytrek.org> wrote:
Finally have some time to review list emails...
On Tue Sep 04 2018 03:41:50 GMT-0400 (Eastern Standard Time), Sami Ketola <sami.ketola@dovecot.fi> wrote:
imapsync always loses data.
Hi Sami,
Can you expand on this?
I used ImapSync to migrate from Dovecot to Office365 a couple of years ago, and didn't notice any issues with it at all.
participants (13)
-
Admin
-
Aki Tuomi
-
Alex La
-
James Brown
-
Jeff Abrahamson
-
Michael Slusarz
-
Reio Remma
-
Rick Romero
-
Robert Schetterer
-
Ruben Safir
-
Sami Ketola
-
Tanstaafl
-
Victor Sudakov