[Dovecot] move mail from server with v1.0 to server w. v2.1?
I'm about to replace my old server. Both servers are on my internal network. The old one ist running debian lenny(32bit) with dovecot 1.0.15 and the new one debian Wheezy with dovecot 2.1.7. I set up the users on both systems identically (same gid uid). My first attempt was tho simply rsync the Maildirs along with the homes. Both deovecot versions are set up with the maildir format, but this results in ~30% doublicated mails on the new server. I then learned about doveadm/dsync, only to figure out that this isn't available in v 1.0.15 and therefor won't work. So which could be the most painless way to solve that?
Am 01.02.2013 17:22, schrieb Frank Lienhard:
I'm about to replace my old server. Both servers are on my internal network. The old one ist running debian lenny(32bit) with dovecot 1.0.15 and the new one debian Wheezy with dovecot 2.1.7. I set up the users on both systems identically (same gid uid). My first attempt was tho simply rsync the Maildirs along with the homes. Both deovecot versions are set up with the maildir format, but this results in ~30% doublicated mails on the new server. I then learned about doveadm/dsync, only to figure out that this isn't available in v 1.0.15 and therefor won't work. So which could be the most painless way to solve that?
imapsync is version agnostic and if you have a database with the user-logins it should be easy to geenrate a shellscript
i did migration of around 200000 messages and some hundret users from Apple OSX EIMS to dbmail/dovecot in 2009 this way
Reindl Harald wrote:
Am 01.02.2013 17:22, schrieb Frank Lienhard:
I'm about to replace my old server. Both servers are on my internal network. The old one ist running debian lenny(32bit) with dovecot 1.0.15 and the new one debian Wheezy with dovecot 2.1.7. I set up the users on both systems identically (same gid uid). My first attempt was tho simply rsync the Maildirs along with the homes. Both deovecot versions are set up with the maildir format, but this results in ~30% doublicated mails on the new server. I then learned about doveadm/dsync, only to figure out that this isn't available in v 1.0.15 and therefor won't work. So which could be the most painless way to solve that?
imapsync is version agnostic and if you have a database with the user-logins it should be easy to geenrate a shellscript
i did migration of around 200000 messages and some hundret users from Apple OSX EIMS to dbmail/dovecot in 2009 this way
sadly, this is not availble from the debian repos. I searched a bit around: syncmaildir, imapsync: only available in in Lenny OR Wheezy, but needed on both servers isync, mailsync: mentioned to hold two maildirs in sync with a more or less difficult to configure setup which leaves me one option (from programms I found); offlineimap: mentioned to backup mails from a remote/ISP mailserver in a local directory of a user. Wonder if that works , when the local dir is one of an imap server.......
Am 01.02.2013 18:19, schrieb Frank Lienhard:
Reindl Harald wrote:
Am 01.02.2013 17:22, schrieb Frank Lienhard:
I'm about to replace my old server. Both servers are on my internal network. The old one ist running debian lenny(32bit) with dovecot 1.0.15 and the new one debian Wheezy with dovecot 2.1.7. I set up the users on both systems identically (same gid uid). My first attempt was tho simply rsync the Maildirs along with the homes. Both deovecot versions are set up with the maildir format, but this results in ~30% doublicated mails on the new server. I then learned about doveadm/dsync, only to figure out that this isn't available in v 1.0.15 and therefor won't work. So which could be the most painless way to solve that?
imapsync is version agnostic and if you have a database with the user-logins it should be easy to geenrate a shellscript
i did migration of around 200000 messages and some hundret users from Apple OSX EIMS to dbmail/dovecot in 2009 this way
sadly, this is not availble from the debian repos. I searched a bit around: syncmaildir, imapsync: only available in in Lenny OR Wheezy, but needed on both servers
needed on both servers is simply not true i have even a template in my admin-backend to generate imapsync scripts out of a db
imapsync --host1 [source] --user1 [user1] --password1 [pwd1] --authmech1 LOGIN --host2 [target] --user2 [user2] --password2 [pwd2] --authmech2 LOGIN --skipsize
Hi Frank,
Frank Lienhard wrote:
Reindl Harald wrote:
Am 01.02.2013 17:22, schrieb Frank Lienhard:
I'm about to replace my old server. Both servers are on my internal network. The old one ist running debian lenny(32bit) with dovecot 1.0.15 and the new one debian Wheezy with dovecot 2.1.7.
imapsync is version agnostic and if you have a database with the user-logins it should be easy to generate a shellscript
sadly, this is not availble from the debian repos.
imapsync is written in perl and available on github: https://github.com/imapsync/imapsync
This is IMHO the best tool where dsync cannot be applied (as in your case). The only drawback of imapsync is that it cannot preserve UIDs since it uploads the messages via IMAP and the server will assign new message UIDs.
Regards Daniel
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Fri, 1 Feb 2013, Frank Lienhard wrote:
My first attempt was tho simply rsync the Maildirs along with the homes. Both deovecot versions are set up with the maildir format, but this results in ~30% doublicated mails on the new server.
Er, a) you got 30% more files on the new server (aka target), than you have on the old server (aka source)??
b) Or do you see them, when you login via IMAP to the the new server?
===
If a), did you ran rsync, while the source host is in production still? Do you use "H" option to preserve hardlinks and/or "--delete" if you run rsync multiple times?
If b), did you verified the duplication with a IMAP client, that does _no_ caching on its own or with a fresh and empty local cache / profile / ... .
Kind regards,
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux)
iQEVAwUBUQ9zZ13r2wJMiz2NAQI67wf/X6XVPzXOygUZWX8s6Wthf8UyHs+Pew9v QKKuwOvm4lb7mF9vzVYMtTFwSHyYUCvSgB+oSPmRpX/bGReSq1+PwpIYQOpKvU+b EpaWTyBMXDyzToPuwnd/VZgnzzVuBa5IVbvg6neLIi7yrco+VjbwQJ1B/VPLHjRO YVusoMn0ddJWNE01ix2w54kBc9eSS54hcIuzkac3fyp2vTLl3LYgBJ0lRUp2ztkX C4sn1ejOqXkfBLc20jL52RW+FW0pH0P7HTCCz+XWkJFmiMXcDEoymBL62fI1MuAh UMNPvUIVcZcT/Q5FA8QGVRnRyKH5S4snRPkHEqSP2A39tDhbM/882A== =XIiO -----END PGP SIGNATURE-----
Steffen Kaiser wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Fri, 1 Feb 2013, Frank Lienhard wrote:
My first attempt was tho simply rsync the Maildirs along with the homes. Both deovecot versions are set up with the maildir format, but this results in ~30% doublicated mails on the new server.
Er, a) you got 30% more files on the new server (aka target), than you have on the old server (aka source)??
b) Or do you see them, when you login via IMAP to the the new server?
===
If a), did you ran rsync, while the source host is in production still? Do you use "H" option to preserve hardlinks and/or "--delete" if you run rsync multiple times?
If b), did you verified the duplication with a IMAP client, that does _no_ caching on its own or with a fresh and empty local cache / profile / ... .
I used -avu, where: -a equals: -rlptgoD (no -H,-A,-X) -u skip file that are newer on the receiver
I stopped the getmail cronjob to ensure that no new messages get fetched while sync. After that I created a new account in Icedove and at first all went well, but after a while I got duoplicates of abou 900 old messages. I admit I only looked that up from Icedove and not in the dir from cli.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Mon, 4 Feb 2013, Frank wrote:
Steffen Kaiser wrote:
On Fri, 1 Feb 2013, Frank Lienhard wrote:
My first attempt was tho simply rsync the Maildirs along with the homes. Both deovecot versions are set up with the maildir format, but this results in ~30% doublicated mails on the new server.
Er, a) you got 30% more files on the new server (aka target), than you have on the old server (aka source)??
b) Or do you see them, when you login via IMAP to the the new server?
===
If a), did you ran rsync, while the source host is in production still? Do you use "H" option to preserve hardlinks and/or "--delete" if you run rsync multiple times?
If b), did you verified the duplication with a IMAP client, that does _no_ caching on its own or with a fresh and empty local cache / profile / ... .
I used -avu, where: -a equals: -rlptgoD (no -H,-A,-X) -u skip file that are newer on the receiver
I stopped the getmail cronjob to ensure that no new messages get fetched while sync. After that I created a new account in Icedove and at first all went well, but after a while I got duoplicates of abou 900 old messages. I admit I only looked that up from Icedove and not in the dir from cli.
When you rsync'ed multiple times, you _must_ use "--delete" as well, because in Maildir status changes, such as "read", "seen", "replied", lables a.s.o., are reflected by changing the filename of the messages. Hence, without "--delete" you get the same message multiple times with different status or labels.
I tried it myself and had 1:1 copies of the mailboxes, regardless if I use the same or a fresh profile in Thunderbird.
I use: rsync -essh -vaH --delete from/ target:/.../to/
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux)
iQEVAwUBURIsEV3r2wJMiz2NAQLNQwgAwRiTbnrPlgmKIYUMwDVWo5tztighb8+b WoYjr4XkK5SkxJH1ZUnsaNzoKYC0j/Zf+PE/kK9C32X+itkIhH8f3L8jITCzj3RR kl6/j/OE3hnMDJNSfoYabmdpzc9knXabj04GVbZNW0cwXX33eU3CQd9yHbno3Raq DVsnCIe23L7sO7giNORUIpZJuhzfri8YpvAdZsypaK0ZfMXbBxYN6lPaoEPkYcKH T91EvjbeGKJGpAKokjtyCIrBbvYPW6ARyD/y7G7qrGFvUqqZTzhEUbEPuZ6kCkFS Zc5bmv90BcQ/O1U6U2jH72A3UeCsCRBl9BWp9Cvs5P5Pbi1kyzvZrA== =GtYM -----END PGP SIGNATURE-----
Am 06.02.2013 11:10, schrieb Steffen Kaiser:
On Mon, 4 Feb 2013, Frank wrote:
Steffen Kaiser wrote:
On Fri, 1 Feb 2013, Frank Lienhard wrote:
My first attempt was tho simply rsync the Maildirs along with the homes. Both deovecot versions are set up with the maildir format, but this results in ~30% doublicated mails on the new server.
Er, a) you got 30% more files on the new server (aka target), than you have on the old server (aka source)??
b) Or do you see them, when you login via IMAP to the the new server?
===
If a), did you ran rsync, while the source host is in production still? Do you use "H" option to preserve hardlinks and/or "--delete" if you run rsync multiple times?
If b), did you verified the duplication with a IMAP client, that does _no_ caching on its own or with a fresh and empty local cache / profile / ... .
I used -avu, where: -a equals: -rlptgoD (no -H,-A,-X) -u skip file that are newer on the receiver
I stopped the getmail cronjob to ensure that no new messages get fetched while sync. After that I created a new account in Icedove and at first all went well, but after a while I got duoplicates of abou 900 old messages. I admit I only looked that up from Icedove and not in the dir from cli.
When you rsync'ed multiple times, you _must_ use "--delete" as well, because in Maildir status changes, such as "read", "seen", "replied", lables a.s.o., are reflected by changing the filename of the messages. Hence, without "--delete" you get the same message multiple times with different status or labels.
I tried it myself and had 1:1 copies of the mailboxes, regardless if I use the same or a fresh profile in Thunderbird.
I use: rsync -essh -vaH --delete from/ target:/.../to/
generally you should use options to REALLY make a folder a 1:1 copy, the params below are used by me since many years for ANY sort of folder sync
--delete-after -tPrlHpogEAX
# -z compress # -t timestamps # -P progress # -r recursive # -l links # -H hard-links # -p permissions # -o owner # -g group # -E executability # -A acls # -X xtended attributes
participants (5)
-
Daniel Parthey
-
Frank
-
Frank Lienhard
-
Reindl Harald
-
Steffen Kaiser