[Dovecot] Courier->dovecot migration script
HI All,
We have the latest Dovecot 1.1.6 running and I need to migrate some POP3 users over from Courier to Dovecot and would need to convert the courierpop3dsizelist to maintain the UIDs. I just need to confirm that the script (http://www.dovecot.org/tools/courier-dovecot-migrate.pl) only works for Dovecot v1.0 and not for v1.1? As the dovecout-uidlist file that is created (by the script) is not in the same format as dovecot-uidlist file that is created by the 1.1.6 server.
Thanks, Warren
On Wed, Nov 12, 2008 at 01:12:45PM +0200, Warren Baker wrote:
I just need to confirm that the script (http://www.dovecot.org/tools/courier-dovecot-migrate.pl) only works for Dovecot v1.0 and not for v1.1? As the dovecout-uidlist file that is created (by the script) is not in the same format as dovecot-uidlist file that is created by the 1.1.6 server.
The migration script generates the files using uidlist format "Version 1". While Dovecot 1.1.x normally uses "Version 3" format, it can also read the older formats (to maintain backward compability), and thus auto-convert it to the newest version.
FWIW, we have used version 1.1.7 of the script for our Couier to Dovecot 1.1.x migration, and it worked well, execpt for cases where Couier's uidlist was too old.
Ulrich
On Wed, 2008-11-12 at 13:12 +0200, Warren Baker wrote:
HI All,
We have the latest Dovecot 1.1.6 running and I need to migrate some POP3 users over from Courier to Dovecot and would need to convert the courierpop3dsizelist to maintain the UIDs. I just need to confirm that the script (http://www.dovecot.org/tools/courier-dovecot-migrate.pl) only works for Dovecot v1.0 and not for v1.1? As the dovecout-uidlist file that is created (by the script) is not in the same format as dovecot-uidlist file that is created by the 1.1.6 server.
Dovecot v1.1 can read v1.0 uidlists just fine, so the script still works. But it's possible that the script doesn't convert all POP3 UIDLs because it wasn't possible with v1.0 (it'll report how many it couldn't convert). Dovecot v1.1 supports storing any kind of POP3 UIDLs in the uidlist file, which would make it possible to create a migration script that preserved all the UIDLs correctly. If you or anyone else wants to try, the v1.1 format uidlist with POP3 UIDLs would look like:
3 V<uid-validity> N<next-uid> <uid> P<pop3 uidl> :<filename>
Where the first line is a header. So for example with 3 different kinds of Courier UIDLs:
3 V1196932607 N125574 1234 P1234 :1225386267.M230815P21206.host 1235 P1196932607-1235 :1225386268.M230815P21206.host 1236 P1225386269.M230815P21206.host :1225386269.M230815P21206.host
2008/11/12 Timo Sirainen <tss@iki.fi>:
Dovecot v1.1 can read v1.0 uidlists just fine, so the script still works. But it's possible that the script doesn't convert all POP3 UIDLs because it wasn't possible with v1.0 (it'll report how many it couldn't convert).
Answers my question why the MUA is downloading all of the mails again.
Dovecot v1.1 supports storing any kind of POP3 UIDLs in the uidlist file, which would make it possible to create a migration script that preserved all the UIDLs correctly. If you or anyone else wants to try, the v1.1 format uidlist with POP3 UIDLs would look like:
3 V<uid-validity> N<next-uid> <uid> P<pop3 uidl> :<filename>
Where the first line is a header. So for example with 3 different kinds of Courier UIDLs:
3 V1196932607 N125574 1234 P1234 :1225386267.M230815P21206.host 1235 P1196932607-1235 :1225386268.M230815P21206.host 1236 P1225386269.M230815P21206.host :1225386269.M230815P21206.host
I actually started adding this to your script already. As soon as I have a working copy I will send it on to you.
Thanks for the info.
Warren
On Nov 12, 2008, at 3:01 PM, Warren Baker wrote:
Where the first line is a header. So for example with 3 different
kinds of Courier UIDLs:3 V1196932607 N125574 1234 P1234 :1225386267.M230815P21206.host 1235 P1196932607-1235 :1225386268.M230815P21206.host 1236 P1225386269.M230815P21206.host :1225386269.M230815P21206.host
I actually started adding this to your script already. As soon as I have a working copy I will send it on to you.
Did you ever finish writing it?
2008/12/12 Timo Sirainen <tss@iki.fi>
On Nov 12, 2008, at 3:01 PM, Warren Baker wrote:
Where the first line is a header. So for example with 3 different kinds
of Courier UIDLs:
3 V1196932607 N125574 1234 P1234 :1225386267.M230815P21206.host 1235 P1196932607-1235 :1225386268.M230815P21206.host 1236 P1225386269.M230815P21206.host :1225386269.M230815P21206.host
I actually started adding this to your script already. As soon as I have a working copy I will send it on to you.
Did you ever finish writing it?
Sorry for the late reply, was on holiday. Yes although its been slightly customized for our environment and its a bit of a hack. I see John Lightsey version is way better than ours and that you have improved upon it. So wont bother sending ours.
Thanks, Warren
Hi Timo
I aplogise if this is documented somewhere (couldn't find it), but could you tell me what the "W<int>" field represents in a v3 dovecot-uidlist file? That is, when Dovecot v1.1.2 converts a v1 dovecot-uidlist file, I see the following (taking the first few lines):
V1 (generated by the courier-dovecot-migrate.pl script): 1 1208331473 30 2 1208332540.H167727P3133.mesg-dev2.mweb.co.za,S=343 3 1208332756.H543324P3161.mesg-dev2.mweb.co.za,S=340 4 1208332857.H236884P3191.mesg-dev2.mweb.co.za,S=339 5 1224664518.H150402P10074.mesg-dev2.mweb.co.za,S=342 6 1224664556.H128444P10080.mesg-dev2.mweb.co.za,S=339 7 1224666352.H992318P10361.mesg-dev2.mweb.co.za,S=339
V3 conversion: 3 V1208331473 N31 2 W353 :1208332540.H167727P3133.mesg-dev2.mweb.co.za,S=343:2,S 3 W350 :1208332756.H543324P3161.mesg-dev2.mweb.co.za,S=340:2, 4 W349 :1208332857.H236884P3191.mesg-dev2.mweb.co.za,S=339:2,S 5 W352 :1224664518.H150402P10074.mesg-dev2.mweb.co.za,S=342:2,S 6 W349 :1224664556.H128444P10080.mesg-dev2.mweb.co.za,S=339:2,S 7 W349 :1224666352.H992318P10361.mesg-dev2.mweb.co.za,S=339:2,
I notice that the W<int>'s are not unique (e.g. W349).
Thanking-you...
-----Original Message----- From: dovecot-bounces+pgamble=mweb.com@dovecot.org [mailto:dovecot-bounces+pgamble=mweb.com@dovecot.org] On Behalf Of Timo Sirainen Sent: 12 November 2008 02:43 PM To: Warren Baker Cc: dovecot@dovecot.org Subject: Re: [Dovecot] Courier->dovecot migration script
On Wed, 2008-11-12 at 13:12 +0200, Warren Baker wrote:
HI All,
We have the latest Dovecot 1.1.6 running and I need to migrate some POP3 users over from Courier to Dovecot and would need to convert the courierpop3dsizelist to maintain the UIDs. I just need to confirm that the script (http://www.dovecot.org/tools/courier-dovecot-migrate.pl) only works for Dovecot v1.0 and not for v1.1? As the dovecout-uidlist file that is created (by the script) is not in
the same format as dovecot-uidlist file that is created by the 1.1.6 server.
Dovecot v1.1 can read v1.0 uidlists just fine, so the script still works. But it's possible that the script doesn't convert all POP3 UIDLs because it wasn't possible with v1.0 (it'll report how many it couldn't convert). Dovecot v1.1 supports storing any kind of POP3 UIDLs in the uidlist file, which would make it possible to create a migration script that preserved all the UIDLs correctly. If you or anyone else wants to try, the v1.1 format uidlist with POP3 UIDLs would look like:
3 V<uid-validity> N<next-uid> <uid> P<pop3 uidl> :<filename>
Where the first line is a header. So for example with 3 different kinds of Courier UIDLs:
3 V1196932607 N125574 1234 P1234 :1225386267.M230815P21206.host 1235 P1196932607-1235 :1225386268.M230815P21206.host 1236 P1225386269.M230815P21206.host :1225386269.M230815P21206.host
New from MWEB: Cellphone and Internet bundles! Bundle your Internet access with your cellular contract from R75 per month. Call 08600 32000 or click here(http://www.mweb.co.za/productsservices/MTALKMobile/tabid/1223/Default.aspx) for more info on the great deals available.
MWEB :-) JUST LIKE THAT
This electronic communication and the attached file(s) are subject to a disclaimer which can be accessed on the following link: Disclaimer - or copy the following URL into your browser - http://www.mweb.co.za/disclaimer. If you are unable to view the disclaimer, please contact abuse@mweb.com for a copy.
W is the size of the file with CR/LF for newlines. It's the same size
courier stores in the POP3 uildlist file.
I wrote an updated version of the courier-dovecot migration script
that attempts to convert in both directions (using the V3 dovecot-
uidlist format) a while back.
It's freely licensed if you want to give it a try.
http://httpupdate.cpanel.net/cpanelsync/EDGE/bin/maildir-migrate
On Nov 12, 2008, at 11:59 PM, Paul Gamble - MWEB wrote:
Hi Timo
I aplogise if this is documented somewhere (couldn't find it), but
could you tell me what the "W<int>" field represents in a v3 dovecot-uidlist file? That is, when Dovecot v1.1.2 converts a v1 dovecot-uidlist
file, I see the following (taking the first few lines):V1 (generated by the courier-dovecot-migrate.pl script): 1 1208331473 30 2 1208332540.H167727P3133.mesg-dev2.mweb.co.za,S=343 3 1208332756.H543324P3161.mesg-dev2.mweb.co.za,S=340 4 1208332857.H236884P3191.mesg-dev2.mweb.co.za,S=339 5 1224664518.H150402P10074.mesg-dev2.mweb.co.za,S=342 6 1224664556.H128444P10080.mesg-dev2.mweb.co.za,S=339 7 1224666352.H992318P10361.mesg-dev2.mweb.co.za,S=339
V3 conversion: 3 V1208331473 N31 2 W353 :1208332540.H167727P3133.mesg-dev2.mweb.co.za,S=343:2,S 3 W350 :1208332756.H543324P3161.mesg-dev2.mweb.co.za,S=340:2, 4 W349 :1208332857.H236884P3191.mesg-dev2.mweb.co.za,S=339:2,S 5 W352 :1224664518.H150402P10074.mesg-dev2.mweb.co.za,S=342:2,S 6 W349 :1224664556.H128444P10080.mesg-dev2.mweb.co.za,S=339:2,S 7 W349 :1224666352.H992318P10361.mesg-dev2.mweb.co.za,S=339:2,
I notice that the W<int>'s are not unique (e.g. W349).
Thanking-you...
-----Original Message----- From: dovecot-bounces+pgamble=mweb.com@dovecot.org [mailto:dovecot-bounces+pgamble=mweb.com@dovecot.org] On Behalf Of
Timo Sirainen Sent: 12 November 2008 02:43 PM To: Warren Baker Cc: dovecot@dovecot.org Subject: Re: [Dovecot] Courier->dovecot migration scriptOn Wed, 2008-11-12 at 13:12 +0200, Warren Baker wrote:
HI All,
We have the latest Dovecot 1.1.6 running and I need to migrate some POP3 users over from Courier to Dovecot and would need to convert the courierpop3dsizelist to maintain the UIDs. I just need to confirm that the script (http://www.dovecot.org/tools/courier-dovecot-migrate.pl) only works for Dovecot v1.0 and not for v1.1? As the dovecout-uidlist file that is created (by the script) is not
inthe same format as dovecot-uidlist file that is created by the 1.1.6 server.
Dovecot v1.1 can read v1.0 uidlists just fine, so the script still works. But it's possible that the script doesn't convert all POP3
UIDLs because it wasn't possible with v1.0 (it'll report how many it
couldn't convert). Dovecot v1.1 supports storing any kind of POP3 UIDLs in the uidlist file, which would make it possible to create a migration
script that preserved all the UIDLs correctly. If you or anyone else wants to try, the v1.1 format uidlist with POP3 UIDLs would look like:3 V<uid-validity> N<next-uid> <uid> P<pop3 uidl> :<filename>
Where the first line is a header. So for example with 3 different
kinds of Courier UIDLs:3 V1196932607 N125574 1234 P1234 :1225386267.M230815P21206.host 1235 P1196932607-1235 :1225386268.M230815P21206.host 1236 P1225386269.M230815P21206.host :1225386269.M230815P21206.host
New from MWEB: Cellphone and Internet bundles! Bundle your Internet
access with your cellular contract from R75 per month. Call 08600 32000 or click here(http://www.mweb.co.za/productsservices/MTALKMobile/tabid/1223/Default.aspx ) for more info on the great deals available.MWEB :-) JUST LIKE THAT
This electronic communication and the attached file(s) are subject
to a disclaimer which can be accessed on the following link:
Disclaimer - or copy the following URL into your browser - http://www.mweb.co.za/disclaimer. If you are unable to view the disclaimer, please contact abuse@mweb.com for a copy.
On Thu, 2008-11-13 at 09:54 -0600, John Lightsey wrote:
W is the size of the file with CR/LF for newlines. It's the same size
courier stores in the POP3 uildlist file.I wrote an updated version of the courier-dovecot migration script
that attempts to convert in both directions (using the V3 dovecot- uidlist format) a while back.It's freely licensed if you want to give it a try.
http://httpupdate.cpanel.net/cpanelsync/EDGE/bin/maildir-migrate
I updated it to write POP3 UIDLs to the file as well: http://wiki.dovecot.org/Migration/Courier http://www.dovecot.org/tools/courier-dovecot-migrate.pl
I removed a lot of code related to tracking POP3/IMAP UID changes. That's no longer necessary.
participants (5)
-
John Lightsey
-
Paul Gamble - MWEB
-
Timo Sirainen
-
Ulrich Zehl
-
Warren Baker