[Dovecot] [PATCH] Pop3 order in courier migration script

[Christoph Bußenius]

Hi,

I found a problem in the courier conversion script 
(courier-dovecot-migrate.pl).  In some cases, it does not correctly 
preserve the order of POP3 UIDLs.

Consider the following case, when courierpop3dsizelist contains:

/2 9000 1327481629
AAA1:2,S 10000 21:1327481629
AAA2:2,S 10000 22:1327481629
AAA3:2,S 10000 23:1327481629
AAA4:2,S 10000 24:1327481629
AAA5:2,S 10000 25:1327481629
BBB11:2,S 10000 11:1327481629
BBB12:2,S 10000 12:1327481629
BBB13:2,S 10000 13:1327481629
BBB14:2,S 10000 14:1327481629
BBB15:2,S 10000 15:1327481629

and courierimapuiddb contains:

1 1327480919 9000
11 BBB11
12 BBB12
13 BBB13
14 BBB14
15 BBB15
16 CCC16
17 CCC17
18 CCC18
19 CCC19
20 CCC20
21 AAA1
22 AAA2
23 AAA3
24 AAA4
25 AAA5

then call
courier-dovecot-migrate.pl --to-dovecot --convert
and it will create dovecot-uidlist:

3 V1327480919 N9010
21 PUID21-1327481629 W10000 :AAA1:2,S
22 PUID22-1327481629 W10000 :AAA2:2,S
23 PUID23-1327481629 W10000 :AAA3:2,S
24 PUID24-1327481629 W10000 :AAA4:2,S
25 PUID25-1327481629 W10000 :AAA5:2,S
9000 :CCC17
9001 PUID12-1327481629 W10000 :BBB12:2,S
9002 :CCC16
9003 PUID15-1327481629 W10000 :BBB15:2,S
9004 :CCC20
9005 PUID14-1327481629 W10000 :BBB14:2,S
9006 PUID13-1327481629 W10000 :BBB13:2,S
9007 :CCC18
9008 PUID11-1327481629 W10000 :BBB11:2,S
9009 :CCC19

The problem here is that the BBB files are not in the same order as in 
courierpop3dsizelist, and will therefore have a different order in the 
UIDL response in POP3.  Also, the CCC files are interleaved while they 
should appear at the end.

The reason is how the script assigns new UIDs:  It does not do this in 
the order of courierpop3dsizelist.  In fact the order is random because 
it depends on how perl traverses hashes.

The attached patch fixes this by sorting the messages: First the POP3 
messages get their new UIDs in order of their appearance in 
courierpop3dsizelist (so that the UIDL order will really be the same as 
in Courier), after that, the IMAP-only messages will be checked for UID 
clashes.  The patch also changes how the $prev_uid variable is handled 
because that was also needed to always get the order right.

The algorithm could probably be optimized to reduce the number of UID 
re-assignments to a minimum, however this would complicate the code and 
would only be noticed in a few cases.

The patched courier-dovecot-migrate.pl will create dovecot-uidlist like 
this:

3 V1327480919 N9010
21 PUID21-1327481629 W10000 :AAA1:2,S
22 PUID22-1327481629 W10000 :AAA2:2,S
23 PUID23-1327481629 W10000 :AAA3:2,S
24 PUID24-1327481629 W10000 :AAA4:2,S
25 PUID25-1327481629 W10000 :AAA5:2,S
9000 PUID11-1327481629 W10000 :BBB11:2,S
9001 PUID12-1327481629 W10000 :BBB12:2,S
9002 PUID13-1327481629 W10000 :BBB13:2,S
9003 PUID14-1327481629 W10000 :BBB14:2,S
9004 PUID15-1327481629 W10000 :BBB15:2,S
9005 :CCC16
9006 :CCC17
9007 :CCC18
9008 :CCC19
9009 :CCC20

Cheers,
Christoph

-- 
Christoph Bußenius
Rechnerbetriebsgruppe der Fakultäten Informatik und Mathematik
Technische Universität München
+49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching

-------------- next part --------------
A non-text attachment was scrubbed...
Name: courier-dovecot-migrate.patch
Type: text/x-patch
Size: 1386 bytes
Desc: not available
URL: <http://dovecot.org/pipermail/dovecot/attachments/20120227/3d6d2756/attachment-0002.bin>