[Dovecot] Courier->Dovecot Migration Issue

[Tony Rutherford]

Tony Rutherford wrote:
> For testing purposes, on a small set of users, we've run the
> courier-dovecot-migrate.pl script.   We are converting from Courier 0.43 to
> Dovecot 1.2.9.  For the most part, things seem to be ok.  However, an iPhone
> IMAP client user is reporting a problem where some of her messages don't
> show up.  Even stranger is the fact that these same messages do appear when
> using Outlook 2k7 for instance.  The common denominator for when the
> messages are missing seems to be whether or not the message appeared in both
> the Courier IMAP and Courier POP3 uid files.  The missing messages appear in
> the CourierIMAP file, but NOT the Courier POP file.  Here is a snippet from
> the resulting Dovecot-uidlist file:
>
>  
>
> 3 V1141844265 N12392
>
> 9338 P00cf2eb6_fadc_4992_ad3a_57cc83a16eab W5146
> :00cf2eb6_fadc_4992_ad3a_57cc83a16eab:2,S
>
> 9788 P0b18a2d6_97dd_4712_a09c_23c7a3d59467 W4055
> :0b18a2d6_97dd_4712_a09c_23c7a3d59467:2,S
>
> 9942 P0b3cfd0b_1fc7_47b3_805f_b369a3bb6c65 W4359
> :0b3cfd0b_1fc7_47b3_805f_b369a3bb6c65:2,S
>
> 10100 Pe178f1ed_d648_47e8_807a_3e2d655d7f84 W12028
> :e178f1ed_d648_47e8_807a_3e2d655d7f84:2,S
>
> 10101 Pe3f28a53_a4d4_4f3d_babe_5df72bc35b47 W13974
> :e3f28a53_a4d4_4f3d_babe_5df72bc35b47:2,S
>
> 10102 Pf3c3024b_2850_43ef_a397_d5cae5c8f537 W7177
> :f3c3024b_2850_43ef_a397_d5cae5c8f537:2,S
>
> 10104 P6e808a68_0a7b_4528_aa67_4cc16361c56b W10817
> :6e808a68_0a7b_4528_aa67_4cc16361c56b:2,S
>
> 10105 P65c8218d_7b2a_43b7_82a4_2743025bd954 W3579
> :65c8218d_7b2a_43b7_82a4_2743025bd954:2,S
>
> 10106 P963cc43e_0355_481e_99d5_5468708bc28f W6410
> :963cc43e_0355_481e_99d5_5468708bc28f:2,S
>
> 10107 P1405aefc_b98b_4a35_a7dd_a69199f724d0 W4703
> :1405aefc_b98b_4a35_a7dd_a69199f724d0:2,S
>
> 10108 P67908fad_ec0a_4786_b0f1_c7cfaa23d09a W127288
> :67908fad_ec0a_4786_b0f1_c7cfaa23d09a:2,S
>
> 10109 :9176f3b2_1927_4657_baa8_bfe4c39e89d4:2,S
>
> 10110 :79047185_aef7_48ec_aff2_6b71bd566acd:2,S
>
> 10111 :efe275a0_a7a8_4f47_b638_45a18bc0b108:2,S
>
>  
>
> The last three entries exist in the Courier IMAP file, but NOT the Courier
> POP file.  The resulting dovecot-uidlist format is different, but to be
> honest, I haven't been able to find the exact specification for the format
> of the dovecot-uidlist file (if it exists).
>
>  
>
> What is really strange to me is: If there was a problem with the migration
> script and resulting dovecot-uidlist file, why would these messages show up
> fine in some client (ex. Outlook) and not others (ex. iPhone).
>
>  
>
> Regards,
>
> Tony
>
>
>   
During the migration, it appears that the emphasis is placed on 
maintaining the Pop3 message sequence...at the cost of possibly changing 
IMAP UIDs.  The iPhone appears to go very strictly by the actual order 
of the IMAP uids...and since they're now out of sync...the messages 
appear to be out of order on the iPhone client.  I understand the 
importance of ensuring that pop clients don't redownload  messages...so 
tough to argue.

Here's a question though.  If the dovecot-uidlist file is deleted (for 
whatever reason), it gets rebuilt by Dovecot.  But, how does it get 
rebuilt?  It does not appear to generate uids based on the date of 
messages...I believe that to be true.  Is there any flag/option in 
Dovecot to build the uidlist file based on message date (uids ordered by 
date)?  The problem is that if they're not ordered by date, again, when 
the iPhone gets the first 200 messages, they may not necessarily be the 
latest 200 messages.  Also...this behaviour does not appear in other 
clients we've tested...TB and Outlook2007.

Tony