[Dovecot] using dsync to convert mailboxes looses caching options

[Mark Zealey]

Hi there,

We're trying to convert users from Maildir to sdbox at present; I'm 
using dsync to achieve this (2.0.16) however when the user's have been 
converted we only get minimal information in the caching files. Is there 
some way to preserve all the caching decisions that were previously made 
so that when the user logs in to the new mailbox we don't have to cause 
an io storm rebuilding the cache that we know was good? Dovecot seems to 
be partially doing this - if i remove the logs/cache from the source 
mailbox no cache files are built in the conversion; if i put them back 
then we get a cache file built but it only contains a few bits of 
information (guid, date.save). Looking into this a bit further i find 
that when the caches are present at source the fields are preserved but 
the 'last used' date and caching decisions are not which I suspect means 
dsync doesn't bother caching on import - only fields with a yes decision 
in the source are copied (but their decision is only copied as a tmp 
with the date of import). For example:

Source idxview cache:

-- Cache fields --
  #  Name                                         Type Size Dec  Last used
  0: flags                                        bit     4 tmp  
2011-11-25 16:09
  1: date.received                                fix     4 yes  
2011-11-26 16:17
  2: size.virtual                                 fix     8 tmp  
2011-11-25 16:09
  3: imap.bodystructure                           str     - tmp  
2011-11-25 16:09
  4: mime.parts                                   var     - tmp  
2011-11-25 16:09
  5: hdr.IMPORTANCE                               hdr     - tmp  
2011-11-25 16:09
  6: hdr.X-PRIORITY                               hdr     - tmp  
2011-11-25 16:09
  7: hdr.CONTENT-TYPE                             hdr     - tmp  
2011-11-25 16:09
...
18: date.sent                                    fix     8 no   
1970-01-01 01:00
19: date.save                                    fix     4 yes  
2011-11-26 16:17
20: size.physical                                fix     8 no   
1970-01-01 01:00
21: imap.body                                    str     - no   
1970-01-01 01:00
... (24 total entries. stripped ones are just hdr. with a tmp decision)

dst cache:

  #  Name                                         Type Size Dec  Last used
  0: flags                                        bit     4 tmp  
1970-01-01 01:00
  1: date.sent                                    fix     8 no   
1970-01-01 01:00
  2: date.received                                fix     4 tmp  
1970-01-01 01:00
  3: date.save                                    fix     4 tmp  
2011-11-26 16:19
  4: size.virtual                                 fix     8 tmp  
1970-01-01 01:00
  5: size.physical                                fix     8 no   
1970-01-01 01:00
  6: imap.body                                    str     - no   
1970-01-01 01:00
  7: imap.bodystructure                           str     - tmp  
1970-01-01 01:00
  8: imap.envelope                                str     - no!  
1970-01-01 01:00
  9: pop3.uidl                                    str     - no   
1970-01-01 01:00
10: guid                                         str     - tmp  
2011-11-26 16:19
11: mime.parts                                   var     - tmp  
1970-01-01 01:00
12: hdr.IMPORTANCE                               hdr     - tmp  
1970-01-01 01:00
13: hdr.X-PRIORITY                               hdr     - tmp  
1970-01-01 01:00
...

and dst cached message for example:

RECORD: seq=57, uid=12207, flags=0x09 (Seen Answered)
  - ext 1 modseq    :          1 (0100000000000000)
  - ext 3 cache     :       5516 (8c150000)
  - cache offset=5516 size=92, prev_offset = 0
     - guid: 1321895282.XXXt,S=7399760,W=7496225
     - date.save: 1322324286 (3e11d14e)

src idx record for this message contains pretty much all fields.

Thanks,

Mark