[Dovecot] batch convert mbox to maildir

[Todd Piket]

This weekend I migrated the university I work for to Dovecot and maildir 
from UW and mbox.  I made some pretty significant changes to mb2md such 
that it converts keywords (aka, custom flags) and builds the 
dovecot-uidlist file based on X-IMAP/X-IMAPbase and X-UID headers.  Both 
of these additions are optional though.  I was also tasked with 
converting not just /var/mail/%u, but also /home/%u and all mbox folders 
for all of users...~30,000 people.  The uidlist file it generates is for 
0.99.14.  I don't know if 1.0 has a different list syntax.  I also added 
the UW uidl patch so Dovecot generates uid numbers the same way UW did. 
  This prevented, most of, our users from noticing we even did anything 
(except for the 12 hour email downtime ;).

In order to convert as quickly as possible we created a wrapper around 
mb2md that forks mb2md processes for a list of users.  The list is in a 
Berkeley db file.  You can specify the maximum number of threads you 
want to use so you don't overload your system.  We ran our conversion on 
4 machines, converting 2 file systems (we have 8) per machine, with 32 
"threads" per master converter.  The backend is an EMC Celerra storage 
array that we read mboxes from and wrote new maildirs to.  We converted 
all /var/mail and user folders, ~13,000,000 (yes, million) 
files/messages in about 3.5 to 4 hours.

If you would like the source for this system I would be happy to provide 
it.  There is no documentation right now as it was a one time only 
thing, but I can try to explain.  If others are interested just let me 
know.  The whole thing is written in Perl and expects to be run as root. 
  It chowns everything in the final steps.

Regards,

  ------------------------------------------------------------
| Todd Piket                        | Email: todd at mtu.edu    |
| Programmer/Analyst                | Phone: (906) 487-1720  |
| Distributed Computing Services    |                        |
| Michigan Technological University |                        |
  ------------------------------------------------------------


Netlink Tech wrote:
> Hello,
> I have used mb2md for an individual user like this:
> 
> su ~username
> cd ~
> ./mb2md -m
> 
> This converted /var/spool/mail/username mbox to 
> /home/users/username/Maildir just as I expected.
> 
> To convert 100s of users the same way would be a chore (su as each user 
> for permissions, etc.).
> 
> Is there a way to batch process all /var/spool/mail/$USER to 
> /home/users/$USER/Maildir with the correct permissions/file owners for 
> each user?
>