Early next week I need to upload over 100,000 emails to an IMAP server as quickly as possible from an Outlook client. I am looking for any methods I can use to (temporarily?) speed up the rate at which dovecot can accept and store IMAP uploads, whether it be storing on local disk, ram disk, etc. I can setup a temporary server on a laptop for example and once the upload has finished I can use standard file copying methods to transfer the mail to stable, permanent storage. I haven't been able to see over about 7 msgs/sec upload speed from a local folder in any mail client to dovecot (only NFS or ZFS backend tested so far with Maildir). Is there something horribly wrong with the speed I am seeing or are there just tricks I can try? Any tips? I'll be working on it all weekend until I find something satisfactory. It seems like I can upload mails to an Exchange server quicker. I'll setup just about anything that my experience allows me to, I can be very resourceful with adhoc hardware and software.
Alternatively I'll take a fast way of converting Exchange email to a tree of local mbox files which I can then run mb2md on. I tried using Thunderbird to Import the mails from Outlook and while it was fast, it messed up the formatting of some of the mails so I don't think I can use that. I tried readpst briefly from libpst but it took a long time to run, took alot of cpu, and was spewing lots of errors so I canceled it.
Thanks for any input!