On Aug 28, 2009, at 8:38 PM, Adam McDougall wrote:
Early next week I need to upload over 100,000 emails to an IMAP
server as quickly as possible from an Outlook client. I am looking
for any methods I can use to (temporarily?) speed up the rate at
which dovecot can accept and store IMAP uploads, whether it be
storing on local disk, ram disk, etc. I can setup a temporary
server on a laptop for example and once the upload has finished I
can use standard file copying methods to transfer the mail to
stable, permanent storage. I haven't been able to see over about 7
msgs/sec upload speed from a local folder in any mail client to
dovecot (only NFS or ZFS backend tested so far with Maildir). Is
there something horribly wrong with the speed I am seeing or are
there just tricks I can try? Any tips? I'll be working on it all
weekend until I find something satisfactory. It seems like I can
upload mails to an Exchange server quicker. I'll setup just about
anything that my experience allows me to, I can be very resourceful
with adhoc hardware and software.
From Dovecot's side the only thing you can do is fsync_disable=yes.
The main problem is probably network latency, because Outlook doesn't
support MULTIAPPEND extension (and perhaps not even LITERAL+
extension?) Did you already try running Dovecot on the same computer
as Outlook (some virtual thingy or maybe it works in cygwin)?
Alternatively I'll take a fast way of converting Exchange email to a
tree of local mbox files which I can then run mb2md on.
If the mails are in Exchange, can't you connect to it using IMAP?