Recommended tool for migrating IMAP servers

[Sebastian Arcus]

On 04/12/17 23:15, Steve Litt wrote:
> On Mon, 4 Dec 2017 12:53:15 -0800 (PST)
> Joseph Tam <jtam.home at gmail.com> wrote:
> 
>> "Davide Marchi" <danjde at msw.it> writes:
>>
>>>> UW-IMAP's mailutil, imapsync, YippieMove and Larch.
>>
>> Whatever you use, *don't* use UW-IMAP's mailutil unless you got lots
>> of time to kill.  It is dreadfully slow -- I used it to export some of
>> my users' mailboxes to Gmail or other remote mail servers, and I could
>> almost cut&paste the messages faster.
>>
>> Like Aki said, if you have the same mailbox format and FS access on
>> both sides, rsync is much simpler.  You can also try exporting the
>> old mailboxes via NFS, and with some artful symlinks, march through
>> your user mailboxes replacing the symlinks with the instantiated
>> local copies and have almost zero downtime.
>>
>> Joseph Tam <jtam.home at gmail.com>
> 
> Another possibility is to use an email client like Claws-Mail, which is
> very fast, create two accounts: One for the old IMAP, one for the new
> one, and just copy trees.

I've never used Claws-Mail - so I don't know how well it does. But I've 
done a lot of data migrations between imap servers and accounts - and on 
many occasions I had to use Outlook or Thunderbird to do it. It is 
extremely inefficient, specially if the user has a large number of 
subfolders - as neither software seem to be able to copy subfolders in 
block. The other problem I stumbled on is that you can initiate a large 
transfer (5000 emails at once), the connection, the client or even the 
server chokes at some point for various reasons - and more often than 
not it restarts the transfer from the beginning - and you end up with 
lots of duplications. Specially if transferring through a non-local, 
slower link (ADSL or something similar), I had to end up moving blocks 
of a few hundred emails at a time - then rinse and repeat. Not very 
happy memories when you are dealing with 30,000 emails mailboxes with 
100 subfolders!

To answer the OP, if you have full access to the destination server, and 
the two servers are not Maildir or mbox storage (so copying directly is 
not an option), another possibility might be to use getmail. I am about 
to test this in a few weeks time. Getmail has an option to download 
email from all folders of the originating account. I think I would 
script it with the following options:

1. Download 100 emails at a time - so that if something goes wrong it 
doesn't risk screwing up a large batch (getmail has this option).
2. Download emails from all folders of the account.
3. Delete emails from source after downloading - so that if things go 
wrong, or you have to interrupt the process, you know what has already 
been transferred.
4. Check for success exit code, and repeat until everything is copied.

I haven't tried the above yet, but I think it should be quite robust and 
efficient, and able to run unattended once started.