[Dovecot] Migration questions...

[Richard Hobbs]

Hello,

Timo Sirainen wrote:
> On Mon, 2009-05-11 at 15:05 +0000, Richard Hobbs wrote:
>> Hi All,
>>
>> We are soon to migrate our mail server from one piece of hardware to
>> another and we would like to take this opportunity to optimize things.
>> As a result, we would like to replace "uw-imapd" and "qpopper" with
>> "dovecot". The version we will be installing is 1.1.13-2, as this is
>> what is available through the latest Debian stable backports. We will
>> also be using exim to deliver mail (through dovecot's deliver mechanism,
>> of course).
>>
>> So... We are currently using the mbox format with uw-imapd, and would
>> like to migrate to the fastest solution possible with dovecot on the new
>> hardware. My understanding is that "multi-dbox" is not an option in this
>> version anyway, maildir is OK, but not great, and "single-dbox" is
>> therefore going to be the highest performing solution. Is single-dbox
>> the fastest way to go, considering we're going to be using email in the
>> following ways:
> 
> Single-dbox is the highest performing, but note that it's not as much
> tested as mbox and Maildir code. I think it should work ok, but I'm not
> aware of any larger installations using dbox currently. So in case you
> find a problem, you might have to upgrade/patch Dovecot to get it fixed
> and that would require compiling from sources.

In that case (and with a little further investigation which i've just
done) we've decided to go with maildir! That is still going to be
significantly better performing than mbox, right?

Also... do you know how uw-imapd & maildir compares to dovecot & maildir
in terms of performance? Does dovecot still use indices with maildir?

>> One problem that may arise is the fact that when we migrate, all msg
>> UIDs will be lost. If i'm not mistaken, this means that all emails will
>> be treated by the mail client as brand new, and if through IMAP, will
>> all go bold, and if through POP3, will all be downloaded again (if still
>> on the server) and therefore duplicated in the mail client. If this is
>> the case, is there anything we can do to stop this happening? Does the
>> "Convert" plugin does this job well?
> 
> mbox -> Maildir conversion can preserve both IMAP and POP3 UIDLs using
> an external script. Maildir -> dbox conversion can also preserve both,
> but that causes Dovecot to use this "hybrid Maildir-dbox format", which
> is slower than the full native dbox.

That'd good to know. Do you happen to know where I can get a copy of
this "external script" you speak of? Will it simply be included in the
debian package (probably)?

Also, given that i'm going to have to test this, i will obviously be
running the conversion on a copy of the live data, and then i'll have to
run the conversion again during the migration outage - will i need to
delete all the data and basically start again, or is it incremental?

>> 8. Configure the "convert" plugin to begin converting the mail to dbox
>> format.
>>
>> 9. Run something manually (if possible) to convert mailboxes before
>> people connect, so the task is already done by the time the outage is over.
> 
> There's convert-tool that you could use. I don't know if Debian packages
> it.

OK, fair enough... if this is a different script to the one you mention
above, could you pls let me know where i can get this too?

>> Basically, i suppose i'm asking, with the highest performing mailbox
>> option, is dovecot going to run faster with 2 individual arrays each
>> made from 2 disks, or a single 4-disk array (in which case we'd go
>> RAID
>> 0+1)?
> 
> My guess is that two RAID-1s would be faster, but I haven't really done
> any benchmarking. Anyway index files are 10-30% of the mailbox size, so
> the index-disks would be using a lot less disk space.

I assume you are talking about dovecot with maildir here, right?

Also, what would we put on each array? Are the inboxes still stored
separately to the IMAP folders when using dovecot and maildir?

Would it be best to put all data on one array, and the indices on the
other? We're basically after the fastest way to distribute the data! :-)

>> Also... would it be useful to turn off "atime" when we mount the
>> volume(s) or does dovecot rely on this?
> 
> Dovecot doesn't rely on atime updates, so turn them off.

Will do!

Thanks again - your help is greatly appreciated :-)

Richard.

-- 
Richard Hobbs (IT Specialist)
Toshiba Research Europe Ltd. - Cambridge Research Laboratory
Email: richard.hobbs at crl.toshiba.co.uk
Web: http://www.toshiba-europe.com/research/
Tel: +44 1223 436999        Mobile: +44 7811 803377
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3306 bytes
Desc: S/MIME Cryptographic Signature
Url : http://dovecot.org/pipermail/dovecot/attachments/20090512/7c57743a/attachment.bin