[Dovecot] dovecot evaluation on a 30 gb mailbox

[Ed W]

On 23/06/2010 12:22, Rajesh M wrote:
> did the tests once again on this large email box
> i set
> maildir_very_dirty_syncs = yes
> this was done in the morning and dovecot was restarted
>
> i logged after about 4 hours after the previous login
> again it took around 5 mins to login
>
> i was monitoring my server load which around 1.5 - 2 on my dual core dual
> xeon machine
>
> this increased to around 8-9 during the login process
>
> the no of emails in the box had increase by around 7500 emails
>    


When you ask Dovecot to open a mailbox I believe it builds an index of 
the key fields in all messages in the inbox.  This means it needs to 
scan the list of all files in the mailbox to see if any new files are in 
the directory (so login to the server and time "time ls -al" just before 
using dovecot - this is a measure of the time the OS needs just to list 
all the files in the mailbox (I presume maildir, not mbox?)

Secondly 7,500 mails over 5 mins means an indexing rate of 25 
mails/sec.  This would not be out of order for a heavily fragmented 
drive which is IO bound?  Each file needs to be opened to scan the 
headers so likely you need one disk seek and I guess it's easy to be IO 
bound?  What does iotop show you during dovecot's thrashing?

Dovecot2 has an mdbox option which sounds like it could be beneficial 
for your performance requirements (but it's not "stable" yet)

Otherwise I guess you need to investigate dovecot's delivery agent which 
does incremental index updates at deliver time (pay the cost one email 
at a time rather than every 7,500 emails).  Or consider alternative 
filesystems which are more performant for this requirement?

Good luck!

Ed W