[Dovecot] Courier migrating issues: indexes, maildirsize, update query

[Timo Sirainen]

On Wed, 2007-05-23 at 12:57 +0200, Jan van den Berg wrote:
> I'm in the process of testing Dovecot to see whether it meets our needs
> to replace our current Courier setup which serves well over 100.000
> mailboxes (pop3 and imap: mysql with NFS)
> So far dovecot seems pretty straight forward; however I ran into a
> couple of things that I'm curious about.
> 
> 1. What's the deal with
> dovecot.index/dovecot.index.log/dovecot.index.cache. I understand
> Dovecot uses this for POP3 primarily

I guess you meant to say "for IMAP", which is correct.

> but this will no doubt cause a lot of overhead on our platform. 

Yes, they bring a bit too much extra overhead for standard download
+delete POP3 users. v1.1's indexes will work better with POP3 (and most
likely also with NFS).

> Also getting this to work with our NFS
> setup would be a pain (locking etc.). I've been following this
> discussion:
> http://www.dovecot.org/list/dovecot/2006-January/thread.html#10758; and
> it ends with C. Malony suggesting that an disable index option would be
> in place; but no further action is taken. Basically I want to get rid of
> the index files (courier also doesn't use any). Any suggestions; will
> there be a disable-index option?

Have you read http://wiki.dovecot.org/NFS?

You can anyway disable indexes with appending :INDEX=MEMORY to
mail_location setting, but it's probably not such a good idea. POP3
requires that the messages' virtual sizes are known. Dovecot initially
calculates by reading the whole message's contents and then storing the
size to dovecot.index.cache file. If you've disabled indexes, it means
that all the messages' contents are read at the beginning of each POP3
session.

Courier solves this by keeping the message sizes in courierpop3dsizelist
file. I'm thinking about doing something similar for Dovecot v1.1 also.
Alternative way would be to add the file's virtual size into the maildir
filename itself (see ,W=<vsize> in
http://wiki.dovecot.org/MailboxFormat/Maildir), but Dovecot doesn't add
them internally (but it does use them if they exist) and I'm not sure
how to configure other MDAs to do this.

> 2. One of the reasons we want to get rid of Courier is because the
> maildirsize is often incorrect. Whether this is because of a bad Courier
> implementation or NFS issues or whatever, I haven't figured out yet. But

Dovecot's maildirsize implementation works like Courier's, so it's
probable that it's just as broken with NFS:

	/* We rely on O_APPEND working in here. That isn't NFS-safe, but it
	   isn't necessarily that bad because the file is recreated once in
	   a while, and sooner if corruption causes calculations to go
	   over quota. This is also how Maildir++ spec specifies it should be
	   done.. */

> I'd like to test this function in Dovecot; but there seems to be very
> little documentation. I  read the link
> http://wiki.dovecot.org/Quota/Dict but this info is too scarce. Any
> pointers on where to go?

Dict quota doesn't work very well with multiple simultaneous connections
either with v1.0. This has already been fixed for upcoming v1.1 though.

So, I don't think Dovecot's POP3 implementation is going to help you in
any way. Some people are even using Dovecot IMAP + Courier POP3
combination for now.

> 3. In our MySQL setup we have a field with the latest poptime and
> pophost (IP). This info can come in handy when troubleshooting or
> filtering out inactive mailboxes. With our Courier setup this field gets
> updated by a rather elaborate script that checks the logs and runs
> updates queries. I could get this to work for Dovecot; but with Dovecot
> being more actively in development, I wonder could this be a feature;
> like after a user_query, another update query?

http://wiki.dovecot.org/PostLoginScripting at least works. I've thought
also about some post_login_query but I don't know if it's worth the
trouble.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://dovecot.org/pipermail/dovecot/attachments/20070523/a635aec5/attachment.bin