[Dovecot] Loadable module

[Timo Sirainen]

On Tuesday, Jul 22, 2003, at 18:11 Europe/Helsinki, Maikel Verheijen 
wrote:

> But if I read your suggestions right, you propose a "fixed" change in 
> the maildir functions that call the quota update functions if 
> necessary. So perhaps we should make a quota_enabled = yes|no flag int 
> the config file, and use that? I would still be trying to make the 
> quota update functions for you, and try to put them in.

No, the core code wouldn't know anything about quota. The module would 
simply grab the calls to mailbox interface itself.

Actually I'd really like if the same quota code was mail storage 
independent. So if you accessed the messages only through Dovecot's 
APIs the same code would work with maildir, mbox and whatever future 
formats I come up with. But I'm not sure how good idea this is with 
current APIs .. you probably could do it but it could be done faster.

> > expunge(): nothing special, just update the quota file after expunges
> Expunge should recalculate the whole mailfolder?

Well .. it would be faster if it didn't have to. Problem is that you 
might not know the message's size always. Also there's some locking 
problems / race conditions if you try to calculate the new quota before 
actually expunging.

> > copy(): this is somewhat problematic, if there wasn't the
> > maildir hardlinks you wouldn't have to touch it at all..
>
> A copy can duplicate a message in the same store, but do hardlinks 
> work on nfs? If not, we should add the message to the quota list. And 
> maybe we should just add size of the email anyway, it won't be that 
> much of a loss to the user, and it would save us a lot of checking. :)

Hard links work with NFS. But the problem is adding the message to 
quota list. Normally copy() reads the messages and saves them using the 
normal save*() functions, so the quota checking would be done 
automatically. With hard links it doesn't go through save*(), so you'd 
have to do some special kludging if you wanted to account that copied 
message from quota too.

One possibility is to just go through the messageset, get the message 
sizes, call copy(), see if quota was accounted by save*() functions, if 
not do it yourself.

Another possibility is to fix the API by splitting all functions which 
handle multiple messages at once and replace them with iterator 
functions. ie. fetch*(), search*() and save*() are ok now, but 
expunge() and copy() are not. Something  like:

// initialize. very much like fetching messages.
struct mail_copy_context *(*copy_init)(struct mailbox *box,
	enum mail_fetch_field wanted_fields, const char *messageset, int 
uidset);
// copy next message. returns pointer to original message where you can 
find it's info
// including message's size
struct mail *(*copy_next)(struct mail_copy_context *ctx);

You'd call copy_init() with wanted_fields being MESSAGE_FETCH_SIZE (or 
actually I should probably add _REAL_SIZE which would tell how much 
actual disk space it takes rather than the size with virtual CRs). Then 
you'd call copy_next(), use some kludge to see if it called your 
save_next(), if not you get the message size with mail->get_size(mail) 
and add it to quota.

Expunging would work pretty much the same way:

struct mail_expunge_context *(*expunge_init)(struct mailbox *box,
	enum mail_fetch_field wanted_fields);
struct mail (*expunge_next)(struct mail_expunge_context *ctx);

Only difference would be that mail->get_size() might return you -1, ie. 
unknown. In that case you'd have to recalculate the quota for the 
mailbox.

Hmm. There's still a few small problems. And the _next() calls should 
probably be split to two, so you can first fetch the record, then 
decide if you really want to copy/expunge that message by calling 
_doit() or something. Umm. Actually then I might just as well use the 
normal fetch*() interface and just add a few new methods to struct mail:

int (*expunge)(struct mail *mail);
int (*copy)(struct mail *mail, struct mailbox *dest);

I think that'd work. :) I'll try to make these changes to CVS soon.

> And how about a user that has no quota file (yet), can we "recreate" 
> them? Maybe do a quota lookup for the user (database or file)? This 
> can be done using a module I guess, so it could be "user > configurable".

Depends on how you want the quota stuff to work. I'm not sure about 
that yet. Maildir++ quota seemed a bit slow. It would require 
rescanning _all_ the mailboxes once in a while. That's kind of 
pointless if only a few mailboxes have changed. Of course we could 
internally calculate the quotas per mailbox and combine them all to one 
Maildir++ compatible quota file.