[dovecot] Re: Architectural questions

[Timo Sirainen]

On Sat, 2002-10-19 at 18:10, Thomas Wouters wrote:
> Well, if I recall correctly, we added an 'X-Offset' header which pointed to
> the exact (relative) byte offset for the next 'From ' line. It made our
> pop3d (a modified qpopper 2.3 by the way) a much happier puppy. I'm not sure
> what the difference with Content-Length was. I could find the sources, I
> suppose; since we disabled mbox-inbox support we aren't using that code
> anymore.

Content-Length saves just the size of mail body, so it can be skipped
over. I implemented it mostly because mutt doesn't escape the "From "
lines when saving mails so it was a bit difficult sometimes to figure
out if the From-line means a new mail or if it was just written into the
mail body.

I'm not sure what I should do with Dovecot, both From-line escaping and
Content-Length writing makes it annoyingly slower then now with more 
code..

> > Also it would be possible not to use index files at all but just keep
> > them in memory. I've been fixing code to make this possible and somewhat
> > fast.
> 
> Hmm. I'd have to look at the code to say for sure, but I think we could live
> with keeping them in memory. Accessing the same mailbox from two different
> clients at the same time is not something we're too worried about, at the
> moment.

Well, Outlook (and OE I think) opens two simultaneous connections
sometimes to fetch mails.

Not having index files doesn't affect the possibility to have multiple
connections, but it affects the overall performance because it needs to
do more I/O, especially with mbox.

> > I've also thought I might as well make it possible to read the mbox
> > inbox from /var/mail or whereever it is. Pretty easy to do, but .lock
> > file is problematic if new files can't be added to the /var/mail
> > directory.
> 
> Our /var/spool/mail subdirectories are mode 01733 (drwx-wx-wt) owned by
> root, so creating files and removing them is not an issue, but reading the
> directory is. You can of course still check for existance of specific
> filenames.

OK, no problem then.

> > How exactly does the U flag work? I hope it's before the ':' character
> > like Courier's S=filesize? Otherwise U=1234 would be thought of as 6
> > different flags which isn't very good since Dovecot reorders them as
> > 1234=U.
> 
> No, it can't be before the :, because the UID is generated by UW-IMAP, and
> the maildir spec says you can't change the uniqe part of the name, just the
> info :) Here are some examples. The ',U*' is the UID.
> 
> _k2,6NtZ9.maildrop4.xs4all.nl:2,S,U1030712092
> _fmT,O63l8.maildrop8.xs4all.nl:2,RS,U1026644784
> 990612135.16312.000000002.maildrop2.xs4all.nl:2,S,U991994304
> 993058841.maildrop7.49267:2,S,U993058888

Well, maildir spec also doesn't say you can add flags with parameters
using comma separators :) But I think that's good enough extension that
Dovecot could support as well.

Programs supporting Courier's Maildir++ quota writes the mails
immediately like "something,S=size". Something like that could have been
done by UID-capable mailers too, since UID won't change.

> As long as dovecot doesn't read a different meaning into those flags
> (ignoring them is just fine) we should be fine. I don't think we'll have
> many customers switching back and forth between dovecot and UW-IMAP, just
> people switching from UW-IMAP to dovecot.

Keeping the UIDs untouched when changing could be important to some
people whose mail clients can save some extra information related to
specific messages and use UID to identify the mails. I think Evolution
does this with it's labels and Follow-Up marks, but I'm not sure.

Dovecot currently doesn't try to keep the UIDs too heavily itself
either, if it notices some corruption it just recreates the index files
with new UIDs. Supporting in-memory indexes requires still saving the
UIDs somewhere in disk so this should get fixed while doing it.

> I'm not too worried about bugs. I've yet to see a piece of software that we
> don't find oodles of small and large bugs in just by installing and trying
> to run on our clientbase. That's what testing is for :) But I wouldn't mind
> being happily suprised by dovecot, we'll see :)

We've had 3 people using it for a few months now, one of them still gets
sometimes "message not found" error from Outlook Express, I've yet to
figure out when exactly that happens. The mail isn't lost and restarting
OE helps, so it's probably something to do with having those two
simultaneous connections (and I'm just now making bigger changes there).
Other than that it's worked quite fine :)

Then of course CVS has a lot large changes which haven't been tested
much yet. Hopefully I'll get them fixed well enough this weekend to be
able to start using it myself.