July 2003 - dovecot - dovecot.org

[Dovecot] Why does dovecot confuse mutt?
by Amelia A Lewis 04 Oct '03

04 Oct '03

Heylas, I just switched my primary mail server from courier imap to dovecot (I've been waiting to do this; dovecot showed up in debian testing for alpha, at last). So, using the same set of maildirs, there are slightly different behaviors. First, of course, dovecot displays the hierarchy as I thought it ought to be, such that INBOX is a sibling of all other top-level boxes, not the parent of everything. This is A Good Thing[tm]. That works as expected. However, as far as mutt is concerned, *every* mailbox has child folders. My muttrc contains the line: set folder=imap://localhost/. (and the home folder is set to imap://localhost/INBOX; the idea is to do all mail munging through the server, never directly touching the maildirs with an MUA). Oh, and I'm using maildir (~/Maildir). In mutt, from the currently-selected mailbox, one can use 'c' to change boxes, and ? displays a navigable folder hierarchy. If a folder contains only mail (no child folders), it shows up as "FolderName", if it has child folders (whether or not it has mail), it shows up as "FolderName.". You can display the mail in a folder that has child folders, but it requires a different keystroke than the default (enter selects a folder, which means showing child folders if there are any, showing mail if there aren't; to display mail in a folder that has child folders, space). This is tripping me up a little, because of course the commands are mostly bred into my fingertips, and it's a difficult adjustment. I hit return, and I see a folder that contains the "child" folder "../". Is this a glitch in mutt's handling of information that I ought to report to mutt maintainers? Is it a glitch in dovecot's presentation of information that I ought to report to ... err, well. *laugh* My other primary mail client seems to cope beautifully with the information that's supplied (sylpheed, if it matters). But then, sylpheed also coped perfectly when courier was supplying information. I gather that courier and dovecot are reporting something slightly different, somehow, but I don't really know what, and I don't know which should be considered "correct". Amy!

5 5

[Dovecot] user database clarification
by Jared 22 Aug '03

22 Aug '03

Hi, this question gets back to a thread I started back on July 8th. (gosh how time flies) I still am not clear on why PAM is not supported for the user database. I'm in an environment that has users both in local flat files (/etc/passwd and /etc/shadow) AND in an LDAP database. Is it possible for this set up to work with dovecot? (both local and ldap users have imap/pop?) I'd prefer to just have Dovecot use PAM for user db and the password database so that I can configure PAM to try both local users and LDAP users. Thanks a lot. -jared

3 4

[Dovecot] corrupt mbox, mailboxes not found, and message read status
by Jared 04 Aug '03

04 Aug '03

I'm running dovecot 0.99.10-0.rc2 deb packages for Debian Woody from braincells.com. My old environment was a server running UW-IMAP for IMAP and POP services. We are still using mbox for all mailboxes. I use IMP for web based access. Forgive me if these issues have already been discussed or fixed in cvs. Any additional pointers to this info. would be appreciated. I have noticed a few issues. ---- I've seen a whole slew of entries like the following: pop3(username): Error indexing mbox file /home/username/mail/inbox: LF not found where expected According to this mailing list post from June 16, 2003: http://dovecot.procontrol.fi/list/dovecot/2003-June/001668.html it may be because of adjacent From mail headers in a message. I've had one user that this was not the case. They said they were deleting lots of messages in IMP and when he went back to his inbox, it was empty. When I looked at the mbox file, a message had been truncated. When I deleted the imap index and removed that truncated message, the mailbox became visible again. Has the adjacent From issue been fixed? Is there a way to find out more specifically what part of the mbox file dovecot is unhappy about? ie. a line number. ---- Inside IMP several users have set up a filter to send messages marked as spam by SpamAssassin into a spam folder. When they log-in or click the filter icon, the filter runs. However, now when the the filter runs, it reports: "Problem with filter rule 1: folder does not exist: mail/spam". However, the spam folder is still there, and when I open that folder in IMP, it opens just fine. I've already changed IMP's folder prefix from 'mail/' to ''. I've also tried deleting the imap index. The message still shows up and the filter fails to run. ---- With the old set-up, when I check mail with a pop3 client, the UW pop server modifies the Status and/or X-Keywords headers for the message. When the mailbox is viewed with an IMAP client such as IMP, messages that have already been downloaded via pop are marked as read. This doesn't seem to happen with dovecot. I found a Oct 19 2003 thread ( http://dovecot.procontrol.fi/list/dovecot/2002-October/000774.html ) on the mailing list where Timo said he was intending to use the X-Keywords trick. Did this ever get implemented? ---- Thanks, -jared

5 6

RE: [Dovecot] corrupt mbox, mailboxes not found, and message rea d status
by Maikel Verheijen 31 Jul '03

31 Jul '03

[SNIP] > http://dovecot.procontrol.fi/list/dovecot/2002> -October/000774.html ) > > on the mailing list where Timo said he was intending to use the > > X-Keywords trick. Did this ever get implemented? > > X-Keywords have been used for a long time now. It's possible > to set \Seen flag for messages downloaded with POP3, but I > don't know if it's worth the trouble. Who uses POP3 and IMAP > mixed anyway? > A lot of our customers, since we only supported pop3 in the past, and our new webmail uses imap. :) And a lot of people don't click on "This is an imap server" when creating their account in for instance outlook-express. Kind regards, Maikel Verheijen.

3 2

[Dovecot] New index file code
by Timo Sirainen 31 Jul '03

31 Jul '03

I just had to write this even while it's not just yet in CVS, but it's only a few fixes away from being committed.. My year long dream has finally came true :) [Summary: Next release will have REALLY kickass indexes] Previously the biggest problem with caching message data in indexes was that we had to do it while syncing the mailbox, before client even saw those messages. This caused index rebuilds to be very slow since it had to go through all mails and cache all of them, even if client never would access them. Alternative was to not cache anything at all, but that wasn't really good idea either. Now Dovecot caches the data when client is actually requesting it. That way it will never do any extra work for caching something that client isn't currently interested in. There's still the anticipatory caching for data that client is likely to be interested in later, but that is done only when it doesn't cause extra disk I/O. The reason why it took this long was because it was previously too difficult with read/write locking that indexes required. I couldn't have relied on it working since changing from read lock -> write lock could deadlock and dropping the lock in the middle would have caused even more problems. So, what I did was to rewrite the cache file handling completely: - It doesn't require any locking to read from it. - It will be NFS-safe without being too costly. - The file format itself is now architecture-independent. - Uses less space than before - OpenBSD support is now worse than before, even worse than NFS-support. They should get that unified cache done. Cached data is also selected better: - Specifically requested message headers are cached (FETCH, SEARCH, SORT, THREAD, etc.) - IMAP ENVELOPE isn't treated in any special way anymore. It's treated just as if you had requested HEADER.FIELDS (Date Subject From ...etc.). - Maildir filenames aren't constantly updated in the cache file anymore. This takes more memory now, but should reduce disk I/O. There's a few more tweaks that I'll probably add later: - Maildir: When compressing cache file we update mail's filename to current one so if message's flags aren't changed often, it won't use any extra memory either. - A normal text/plain message doesn't need to have it's IMAP BODYSTRUCTURE cached. It only needs a single bit set in flags. Should reduce used space quite a lot. - Message with only one body part doesn't need to have it's body structure stored at all since it's mostly just useful for fetching body parts. Plus the body structure could be generated if message sizes were known. All this work was only for index cache file (ex. .data file). There's still the main .imap.index file and modify log files. Modify log should be easy to make NFS-safe - I'm currently using some stupid file locking to figure out when the file is safe to overwrite, but I should have simply replaced the old file with rename(). That will solve some other problems as well as make the code simpler. It should be possible to be made read-lockless as well. The main index file is more problematic though. It contains a lot of changing fields in the header, such as number of messages, number of seen messages, etc. I can't think of any way to make it safe to read these fields without locks (my previous ideas didn't actually work). Lockless reads are pretty much a must for scalable NFS-safety. I think it could be done by simply removing all the constantly changing headers. If you want to know how many seen messages there are, just read all the records in the file and count them. Expunging would be done by rewriting the file and rename()ing it over the old file. Appending new messages is a bit tricky to do safely, but I think I know how to do that too.. read/write locks allow the changing headers, but there's locking contention problems with shared mailboxes.. I think I'll make it optional how to do this. Oh, and I also thought how indexes would work with shared mailboxes. You could use one shared cache file, but each user would have their own main index and modify log. That would allow storing per-user flags in the index file and also expunge (hide) messages by removing them from user's index.

1 0

[Dovecot] Is dovecot NFS safe?
by Adam Shand 29 Jul '03

29 Jul '03

I'm migrating our mailserver from cyrus 2.0 to Postfix and Dovecot. We're using maildirs delivered to /var/maildir, which is mounted via NFS from a Netapp. My understanding was that maildir was completely happy in an NFS environment but I've seen a couple messages here recently which mention the dovecot index files not being NFS safe. Are their any caveats or gotcha's that I need to be aware of? Thanks, Adam.

4 5

[Dovecot] Namespaces
by Timo Sirainen 27 Jul '03

27 Jul '03

0.99.11-test4 release: http://dovecot.procontrol.fi/test/ Finally namespaces somewhat working. Example: namespace private { separator = / prefix = INBOX/ location = maildir:~/Maildir } namespace private { separator = / prefix = #mbox/ location = mbox:~/mail } Problems: - Mailbox listing lists only the first namespace unless you clearly ask mailboxes under some namespace. Not really a problem, but related to: - There should be only one common subscriptions file for all namespaces. This needs some thinking how to do it well. - Having multiple different mailbox formats in one namespace still isn't possible. Needs some more thinking as well.

1 0

[Dovecot] API changes and plugins
by Timo Sirainen 26 Jul '03

26 Jul '03

There's been quite a lot of changes in CVS lately, mostly related to API changes but also some index file changes. They've both had very little testing, so there's probably a few bugs left. Especially mbox users should be careful. pop3 might also be deleting wrong messages.. Anyway. The new mail storage API should allow plugins to hook into pretty much any operation they want to. It's not exactly pretty with C syntax, but I think conceptually it's nice object oriented design :) There's example (non-working) quota plugin at: http://dovecot.procontrol.fi/patches/quota.c I also built 0.99.11-test3 from current CVS: http://dovecot.procontrol.fi/test/

1 0

[Dovecot] Dovecot on Mac OS X?
by jennyw 25 Jul '03

25 Jul '03

Just wondering if anyone has successfully gotten Dovecot to run on Mac OS X? If so, I'd love to hear what your experience has been! Thanks! Jen

2 1

FW: [Dovecot] Is dovecot NFS safe?
by Maikel Verheijen 25 Jul '03

25 Jul '03

This was ment for the list :) Sorry Stuart, for the double email... Regards, Maikel. > -----Original Message----- > From: Maikel Verheijen > Sent: Friday, July 25, 2003 3:40 PM > To: 'Stuart Henderson' > Subject: RE: [Dovecot] Is dovecot NFS safe? > > > Hi, > > Be aware that also your .subscriptions file and .customflags > are not nfs safe (yet) in dovecot-0.99.10. > > I know Timo is working on that though :) > > Kind regards, > > Maikel Verheijen. > > Ps: I have a "crude" patch to make .subscriptions and > .customflags nfs aware (I can not guarentee it is nfs safe ;-). > My own setup runs on nfs with the indexes in memory. > > > -----Original Message----- > > From: Stuart Henderson [mailto:stu@spacehopper.org] > > Sent: Friday, July 25, 2003 2:13 PM > > To: Adam Shand; dovecot(a)procontrol.fi > > Subject: Re: [Dovecot] Is dovecot NFS safe? > > > > > > > I'm migrating our mailserver from cyrus 2.0 to Postfix > and Dovecot. > > > > > > We're using maildirs delivered to /var/maildir, which is > > mounted via > > > NFS from a Netapp. My understanding was that maildir was > > completely > > > happy in an NFS environment but I've seen a couple messages here > > > recently which mention the dovecot index files not being NFS safe. > > > > > > Are their any caveats or gotcha's that I need to be aware of? > > > > You'll probably want to keep them in memory or on local disk. See > > <http://dovecot.procontrol.fi/doc/nfs.txt> for more information - > > default_mail_env needs either ":INDEX=/local/path/to/indexes/%u" or > > ":INDEX=MEMORY", all machines accessing the mailstore should have > > sync'd clocks, and all imap servers need separate hostnames. > > > > --Stu > > >

1 0