- dovecot - dovecot.org

[Dovecot] New index / mailbox API
by Timo Sirainen 25 Nov '03

25 Nov '03

Today morning I had an idea how to make indexes more scalable with multiple concurrent writers. As a side effect it also made the locking issues much more simpler. So I thought I'd go and rewrite all the indexing code, it had gotten way too ugly and difficult to understand and maintain. I also thought to separate the index handling completely from mailbox handling. So there would be lib-index and lib-mailbox. lib-storage would then tie them together. Mailbox API still feels somewhat dirty though.. I didn't want it to depend on lib-index and I didn't want to duplicate any lib-index functionality in it to avoid uselessly allocating memory, so I can't reference to mails directly with their sequence number or UID. So I'm passing (void *id, size_t id_len) around - with maildir that would be the filename and with mbox it would be current offset in file. Suggestions welcome about both APIs. Some of the changes in indexes and it's API: - "modify log" -> "transaction log" which was the big change. All flag updates and expunges are first written to transaction log and only when committing it they will be moved into the main index. The transaction writes require the file locked for only very short time, so with lots of concurrent writers it means that several sessions could have written their wanted changes to log file, but they're all blocking on getting them moved into main index. When one session finally gets a lock in the main lock file, it writes everyone's changes into it. If lock can't be acquired in a second or so, the main index will be rewritten into temp file and rename()d over. This means that we can have tons of sessions reading and writing to same shared index file and no session can block others from reading or writing. Currently yoou could easily block changes for a very long time with eg. FETCH 1:* BODY.PEEK[] and reading it 1 bytes/sec. - mail_index will be a class interface, so it's implementation can be changed at runtime. In-memory indexing is going to be a totally different implementation this time with better memory usage. Could be useful for optimizing some special cases as well. - You'll get "views" into mailbox. A view will take care of keeping sequences synchronized according to what each client session thinks they are. Currently lib-storage does that in kind of ugly way. Hopefully this means that we can get rid of sequence parsing in lib-storage completely and move it into imap-only code. - You don't directly manipulate with locks and there's no rules of in which order you have to do something. You get views and transactions and use them. - You can have multiple views and transactions, so one opened index could be used to handle multiple client sessions. I'm not sure if that will ever be useful though :) - Error correction will be done automatically this time. Whenever some error is noticed, it will be fixed immediately and unless it changes syncing state, it doesn't force client to disconnect. Also it hopefully will be possible to change from disk-indexing to memory-indexing on the fly in case you run out of disk space. - I'm trying to get it NFS-safe.. - mail-cache that I recently rewrote doesn't change. The bad thing is that it seems to be broken in some way and probably needs rewriting anyway.. And I'm not sure if the problem is just in the code, or if the design itself is somehow broken with it. I'm fearing that memory doesn't somehow keep up with changes by other sessions, since we're not locking cache file for reading..

5 9

[Dovecot] 0.99.10.3 released
by Timo Sirainen 25 Nov '03

25 Nov '03

- SUBSCRIBE broke subscription lists - LIST code rewritten, children flags should be correct now - SORT and THREAD could have given invalid replies - Partial BODY[...] fetches might have returned wrong data or at least performed worse than was necessary Maybe Entourage was having problems with subscriptions?

3 6

[Dovecot] deliver mail with postfix to dovecot for mailinglists
by Sven Kirmess 25 Nov '03

25 Nov '03

This is a bit off topic on that list. But the question is required because of a limitation of dovecot. It cannot have shared mailboxes into which one would collect mailing list mails. I assume one has already solved that problem and therefore ask this question on that list. I try to do the following: I would like to setup an alias dovecot(a)kzone.ch and subscribe this alias to the dovecot mailing list. Then I would like to create an alias to run maildrop with the correct parameter to put it into the directory ~ski/Maildir/.dovecot/. Of course, there is no user dovecot on my system and if there would be one she couldn't write to a directory owned by me. Is there a trick to do the following: ski -> ~ski/Maildir dovecot -> ~ski/Maildir/.dovecot postfix -> ~ski/Maildir/.postfix Any other idea? Any good documentation on that? Sven

3 3

[Dovecot] Compiling dovecot
by Ted S. Garaventa 24 Nov '03

24 Nov '03

Still trying to compile dovecot to run on solaris8 x86. The program compiles without errors but does not run. The truss out put I get running the program is illegal system call by program: /usr/local/sbin/dovecot Any help is welcome.

4 3

[Dovecot] entourage + dovecot bug?
by Payton 24 Nov '03

24 Nov '03

I have dovecot set up to run IMAP services without SSL (I have tested many other configurations as well, but this reproduces the problem the simplest). When attempting to connect to the imap server using Entourage for OS X on a mac, I receive the following error: Mail could not be received at this time. The server returned the following error . Dovecot reports a successful login and nothing else in the mail log. All other email clients I have tried (mutt, Mail.app) work fine with the IMAP server. I have also tried this on multiple clients with the same results. All this leads me to believe a bug exists in Entourage, however: I also have a Red Hat 7.3 server running qmail and dovecot in nearly the same manner (except that it is using dovecot 0.99.10 and shadow for password authentication, instead of dovecot 0.99.10.2 and ldap password authentication). Entourage (and all other clients) works just fine with this setup. I'm not certain if the problem lies in some tweak between 0.99.10 and 0.99.10.2 or if it is something with the red hat system itself, but I can't find anything. I am using dovecot 0.99.10.2 on a Red Hat 9 system using qmail 1.04 (netqmail) and checking with the latest version (update 10.1.5) of Entourage on Mac OS 10.2.8. Thanks for any help! -Payton

6 8

[Dovecot] IMAP subscribe/unsubscribe broken in 0.99.10.2
by Scott Heavner 23 Nov '03

23 Nov '03

I saw this problem with 0.99.10.2 and had to revert back to 0.99.9.1. As I recall the subscription stuff was not respecting the end-of-line as a field separator. It would just cut the old text and drop the new in it's place. Or it might cut an inappropriate entry that should have been a partial string match. FolderOneOhNine FolderOne FolderThree Trying to delete FolderOne would probably [i didn't save any botched subscription files] result in this OhNine FolderOne FolderThree -scott

1 0

[Dovecot] 0.99.10.1 released
by Timo Sirainen 23 Nov '03

23 Nov '03

Since release candidate I added fix for CRLF + partial BODY[] fetching. I hope everything works :) I've been running it with mbox for a while and it seems to work fine. v0.99.10.1 2003-11-10 Timo Sirainen <tss(a)iki.fi> * mbox: \Draft and \Deleted flags used opposite flag chars in X-Status header. We were incompatible with other mbox accessing software. WARNING: Upgrading from previous version doesn't automatically swap the flags, so be careful not to accidentally expunge messages that had their \Draft flag changed to \Deleted. * Configuration file changes: - Whitespace at end of line is stripped, use quotes if you need it - # comments are supported after key=value lines. if you need '#' character, quote the value - Both " and ' quotes are supported. If you need to use them, '\' can be used for escaping. - mbox: COPY into same mailbox didn't work and could have corrupted the mailbox - Using Dovecot without index files would crash after using a while - Partial BODY[header] or BODY[part] fetches were buggy if client requested more data than was available in the header/part. - Partial BODY[...] fetches were buggy with messages that had CRLFs - Some BODY and BODYSTRUCTURE replies missed data for message/rfc822 MIME parts causing clients to break - SORT (SUBJECT) was buggy - Timezone fixes with Date-header

11 26

[Dovecot] Compiling Dovecot
by Ted S. Garaventa 21 Nov '03

21 Nov '03

Just found out about Dovecot and it seems to work great. My problem is that it compiles and runs on Redhat just fine. On the other hand it compiles on suse and solaris but will not work. When I start dovecot it silently stops running. No entries in the log or anything. Are there any specific instructions for compiling it on solaris x86 using gcc. Thanks

2 1

[Dovecot] SSL and certificate authorities.
by James Tyson 20 Nov '03

20 Nov '03

Hi all. Usually for in-house use and SSL I would just generate a self-signed certificate because most clients either ignore it or only ask the first time the account is configured. In terms of offering the service to our customers is there any value of getting someone like thawte or instantssl to sign a certificate for imaps/pop3s/smtp? Also, is there a configuration directive for dovecot to add the issuers ca bundle similar to apache's SSLCACertificateFile? And thanks for writing such a kick-arse imap server. It blows courier out of the water! --- James Tyson Director, Giant Robot Ltd http://www.giantrobot.co.nz/

4 6

[Dovecot] patch and bug report (from an other in fact)
by DINH Viet Hoa 20 Nov '03

20 Nov '03

When messages marked \Recent are deleted and expunged, the patch now sends the new message count and the new recent messages count (although the message count is not really necessary). >From RFC 3501 errata (ftp://ftp.cac.washington.edu/imap/rfc3501-errata) << Section 6.4.3, page 49: old: The EXPUNGE command permanently removes all messages that have the \Deleted flag set from the currently selected mailbox. Before returning an OK to the client, an untagged EXPUNGE response is sent for each message that is removed. new: The EXPUNGE command permanently removes all messages that have the \Deleted flag set from the currently selected mailbox. Before returning an OK to the client, an untagged EXPUNGE response is sent for each message that is removed. Note that if any messages with the \Recent flag set are expunged, an untagged RECENT response is sent after the untagged EXPUNGE(s) to update the client's count of RECENT messages. >> The bug is that when we connect to the IMAP server through a command (ssh /usr/libexec/imap for example). \Recent flags are not removed when we have finish the access to the mailbox. (I have not yet investigated about that) Besides that, did you found a solution for the message parser ? (the bug in FETCH ENVELOPE that I noticed personnaly to Timo) -- DINH V. Hoa, "s/^\(\(\\.\|[^\[]\|\[\(\^.\|[^^]\)[^]]*\]\)*\)\(\\[^\*[]\)/\1\\\4/" -- Stéphane CHAZELAS

2 1