July 2004 - dovecot - dovecot.org

Re: [Dovecot] [PATCH, RFC] add APOP authentication mechanism
by Timo Sirainen 06 Jul '04

06 Jul '04

On 1.7.2004, at 16:26, Andrey Panin wrote: >> + if (strcmp(auth_request->protocol, "POP3")) { >> + if (verbose) >> + i_info("apop(%s): wrong protocol %s", >> get_log_prefix(auth_request), >> + auth_request->protocol); >> + mech_auth_finish(auth_request, NULL, 0, FALSE); >> + return TRUE; >> + } >> >> I don't think there's really any point in checking this? Maybe someone >> really wants to use APOP with another protocol :) > > This will be RFC violation isn't it ? (I'll send a public reply, the last question could need input from others) Well, it's not compatible with any RFC of course, but I don't think it would violate anything? I'd just rather like to avoid unnecessary code bloat. Well, except now that I thought about the logic instead of just the code, there are problems. APOP prevents replay attacks by sending different timestamp in greeting, but you're sending the greeting to dovecot-auth as user input to APOP authentication mechanism. So, a user could simply say "AUTH APOP" and send wanted timestamp as input. One way to fix this would be to disallow APOP mechanism to be used with AUTH command in pop3-login and require that the request was done with POP3 protocol. But I think that's pretty ugly kludge. It also requires that processes connecting to dovecot-auth are trusted, which is something I've tried to avoid as much as possible. So dovecot-auth doesn't really know if it's pop3-login which connects to it.. Better fix would be to get part of the timestamp from dovecot-auth itself. It could send a growing ID number as handshake which would be used as part of the timestamp, and APOP mechanism processing would check that timestamp really contains the ID. So timestamp would be <authid.unixtime@hostname>. That also makes the PID go away, which was another thing I would have suggested. POP3 RFC also says: It is conjectured that use of the APOP command provides origin identification and replay protection for a POP3 session. Accordingly, a POP3 server which implements both the PASS and APOP commands should not allow both methods of access for a given user; that is, for a given mailbox name, either the USER/PASS command sequence or the APOP command is allowed, but not both. But I don't know if there's any nice way to do it, except perhaps by defining APOP password scheme which would actually be plaintext but wouldn't work with anything but APOP authentication. Do any other servers prevent it either?

2 5

[Dovecot] Debian dovecot news
by Jaldhar H. Vyas 04 Jul '04

04 Jul '04

I uploaded 0.99.10.6 to sid. Owing to the importance of the dot lock fix, I've tried to have this pushed to sarge quickly. Let's see how that goes. I've prepared a backport of 0.99.10.6 to woody which should be showing up on www.backports.org soon hopefully. In the meantime, if you desperately need it and don't mind working out dependencies yourself, you can find it at http://www.braincells.com/backports/ I uploaded 1.0test18 to http://src.braincells.com/dovecot-test/ (I'll do test20 tomorrow) I told Timo I'd port dovecot's gnutls support from gnutls7 to gnutls10 but I have absolutely no time for it so if someone would volunteer, contact me, I'd really appreciate it. -- Jaldhar H. Vyas <jaldhar(a)debian.org> La Salle Debain - http://www.braincells.com/debian/

4 7

[Dovecot] index file regeneration?
by Paul Reilly 02 Jul '04

02 Jul '04

Quick question: How does dovecot keep the message headers index file updated? Eg: if my MDA delivers an extra 10 messages while I'm logged in. How does the message index get updated? At what points is it regenerated? Thanks, Paul

2 7

[Dovecot] 1.0-test24 and some mbox benchmarking
by Timo Sirainen 02 Jul '04

02 Jul '04

http://dovecot.org/test/ Again only mbox fixes. I found some more bugs which could have also caused some of the mbox problems that people reported. They were found when I today thought I'd again do a bit of testing with my favourite 1.4GB mbox. Then I thought I might as well see how it compares against UW-IMAP. First a bit of explanations how they work internally: Dovecot 1.0-test24 ------------------ Dovecot works by trying to do everything in one "sync" function. It reads new mails, inserts missing headers, writes header modifications and expunges messages. Dovecot leaves 100 bytes of padding in every mail's headers, which it can use to avoid moving the rest of the file forward when it needs to insert space. However, if there's not enough space (and with new mails there isn't), it needs to do the moving. Moving is done by first reading messages and counting how many bytes we're short. Once we've seen enough padding we'll rewrite those messages. If we read until end of file, we'll grow the file and rewrite the messages. Rewriting is made backwards, so we don't have to do much buffering, and in case the rewriting gets interrupted (crash, power loss, etc.) the data loss is very small, a few kilobytes maximum. UW-IMAP 2004 ------------ UW-IMAP first only reads the mbox with SELECT, rewrite is only triggered by LOGOUT, EXPUNGE and CHECK commands. UW-IMAP also inserts padding, but less than Dovecot. It tries to get added headers to fit into 50 bytes, and what's not used is left as padding. Normally this seems to get it around 15-20 bytes of padding. Perhaps I should shrink it from Dovecot too. Rewriting works by reading the file forward into buffer and writing the changes as needed. This is quite fast, but it means the buffer can grow large, and if the rewrite gets interrupted everything in the buffer gets lost. In my test mbox this would have been 23MB of lost data, but normally much less. Benchmarks ---------- I simply rewrote a 1.4GB mbox containing 361052 mails, Linux kernel mailing list archives from years 96-02. Computer is Athlon XP 2700+ with 1GB of memory. reads/writes were counted using Linux's iostat command. Nothing else was being used in that partition, so the numbers should be accurate. Except UW-IMAP's read count is a few blocks too much because I got tired of waiting it and started looking into the mbox to see how far it had gotten. Read counts could also be somewhat wrong if some of the mbox was already in buffer cache. I tried to trash it anyway by catting 2x4GB of data into /dev/null. Kernel used around 900MB of memory for caching. Total CPU times may also be a bit off, as the computer was being used at the same time. Dovecot 1.0-test24 ------------------ reads : 4007432 blocks = 1956 MB writes: 2947381 blocks = 1439 MB original mbox : 1420611590 B = 2774632 blocks = 1354 MB rewritten mbox: 1472684487 B = 2876336 blocks = 1404 MB indexes : 14452732 B = 28227 blocks = 14 MB 7221164 dovecot.index 10436 dovecot.index.cache 7221132 dovecot.index.log (this will be truncated after a while) - 16064 VSZ, 8012 RSS after SELECT completed - 14MB is mmaped index files - 216kB heap left of which 70kB actually in use - heap usage was 25MB VSZ/RSS constantly while syncing, but allocations were so large that libc used anonymous mmap()s so they got dropped after sync - VSZ peaked at 32MB, most likely because index file(s) were temporarily being mmap()ed more than once - 63.37s user - 16.56s system - 24% cpu - 5:22.75 total UW-IMAP 2004 ------------ Reading: - memory: 70044 VSZ, 67724 RSS - CPU: 7s user, 53s total Totally: reads : 5549640 blocks = 2709 MB writes: 2875581 blocks = 1404 MB original mbox : 1420611590 B = 2774632 blocks = 1354 MB rewritten mbox: 1444411190 B = 2821115 blocks = 1377 MB - 93416 VSZ, 91120 RSS - 2295.81s user - 23.99s system - 96% cpu - 39:57.90 total Notes ----- Not counting Dovecot's indexes UW-IMAP wrote 21MB less. But Dovecot wrote 27MB more padding, so just by shrinking it Dovecot would have written 6MB less data. Also because Dovecot writes the file backwards, it needs to do some extra jumping around and overlapping writes, but I guess OS nicely merged them. I'm not exactly sure why UW-IMAP uses so much CPU for rewriting, but it does and so Dovecot is over 7x faster in total (with 36x less CPU). Dovecot should also support delaying the rewrite. This is mostly useful for POP3 clients which deletes all the mail at logout, so they won't need the rewriting at all. Dovecot also writes all flag changes to disk immediately while UW-IMAP leaves it later to do more at once. That results in less total I/O as well.

1 0

[Dovecot] 1.0-test22 - maildir fixed?
by Timo Sirainen 02 Jul '04

02 Jul '04

http://dovecot.org/test/ Well, remember when I said: > Here you go, maildir syncing problems finally fixed (I hope). Well, > except there's this "new-dir-only syncing" optimization which I'm not > really sure if it works as it should. Just before I found the expunge bug in index, I was trying to fix the new-dir-only syncing because the code looked wrong. Then I figured out the indexing problem and thought maybe this one wasn't broken after all, and then forgot about it. Finally after lots of debugging, I came to same conclusion again. The code was wrong, and cause of the remaining "Maildir sync: UID < next_uid (312 < 3)" errors. So, please try and see if you still can break this release. Other changes include some crashfixes for cache file and fix for mbox which kept constantly rewriting headers for \recent messages if we had selected the mailbox as read-only (EXAMINE, STATUS).

11 30

[Dovecot] building dovecot 1.0 test19 over fc1
by Luis Miguel Cruz 02 Jul '04

02 Jul '04

[root@fc1-devel tmp]# rpmbuild -ta dovecot-1.0-test19.tar.gz error: line 7: Illegal char '-' in version: Version: 1.0-test19 Seems to be an error into the .spec file I am not sure. I will check it. Did anyone get this problem in the past? Thanks.

2 1