June 2007 - dovecot - dovecot.org

[Dovecot] test program #2: mmaping
by Timo Sirainen 21 Jun '07

21 Jun '07

Attached another test program. I don't expect it to print any errors with any OS, but I'd like to confirm it for non-Linux SMP kernels. (Except for OpenBSD, it doesn't work correctly in it anyway because it doesn't support mixing write()s and mmap())

8 8

[Dovecot] Status update
by Timo Sirainen 21 Jun '07

21 Jun '07

I'm leaving soon and I'll be away until sunday. Sorry about not replying to all mails yet, most of them are somewhat tricky and require some time to reply (other than "I don't know"). The new index code for v1.1 seems to be finally working pretty nicely. After 9 hours of running imaptest I haven't gotten a single error yet. I'll leave it running until I'm back. The new code makes the internal index file design a lot easier to understand. For v1.0 I just wanted to get it working and I didn't dare try to clean up the code, so it was kind of magic sometimes why it worked. :) All that magic is finally gone and I understand completely how everything is supposed to work (and hopefully also how it does work). Most important problem with the new code is that \Recent flags are completely broken. I think they're not completely correctly working in v1.0 either, or at least I'm not sure why they're working even as well as they are now.. Maildir syncing in v1.1 is also a bit bad. For example EXPUNGE command could have to be given twice to get it to work. Again it worked badly with v1.0 also but since the index code worked differently it wasn't visible. Cydir is a pretty nice format for stress testing index code. It's really lightweight so it's practically testing only the index code. Because of it I've found a couple of race conditions in the code that existed also in v1.0 tree (v1.0.2 will fix them). My earlier stress testing with maildir never saw those problems, probably because with maildir there was a lot of other code running as well so the race condition was really unlikely to hit.

1 0

[Dovecot] pop3 creates bad index file
by Onno Molenkamp 21 Jun '07

21 Jun '07

Hi, When I use pop3 to log on to a (Maildir) mailbox that doesn't yet exist in the filesystem, the pop3 server autocreates it for me. This works. Jun 21 10:50:45 mail-04 dovecot: pop3-login: Login: user=<hwx7s(a)gqh7x.test>, method=PLAIN, rip=::ffff:192.168.42.105, lip=::ffff:192.168.42.97 Jun 21 10:50:48 mail-04 dovecot: POP3(hwx7s(a)gqh7x.test): Disconnected: Logged out top=0/0, retr=0/0, del=0/0, size=0 When I log on a second time, it will complain about UIDVALIDITY that has changed, and will disconnect me: Jun 21 10:50:52 mail-04 dovecot: pop3-login: Login: user=<hwx7s(a)gqh7x.test>, method=PLAIN, rip=::ffff:192.168.42.105, lip=::ffff:192.168.42.97 Jun 21 10:50:52 mail-04 dovecot: POP3(hwx7s(a)gqh7x.test): Maildir /data/mail/898/gqh7x.test/hwx7s/Maildir sync: UIDVALIDITY changed (1182415845 -> 1182415852) Jun 21 10:50:52 mail-04 dovecot: POP3(hwx7s(a)gqh7x.test): Disconnected: Mailbox is in inconsistent state. top=0/0, retr=0/0, del=0/0, size=0 Jun 21 10:50:52 mail-04 dovecot: POP3(hwx7s(a)gqh7x.test): Couldn't init INBOX: Internal error occurred. Refer to server log for more information. [2007-06-21 10:50:52] When I log on a third time, everything seems to work from the client side, but the server still complains: Jun 21 10:51:01 mail-04 dovecot: pop3-login: Login: user=<hwx7s(a)gqh7x.test>, method=PLAIN, rip=::ffff:192.168.42.105, lip=::ffff:192.168.42.97 Jun 21 10:51:01 mail-04 dovecot: POP3(hwx7s(a)gqh7x.test): Corrupted index cache file /data/mail/898/gqh7x.test/hwx7s/Maildir/dovecot.index.cache: indexid changed Jun 21 10:51:02 mail-04 dovecot: POP3(hwx7s(a)gqh7x.test): Disconnected: Logged out top=0/0, retr=0/0, del=0/0, size=0 Fourth (and further) logons work without errors. When the Maildir is created by imap or deliver, pop3 works fine from the start. The only real difference I see with idxview between the dovecot.index created by pop3 and one created by imap is that with pop3 sync stamp is 0, and with imap it contains a timestamp. Onno

2 1

[Dovecot] mail_log plugin updates
by Timo Sirainen 21 Jun '07

21 Jun '07

It supports configuration now in devel version: # Events to log. Default is all. #mail_log_events = delete undelete expunge copy mailbox_delete # Group events within a transaction to one line. #mail_log_group_events = # Available fields: uid, box, msgid, size, vsize # size and vsize are available only for expunge and copy events. #mail_log_fields = uid box msgid size Without grouping you get pretty similar output than before: Info: copy: uid=13, dest=Trash, msgid=<20040702035043.7296C300DFC1A@asdf>, size=789 Info: delete: uid=13, msgid=<20040702035043.7296C300DFC1A@asdf> Info: expunge: uid=13, msgid=<20040702035043.7296C300DFC1A@asdf>, size=789 With mail_log_group_events=yes you get output like this: Info: copy: uids=15-17,24,26, dest=Trash, size=1534 Info: delete: uids=15-17,24,26 Info: expunge: uids=15-17,24,26, size=1534 The size is the summed size of all the messages. msgids aren't suppported. "size" means physical message size, "vsize" means the virtual size with CR+LF linefeeds. Does someone want something more?

2 2

[Dovecot] Question about pre-indexing maildirs
by David Favor 21 Jun '07

21 Jun '07

I have a situation with many users and an existing mail delivery agent. Seems like I once saw an imaptest program which I imagine might be used to pre-index maildirs. For example... - remember which users and user maildirs have changed since last pre-index - when system quiet pre-index any maildirs which have had mail delivered and no access by a mail client (using some inotify magic) If this sounds do-able, let me know where I can find an imaptest version which works with latest dovecot trunk code and the syntax for using it. Seems like I saw this once on the dovecot site. Thanks. - David -- Like feeling your best ever, all day, every day? Here's how... Your simple secrets are here - http://RadicalHealth.com

2 1

[Dovecot] how do I download concurrency.c
by Stewart Dean 20 Jun '07

20 Jun '07

When I click on the link I get not found on the server. I run on a 4 way and AIX has some pretty fancy concurrency so it would be interesting. But then I'm leaving this afternoon and won't be back until Monday..... -- ==== Stewart Dean, Unix System Admin, Henderson Computer Resources Center of Bard College, Annandale-on-Hudson, New York 12504 sdean(a)bard.edu voice: 845-758-7475, fax: 845-758-7035

2 1

[Dovecot] testing needed: log file concurrency
by Timo Sirainen 20 Jun '07

20 Jun '07

http://dovecot.org/tmp/concurrency.c I'd want to know what results this program gives with different systems. Please test and reply (but don't bother if someone already replied with the same OS+result). I expect it to print: - SMP kernels: "page size cut" once in a while - UP (uniprocessor) kernels: Nothing - The most important thing is that it never prints "broken data" It might take a while for it to print anything. With my computer it takes anything from a few seconds to a minute or so. See the file itself for compiling/running instructions. So far I've tested only with Linux 2.6.21 x86-64/SMP and a slow Solaris/Sparc/UP. If you're interested in knowing what this is about: Dovecot writes to dovecot.index.log files by first writing the transaction with its size being 0. After that it writes the 4 size bytes again (using a bit special format with all bytes ORed with 0x80). I expected that when another process is read()ing the file and it notices the size being valid (all bytes having 0x80) that the whole transaction could always be read. But looks like if the size happens to be just before a memory page boundary, it's possible that the updated size is read, but the rest of the transaction isn't.

13 18

[Dovecot] (OT) Advice on best way to collect mail from multiple domains and deliver to users via Dovecot IMAP.
by Dave Hatton 20 Jun '07

20 Jun '07

Hi Sorry if this is an inappropriate question to ask here. If it is maybe there is a better place to ask this question? I would like to collect email sent to multiple users (with previously unknown usernames) sent to multiple domains and then deliver this email to a specific set of users on my server based upon some predefined rules, with a catchall for mails that aren't caught by these rules. So as an example, I might have domains:- example.com, example.co.uk, someotherdomain.com and I'd like to say to a sender "just send a mail to anything(a)example.com". I'd like to be able to make up recipients on the fly when subscribing to a service so that I can filter mails to this address out at a later time. The method I devised when I set this up using dovecot .99 is as follows. I currently have the hosting service for the domains forwarding all email to a singular mailbox at my ISP and I user fetchmail to pop3 the mail down from this one mailbox. The mail is then passed to postfix where I use the luser_relay option to map all unknown recipients to a local address so that it doesn't get bounced. Postfix then delivers the mail via the mailbox_command to maildrop where a series of rules inspect the envelope headers and write the mail using the "to" option to an appropriate maildir for the target user. Is this a good approach? Would I be better off using dovecot LDA and sieve? Any advice or ideas would be gratefully received. Daveh

1 0

Re: [Dovecot] testing needed: log file concurrency
by Bill Boebel 20 Jun '07

20 Jun '07

On Tue, June 19, 2007 7:41 pm, Timo Sirainen <tss(a)iki.fi> said: > http://dovecot.org/tmp/concurrency.c > > I'd want to know what results this program gives with different systems. > Please test and reply (but don't bother if someone already replied with > the same OS+result). I expect it to print: ./concurrency writing, page size = 4096 ./concurrency 1 reading, page size = 4096 Red Hat ES3 (2.4.21-32.0.1.EL) athlon i386

1 0

[Dovecot] Error: net_connect(/opt/local/var/run/dovecot/auth-master) failed: No such file or directory
by J Charles Ferrari 20 Jun '07

20 Jun '07

What is auth-master, and how do I create it? Using postfix 2.1.5 dovecot 1.0.0 # /opt/local/etc/dovecot/dovecot.conf base_dir: /var/run/dovecot/ log_path: /var/log/dovecot_error.log info_log_path: /var/log/dovecot.log protocols: imap pop3 ssl_disable: yes login_dir: /var/run/dovecot/login login_executable(default): /opt/local/libexec/dovecot/imap-login login_executable(imap): /opt/local/libexec/dovecot/imap-login login_executable(pop3): /opt/local/libexec/dovecot/pop3-login mail_extra_groups: mail mail_location: maildir:/var/Maildir/%u mail_debug: yes mail_executable(default): /opt/local/libexec/dovecot/imap mail_executable(imap): /opt/local/libexec/dovecot/imap mail_executable(pop3): /opt/local/libexec/dovecot/pop3 mail_plugin_dir(default): /opt/local/lib/dovecot/imap mail_plugin_dir(imap): /opt/local/lib/dovecot/imap mail_plugin_dir(pop3): /opt/local/lib/dovecot/pop3 auth default: verbose: yes debug: yes passdb: driver: pam userdb: driver: passwd socket: type: listen client: master: path: /var/run/dovecot/auth-master mode: 384 user: vmail group: mail Thanks Charles

4 3