- dovecot - dovecot.org

[Dovecot] uiddir mailbox format with benchmarks
by Timo Sirainen 30 Mar '07

30 Mar '07

I cleaned up the lib-storage code a bit so that it's easier to write support for new mailbox formats. It could still use a bit of cleaning up. Anyway, I thought I'd try how easy it would be to implement Cyrus-like mail store which would consist of only Dovecot's index files and "<uid>." message files. Flags and everything else is kept only in Dovecot's index files (so it might not be the best idea to use it in production just yet). Several hours later I ended up having it working. The code contains: ~/cvs/m/dovecot-head/src/lib-storage/index/uiddir% wc *[ch] 144 329 3552 uiddir-mail.c 240 565 6206 uiddir-save.c 532 1367 13628 uiddir-storage.c 55 126 1712 uiddir-storage.h 169 404 4057 uiddir-sync.c 21 48 573 uiddir-sync.h 103 258 2762 uiddir-transaction.c 1264 3097 32490 total uiddir-storage.c is still a bit too bloaty with lots of copy&pasted code. The same with maildir and mbox: maildir: 6022 18232 165605 total mbox: 7859 23283 213686 total Benchmarks show that with small mailboxes it's a lot faster than maildir, and with larger mailboxes it's a bit faster. I also tested it against Cyrus. I found out that fetching ENVELOPE vs. specific headers gives quite a huge performance difference. Cyrus is optimized for ENVELOPE while Dovecot is optimized for specific headers. With larger mailboxes + ENVELOPE Cyrus is faster than Dovecot, while with specific headers Dovecot is always faster. Maybe I'll some day make Dovecot optimize for ENVELOPE also if it sees that client only wants to use it. BTW. Does anyone have better naming ideas than "uiddir"? I'll commit it to CVS HEAD once I'm sure about the name. :) u: Dovecot CVS HEAD + uiddir m: Dovecot CVS HEAD + Maildir c: Cyrus IMAP 2.2.13 ./imaptest list=0 seed=1 secs=60 clients=10 Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 23486 11822 23481 23476 33426 11659 8038 23472 11922 46952 m: 19367 9734 19367 19356 27453 9687 6287 19347 7983 38709 c: 5124 2553 5121 5105 7080 2559 1877 5104 3271 10228 ./imaptest list=0 dele=20 expu=20 seed=1 secs=60 msgs=10000 clients=10 Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 5291 2704 5289 5278 7448 2706 1048 1061 5283 10563 m: 4162 2103 4162 4153 5977 2124 762 853 4153 8304 c: 4302 2157 4301 4289 6021 2072 806 879 4292 8584 ./imaptest list=0 dele=10 expu=10 seed=1 secs=60 msgs=100000 clients=10 Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 3211 1637 3206 3195 4639 1537 287 302 3202 6402 m: 2957 1508 2955 2944 4173 1440 279 306 2948 5894 c: 3933 1929 3933 3920 5604 2020 380 379 3923 7846 ------ ./imaptest list=0 dele=10 expu=10 seed=1 secs=60 msgs=100000 clients=1 Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 1529 745 1529 1528 2184 819 136 162 1528 3056 c: 1579 813 1579 1578 2257 811 171 148 1578 3156 ./imaptest list=0 dele=10 expu=10 seed=1 secs=60 msgs=100000 clients=50 Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 3098 1552 3093 3085 4432 1520 293 334 3089 6173 c: 3664 1765 3659 3596 5106 1802 365 364 3614 7228 ====== FETCH BODY.PEEK[HEADER.FIELDS(from to cc subject)] instead of ENVELOPE: ./imaptest list=0 dele=20 expu=20 seed=1 secs=60 msgs=10000 clients=10 Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 5777 2933 5777 5767 8215 2901 1133 1191 5767 11531 c: 3636 1794 3635 3623 5196 1843 682 722 3627 7254 ./imaptest list=0 dele=10 expu=10 seed=1 secs=60 msgs=100000 clients=10 Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 3869 1959 3868 3861 5452 1943 364 394 3859 7717 c: 3191 1620 3189 3184 4501 1598 315 327 3181 6362 ./imaptest list=0 dele=10 expu=10 seed=1 secs=60 msgs=100000 clients=1 Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 1912 966 1912 1910 2783 940 184 197 1911 3820 c: 1342 696 1342 1341 1888 682 109 126 1341 2682 ./imaptest list=0 dele=10 expu=10 seed=1 secs=60 msgs=100000 clients=50 Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 4618 2320 4610 4604 6558 2266 452 455 4605 9205 c: 2960 1472 2956 2941 4134 1425 281 312 2910 5820

3 5

[Dovecot] sieve-body extension support for deliver LDA's sieve plugin
by Scott Peron 30 Mar '07

30 Mar '07

Hi Timo & Co, Is there any time frame for updating the deliver LDA's sieve plugin code to that from Cyrus IMAPd v2.3.x? I ask as I'm looking for the functionality of the sieve-body extension, as described here: http://www.ietf.org/internet-drafts/draft-ietf-sieve-body-06.txt This requirement has come up as we're upgrading from a Courier-IMAP/maildrop configuration that has some filtering based on message body content. As an aside, if that time frame is undefined or will be long, if I use maildrop instead as my LDA, will I loose the instant delivery notification via IMAP IDLE? Thanks very much, Scott -- Scott Peron, Webmaster, Infolytica Corp. webmaster(a)infolytica.com Tel: (514) 849-8752 x269 Fax: (514) 849-4239 http://www.infolytica.com Place du Parc, 300 Leo-Pariseau, Suite 2222, Montreal, QC, Canada, H2X 4B3

2 1

[Dovecot] Version numbering
by Timo Sirainen 30 Mar '07

30 Mar '07

After v1.0 is released, I can finally get back to sane version numbers. But any comments on which one is better: a) Postfix-style: "1.1.UNSTABLE.YYYYMMDD" -> 1.1.0 (stable) b) Odd-even numbering: 1.1.x (unstable) -> 1.2.0 (stable) With a) style the releases could be done by simply copying a nightly snapshot to releases/ directory and announcing the changes since the last release. I'm not sure if that's good or bad.

27 51

Re: [Dovecot] Version numbering
by Kenneth Porter 30 Mar '07

30 Mar '07

--On Wednesday, March 28, 2007 3:46 AM +0300 Timo Sirainen <tss(a)iki.fi> wrote: > After v1.0 is released, I can finally get back to sane version numbers. Have the RC's released new features as well as fixed bugs? I've been hesitant to upgrade to an RC because they seem to be released so frequently and I have no idea if one is more or less stable than another. This uncertainty is a big show-stopper to upgrading. It would help if, for a given RC, we knew exactly what known issues that RC had. (Is that in the wiki?) I'd suggest that a "version" be assigned only to something that should get spread around. A nightly snapshot should get the version it was based from and a timestamp suffix of when it was extracted from CVS. (Were this Subversion, I'd append the repository revision, instead.) The snapshot is not a "version" but a convenient download from CVS. It should be exactly what you get with the same timestamp used in a checkout command. Apache's versioning scheme (posted by John Peacock) looks good to me: <http://apr.apache.org/versioning.html>

1 0

[Dovecot] Forbidding folder delete
by Kenneth Porter 30 Mar '07

30 Mar '07

Is there some way I can forbid the deleting of folders? I have a Thunderbird user who twice now has accidentally deleted a folder, forcing me to go fetch it from the nightly backup. I'd like to prevent this from happening again, and the easiest way to me is to block folder deletion. I'd rather do that "manually" from the shell than leave it available from the client GUI.

2 1

[Dovecot] Copy to sent mail timeouts
by Jeff Tucker 29 Mar '07

29 Mar '07

Hi. I'm running the dovecot IMAP that comes with Ubuntu Edgy, which is 1.0rc2. I realize this is old, but I'll ask my question in case anyone knows about this problem and how to fix it, or knows that a later version definitely fixes it. What I'm seeing is that the IMAP server works perfectly, except for occasional timeouts when the user sends an email with attachments, causing that email then to be copied to their sent mail folder. The behavior seen is that Thunderbird, for instance, will say it is copying to the sent mail and the progress goes up to 100%, but then stops and Thunderbird eventually says the action timed out. I don't see anything in any dovecot logging and the file does not appear in the sent mail. I don't have any other kind of timeout issues. I can move mails around folders, read large emails, or whatever. The problem seems worse with large emails, but some times it will copy 15 MB emails and some times it won't. I can't find that any particular email always fails. Any ideas? As a last resort, I'll build the latest rc from scratch, but I like the automatic installation, notification of security problems, etc. I get with the distro packages. Thanks Jeff

2 2

[Dovecot] Help with Virtual user setup.. need pointers
by mailagent 29 Mar '07

29 Mar '07

Dear Andre, My name is Michael Joyner. I don't know if you are the right person. I had a friend in Trinidad with a brothers name, Chris. If this is you could you please reply. I live in the United States and am trying to find my friend. Please disregard is if this information does not pertain to you. Regards, Michael Joyner cid:image001.jpg@01C76B49.22E36120 * Address 21006 Devonshire St., Suite 206 Chatsworth, CA 91311 ( Phone 818-256-1939 7 Fax 818-256-1934 8 Email mj(a)refipal.com <mailto:elisa.chavez.ppys@StateFarm.com>

1 0

[Dovecot] chroot and raw logging
by Michal Soltys 29 Mar '07

29 Mar '07

I'm not sure if it's even intended to work in this particular case, but there's a problem when chroot is explicitely defined and rawlogging is used. Rawlog respects chroot point, without being actually chrooted. Consider configurtion as below, with two cases (dovecot.rawlog dir is of course created earlier, with appropriate uid/gid). 1) dovecot.userdb: test::10000:10000::/home/vmail/test.tld/test/./::userdb_mail=maildir:~/Maildir: ktrace will show: 18206 rawlog CALL sigprocmask(0x3,0) 18206 rawlog RET sigprocmask -65793/0xfffefeff 18206 rawlog CALL lstat(0x81fbe028,0xcfbcc980) 18206 rawlog NAMI "/dovecot.rawlog" 18206 rawlog RET lstat -1 errno 2 No such file or directory 18206 rawlog CALL sigprocmask(0x1,0xffffffff) 2) dovecot.userdb: test::10000:10000::/home/vmail/test.tld/./test::userdb_mail=maildir:~/Maildir: 1116 rawlog CALL lstat(0x85768028,0xcfbc1d20) 1116 rawlog NAMI "test/dovecot.rawlog" 1116 rawlog RET lstat -1 errno 2 No such file or directory 1116 rawlog CALL sigprocmask(0x1,0xffffffff) 1116 rawlog RET sigprocmask 0 I also noticed something else (not rawlog related). If I do a silly (blame my curiosity ;) thing like: valid_chroot_dirs: / and set chroot point as /./home/vmail/test.tld/test/ in passwd-file Dovecot will try to stat home/vmail/test.tld/test/ (and fail w/o beginning /), then chdir to /tmp, start pop3 or imap process, and recreate home/vmail/test.tld/test under /tmp. 17027 dovecot CALL chdir(0x87a4b29c) 17027 dovecot NAMI "home/vmail/test.tld/test/" 17027 dovecot RET chdir -1 errno 2 No such file or directory 17027 dovecot CALL setitimer(0,0xcfbe7020,0xcfbe7010) 17027 dovecot RET setitimer 0 17027 dovecot CALL seteuid(0) 17027 dovecot RET seteuid 0 17027 dovecot CALL chdir(0x3c001b72) 17027 dovecot NAMI "/tmp" 17027 dovecot RET chdir 0 ... 17027 dovecot CALL sigprocmask(0x3,0) 17027 dovecot RET sigprocmask -65793/0xfffefeff 17027 dovecot CALL execve(0x87a4bc38,0x87a4bc70,0x84fdb900) 17027 dovecot NAMI "/usr/local/libexec/dovecot/pop3" ... 17027 pop3 CALL mkdir(0x85f5a2b0,0x1ff) 17027 pop3 NAMI "home/vmail/test.tld/test//Maildir" 17027 pop3 RET mkdir -1 errno 2 No such file or directory 17027 pop3 CALL mkdir(0x85f5a2e0,0x1ff) 17027 pop3 NAMI "home/vmail/test.tld/test/" 17027 pop3 RET mkdir -1 errno 2 No such file or directory 17027 pop3 CALL mkdir(0x85f5a328,0x1ff) 17027 pop3 NAMI "home/vmail/test.tld/test" 17027 pop3 RET mkdir -1 errno 2 No such file or directory 17027 pop3 CALL mkdir(0x85f5a348,0x1ff) ... and so on, then it creates all the home/vmail/... dirs Configuration: openbsd 3.9, i386, dovecot 1.0-rc28 dovecot.userdb: test::10000:10000::/home/vmail/test.tld/test/./::userdb_mail=maildir:~/Maildir: dovecot.passdb: test:{PLAIN}test: base_dir: /var/dovecot/ protocols: imap imaps pop3 pop3s ssl_listen: * ssl_ca_file: /etc/ssl/cert_bundle.pem ssl_cert_file: /etc/ssl/ca_ppgk/certs/fetch_crt.pem ssl_key_file: /etc/ssl/ca_ppgk/private/fetch_key.pem verbose_ssl: yes login_dir: /var/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login login_greeting: Dovecot IMAP server ready. login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(pop3): no valid_chroot_dirs: /var/mail:/home/regular:/home/vmail verbose_proctitle: yes first_valid_uid: 1999 first_valid_gid: 10 mail_location: mbox:~/Mail:INBOX=/var/mail/%u mail_debug: yes fsync_disable: yes lock_method: fcntl maildir_copy_with_hardlinks: yes mbox_read_locks: fcntl mbox_write_locks: fcntl mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 imap_client_workarounds(default): outlook-idle delay-newmail imap_client_workarounds(imap): outlook-idle delay-newmail imap_client_workarounds(pop3): outlook-idle pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %08Xu%08Xv pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh auth default: mechanisms: plain login verbose: yes debug: yes passdb: driver: passwd-file args: /etc/dovecot.passdb passdb: driver: bsdauth userdb: driver: passwd-file args: /etc/dovecot.userdb userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master:

2 2

[Dovecot] passwd-file contents are not being used
by Randolph Kahle 29 Mar '07

29 Mar '07

I am setting up my dovecot installation and I am not getting results that I expected for the authentication phase. I expected: * PAM authentication would not be attempted * passwd-file would provide gid, uid, home, mail I notice that even thought it appears that this information is returned from auth(default) in the IMAP(randykahle) phase, it is not using that information - it used uid=502, etc. Here is my log output: ----- Mar 29 05:59:08 kahle dovecot: auth(default): pam(randykahle, 72.211.130.251): lookup service=dovecot Mar 29 05:59:08 kahle dovecot: auth(default): pam(randykahle, 72.211.130.251): pam_authenticate() failed: Authentication failure Mar 29 05:59:08 kahle dovecot: auth(default): passwd-file /etc/ dovecot/userdb: Read 1 users Mar 29 05:59:08 kahle dovecot: auth(default): passwd-file(randykahle, 72.211.130.251): lookup: user=randykahle file=/etc/dovecot/userdb Mar 29 05:59:08 kahle dovecot: auth(default): client out: OK 1 user=randykahle home=/var/mail/vhosts/com.rosalita/ randy mail=maildir:~/Maildir uid=5000 gid=5000 Mar 29 05:59:08 kahle dovecot: auth(default): master in: REQUEST 26 11935 1 Mar 29 05:59:08 kahle dovecot: auth(default): passwd(randykahle, 72.211.130.251): lookup Mar 29 05:59:08 kahle dovecot: auth(default): master out: USER 26 randykahle system_user=randykahle uid=502 gid=502 home=/ home/randykahle Mar 29 05:59:08 kahle dovecot: IMAP(randykahle): Effective uid=502, gid=502, home=/home/randykahle Mar 29 05:59:08 kahle dovecot: IMAP(randykahle): maildir: access(/ home/randykahle/Maildir, rwx): failed: No such file or directory Mar 29 05:59:08 kahle dovecot: IMAP(randykahle): maildir: couldn't find root dir Mar 29 05:59:08 kahle dovecot: IMAP(randykahle): mbox: root exists (/ home/randykahle/mail) Mar 29 05:59:08 kahle dovecot: IMAP(randykahle): mbox: INBOX exists (/ var/mail/randykahle) Mar 29 05:59:08 kahle dovecot: IMAP(randykahle): mbox: root=/home/ randykahle/mail, index=/home/randykahle/mail, inbox=/var/mail/randykahle Mar 29 05:59:08 kahle dovecot: imap-login: Login: user=<randykahle>, method=PLAIN, rip=72.211.130.251, lip=72.249.22.55 Mar 29 05:59:11 kahle dovecot: IMAP(randykahle): open(/var/ mail/.temp.kahle.org.11965.76100dca2c5b2009) failed: Permission denied Mar 29 05:59:11 kahle dovecot: IMAP(randykahle): file_lock_dotlock() failed with mbox file /var/mail/randykahle: Permission denied Mar 29 05:59:16 kahle dovecot: IMAP(randykahle): Disconnected ----- Here is my passwd-file user information line: ----- randykahle:{plain}xxxxx:5000:5000::/var/mail/vhosts/com.rosalita/ randy::home=/var/mail/vhosts/com.rosalita/randy mail=maildir:~/ Maildir uid=5000 gid=5000 ----- Any tips on how to get the information in the passwd-file used by the IMAP process would be appreciated. -- Randy

2 1

[Dovecot] dovecot mmap() complaints: No such device
by sergey ivanov 29 Mar '07

29 Mar '07

I'm trying to use dovecot-deliver with sieve plugin, delivering to Maildirs. Everything seems working O.K., but I'm getting the following lines in /var/log/maillog: --- Mar 29 00:06:00 alt64 dovecot: imap-login: Login: user=<seriv>, method=PLAIN, rip=66.80.117.2, lip=192.168.10.8, TLS Mar 29 00:06:00 alt64 dovecot: IMAP(seriv): mmap() failed with index file /home/seriv/.imap/dovecot.index: No such device ... Mar 29 00:06:19 alt64 deliver(seriv): mmap() failed with index file /home/seriv/.imap/dovecot.index: No such device Mar 29 00:06:19 alt64 deliver(seriv): msgid=<460B3B13.7010000(a)omniti.com>: saved mail to INBOX Mar 29 00:06:19 alt64 postfix/local[15166]: 6294A141C269: to=<seriv(a)alt64.menlo>, relay=local, delay=0.24, delays=0.03/0.01/0/0.2, dsn=2.0.0, status=sent (delivered to command: /usr/lib64/dovecot/deliver -d ${USER}) Mar 29 00:06:19 alt64 postfix/qmgr[20933]: 6294A141C269: removed --- I have tried to enable rawlog, but have corresponding rawlog file do not explain the problem for me: --- 3 select "INBOX" 4 UID fetch 1:* (FLAGS) 5 UID fetch 43498:43501 (UID RFC822.SIZE FLAGS BODY.PEEK[HEADER.FIELDS (From To Cc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type)]) 6 UID fetch 43500 (UID RFC822.SIZE BODY.PEEK[]) 7 uid store 43500 +FLAGS (NonJunk) 8 IDLE --- Dovecot compiled from cvs branch_1_0 at March 25. --- $ sudo dovecot -n Password: # /etc/dovecot/dovecot.conf ssl_cert_file: /etc/postfix/cert.pem ssl_key_file: /etc/postfix/key.pem login_dir: /var/run/dovecot/login login_executable: /usr/lib64/dovecot/imap-login mail_extra_groups: mail mail_location: maildir:/home/%u/.imap mail_executable: /usr/lib64/dovecot/rawlog /usr/lib64/dovecot/imap auth default: passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 438 --- /home mounted as zfs-fuse filesystem. I guess the problem due to fuse mounted filesystem. -- With best regards, Sergey Ivanov.

2 2