February 2007 - dovecot

[Dovecot] POP3 UIDL
by FiL ＠ Kpoxa 07 Feb '07

07 Feb '07

I have installed rc22 and tried it with my mbox-based mail - fine. Then I converted that mbox to Maildir to see if new feature (saving state without QUIT) works - first run - I get time out. Second run - I get rest of the Maildir scanned and get into mailbox. Fine. but.... # telnet enders2 110 Trying 127.0.0.1... Connected to localhost.localdomain (127.0.0.1). Escape character is '^]'. +OK Dovecot ready. user filonovd +OK pass ****** +OK Logged in. uidl +OK . where is the list of pop3 uids ? When I connect to mbox I get something like .......... 84904 d8f096c72c24719bd7d13d78a3694114 84905 9074b40ab4283ec38a151587f34690d9 84906 06a40afeaca43fc76bea2231b13b1a33 84907 cdae771ec2407df89a25321f3792d9e9 84908 9f84415625d77dd3cde7d9441847318e but with Maildir I get nothing. Doesn't look like proper behavior. P.S. mail_location = maildir:~/Maildir #mail_location = mbox:~/mail:INBOX=/var/mail/%u pop3_reuse_xuidl = no pop3_uidl_format = %m FiL

6 10

[Dovecot] negative quota usage causes over-quota messages with dictionary quota backend
by Marshal Newrock 07 Feb '07

07 Feb '07

Using dovecot 1.0.rc21, with quotas being stored in mysql, I have found that if the quota usage stored is negative, then when a message is given to dovecot deliver, the account is reports as being over quota. The squirrelmail quota usage plugin says that the quota is at around 40 million percent (I should have written down the exact number), though I don't know if that's squirrelmail or dovecot coming up with that number. At a guess, I'd say there's an unsigned int being used where the result can be signed. I'm using MySQL 5, so for an easy workaround, I added a trigger that sets the quota usage to 0 if it goes less than 0. I could probably also set the field to unsigned int, but I haven't tested what mysql does if you try to change an unsigned int field such that it would be negative. -- Marshal Newrock, Ideal Solution LLC http://www.idealso.com

1 0

[Dovecot] OTP?
by Micah Cowan 07 Feb '07

07 Feb '07

Hello, I was wondering where I might find more information about using OTP as an authentication protocol with dovecot. In searching, I found a thread from 2004, but not much information about how it has progressed from then. I also saw some promising patches from mid-last year, which if I had to guess are probably the addition of support for the SASL OTP mechanism. Is this available in a release yet? (And, also: does anyone know of Linux mail clients that support the OTP SASL mechanism? I'm hoping Thunderbird might...). If not, is it possible to make it work currently with PAM (or, if that's not possible, maybe Cyrus SASL)? It would be okay with me if it can work with PAM but not prompt with the seed/series number: I can scratch off keys no prob. I tried briefly to get it to work through PAM, and in fact opieinfo told me that the series was being decremented, but auth.log gives Feb 6 15:41:46 mabruk dovecot-auth: (pam_unix) authentication failure; logname= uid=0 euid=0 tty= ruser= rhost=x.x.x.x user=micah ssh works fine with the same PAM settings (both include common-auth, which has:) auth sufficient pam_opie.so auth sufficient pam_unix.so nullok_secure auth required pam_deny.so -- Micah J. Cowan Programmer, musician, typesetting enthusiast, gamer... http://micah.cowan.name/

4 7

[Dovecot] imap crashes on deleting an e-mail with no message-id.
by Tan Shao Yi 07 Feb '07

07 Feb '07

Hi, I'm on RC21 on a Solaris 10 SPARC box. Somehow I managed to receive an e-mail with no message-id and when dovecot tries to delete the e-mail. Viewing the e-mail is not a problem though. I attached truss to the imap process and this was what I received: pollsys(0x000B7A88, 3, 0xFFBFF870, 0x00000000) = 0 pollsys(0x000B7A88, 3, 0xFFBFF870, 0x00000000) = 0 pollsys(0x000B7A88, 3, 0xFFBFF870, 0x00000000) (sleeping...) pollsys(0x000B7A88, 3, 0xFFBFF870, 0x00000000) = 1 read(0, " g 1 b q U I D S T O".., 3747) = 52 alarm(120) = 0 fcntl(6, F_SETLKW64, 0xFFBFF1D0) = 0 alarm(0) = 120 close(8) = 0 open64("/maildirs/tansy/Maildir/maildirsize", O_RDWR|O_APPEND) = 8 read(8, " 1 0 7 3 7 4 1 8 2 4 0 S".., 5120) = 2223 read(8, 0xFFBFE8DF, 2897) = 0 Incurred fault #6, FLTBOUNDS %pc = 0x00086FF8 siginfo: SIGSEGV SEGV_MAPERR addr=0x00000000 Received signal #11, SIGSEGV [default] siginfo: SIGSEGV SEGV_MAPERR addr=0x00000000 I believe it's due to the mail_log plugin that is logging the message-ids being deleted. When I disabled the mail_log plugin, the e-mail gets deleted correctly. Timo, do you think we can have mail_log to handle such exceptions? :) Thanking in advance. Cheers, Tan Shao Yi

2 1

[Dovecot] Default domain name
by Nils Vogels 07 Feb '07

07 Feb '07

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi! I'm trying to setup a dovecot installation and at the same time i'm also moving from system to virtual users. In order for end-users to experience as little impact as possible, I'd like to add a default domainname to the login, if the login doesn't contain a full username. For instance: * User logging in as "user(a)mydomain.com" will get the mailbox for user(a)mydomain.com. * User logging in as "user" will also get the mailbox for user(a)mydomain.com Is there some way to do this, without using the pop3/imap proxy? I've searched the wiki and sample configuration file, but was only able to find entries about authentication rewrites so far, and I don't know if this could be done with auth_username_translation ? Greetings, Nils - -- Those who desire to give up freedom in order to gain security, will not have, nor do they deserve, either one. ~Benjamin Franklin (American Statesman, Scientist, Philosopher, Printer, Writer and Inventor. 1706-1790) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (MingW32) iD8DBQFFyP+3MzNX/a06Wq0RAvQ4AJwNWr7ov8Fw6JhuoOQGiQte51FW9wCfaZ7x Y3VbqigfdqSFgnlA0Aukucg= =owku -----END PGP SIGNATURE-----

2 1

[Dovecot] Indexing Performance Question (was tpop3d vs dovecot)
by Nate 06 Feb '07

06 Feb '07

Since posting the previous thread we've setup a new system (Opteron 2.0ghz, 1gb ram, Fedora 6) for testing. I am hoping somebody very familiar with the indexing portion of dovecot can shed some light on this for me. After much testing, I've come to one primary conclusion that dovecot is possibly unnecessarily scanning or reading files within the maildir directories. Take a mailbox for instance (my test one has 71570 emails in the cur/ folder so the process runs long enough to get data of what it's doing). Follow these steps: 1. First pop3 connection. With no index files whatsoever, it takes nearly a full 10 minutes at 100% io load to scan the 71570 files (748M) directory and build the index files from scratch which look like this: -rw------- 1 postfix postdrop 839K Feb 1 00:43 dovecot.index -rw------- 1 postfix postdrop 1.9M Feb 1 00:57 dovecot.index.cache -rw------- 1 postfix postdrop 560K Feb 1 00:43 dovecot.index.log -rw------- 1 postfix postdrop 3.1M Feb 1 00:43 dovecot-uidlist While it's building index i see a lot of these fly by in lsof pop3 2023 postfix 8r REG 8,3 14425 285947 /var/spool/mail/test.visp.net/natetest/cur/1128488047.V902I7d42e.s3.visp.net:2, No problem here, the disk can only go as fast as the disk can go and thats a *lot* of mail. Although, one curious thing here, it appears that dovecot is reading or at least scanning the entire contents of the files within the cur/ directory to generate an index. I assume this by watching my disk cache go from 100MB to 600MB. Is it necessary to do that or only read the top X lines of each file, is that even possible? 2. Second connection. Near instantaneous results from a login and LIST. Cannot perform an lsof fast enough to see what files it's opening. This step can be repeated with nearly the same results. *rebooted machine to clear all cache* 3. Third connection. It takes ~40 seconds to display a LIST after login. The index files are untouched; however, during the 40 seconds I'm waiting, I can catch a few of these in lsof. pop3 1971 postfix 10r REG 8,3 4196 205963 /var/spool/mail/test.visp.net/natetest/cur/1157863512.V902I1f3867.s3.visp.net:2,S This tells me even though there is no new email, it is re-scanning the cur/ directory somewhat. It doesn't take near as long as the initial index build, but certainly takes longer than it would for the system to read any or all of the listed index files above. Why would dovecot need to even enter cur/ if the index file is up to date? Is it verifying that the index file is up to date by re-reading some or all of the many files in cur/? If so, could it not get the info it needs by simply doing the equivalent of a ls, which would take considerably less time? The disk cache is only filled up an additional 30meg vs an additional 500meg from connection 1, so it's certainly not reading as much data. (also makes me wonder why the initial scan/index build could not use this lesser time consuming function if thats what it's doing). Additional information: Running strace on the initial indexing looks like this... ------------------------------------------- open("/var/spool/mail/test.visp.net/natetest/cur/1115929738.V902I67c13.test.visp.net:2,", O_RDONLY) = 8 fstat(8, {st_mode=S_IFREG|0600, st_size=4667, ...}) = 0 pread(8, "Return-Path: <mehl8nl3n8osm3r83a"..., 4096, 0) = 4096 pread(8, "zdik8dyrnffv6bbrue.itownshipno12"..., 4095, 4096) = 571 pread(8, "", 3524, 4667) = 0 close(8) = 0 open("/var/spool/mail/test.visp.net/natetest/cur/1115931742.V902I67c42.test.visp.net:2,", O_RDONLY) = 8 fstat(8, {st_mode=S_IFREG|0600, st_size=3881, ...}) = 0 pread(8, "Return-Path: <sender-5-8759012-9"..., 4096, 0) = 3881 pread(8, "", 215, 3881) = 0 close(8) = 0 ------------------------------------------- Running strace on the 3rd connection looks like this... ------------------------------------------- open("/var/spool/mail/test.visp.net/natetest/cur/1114490671.V902I6011f.test.visp.net:2,", O_RDONLY) = 10 fstat(10, {st_mode=S_IFREG|0600, st_size=5367, ...}) = 0 pread(10, "Return-Path: <qflfmq82vz8zg-u6ci"..., 4096, 0) = 4096 pread(10, "gs/rpoc_District6Spauldings.php?"..., 4095, 4096) = 1271 pread(10, "", 2824, 5367) = 0 close(10) = 0 open("/var/spool/mail/test.visp.net/natetest/cur/1114496568.V902I60bf5.test.visp.net:2,", O_RDONLY) = 10 fstat(10, {st_mode=S_IFREG|0600, st_size=2047, ...}) = 0 pread(10, "Return-Path: <removed(a)aaa-calif.c"..., 4096, 0) = 2047 pread(10, "", 2049, 2047) = 0 close(10) = 0 Thank you for the help! - Nate

8 27

[Dovecot] Problem with vpopmail auth
by Fred Moyer 06 Feb '07

06 Feb '07

Hello, I've install 1.021 and am having some problems getting vpopmail authentication to work correctly (running 0.99 in production, upgrading to 1.021). I'm using vpopmail-5.4.16. Here are the authentication debug statements in the log, my dovecot.conf changes from default, and the schema of my vpopmail database. Any ideas / suggestions? I've commented out all authentication except for vpopmail. I took a look around the source but am not that familiar with the vpopmail api. Thanks dovecot: Feb 05 22:23:31 Info: auth(default): client in: AUTH 1 PLAIN service=IMAP lip=10.0.0.2 rip=10.0.0.1 resp=<hidden> dovecot: Feb 05 22:23:31 Info: auth(default): vpopmail(fred(a)redhotpenguin.com,10.0.0.1): crypted password=rightpasswd dovecot: Feb 05 22:23:31 Info: auth(default): client out: OK 1 user=fred(a)redhotpenguin.com dovecot: Feb 05 22:23:31 Info: auth(default): master in: REQUEST 1 13457 1 dovecot: Feb 05 22:23:31 Info: auth(default): vpopmail(fred(a)redhotpenguin.com,10.0.0.1): vget_assign(redhotpenguin.com) failed dovecot: Feb 05 22:23:31 Info: auth(default): master out: FAIL 1 dovecot: Feb 05 22:23:31 Info: imap-login: Internal login failure: user=<fred(a)redhotpenguin.com>, method=PLAIN, rip=10.0.0.1, lip=10.0.0.2 852 # vpopmail authentication 853 # http://wiki.dovecot.org/AuthDatabase/VPopMail 854 passdb vpopmail { 855 # args = cache_key=%u dovecot 856 # [cache_key=<key>] - See cache_key in PAM for explanation. 857 # args = cache_key=%u 858 } 907 # vpopmail 908 # http://wiki.dovecot.org/AuthDatabase/VPopMail 909 userdb vpopmail { 910 } +-----------+-----------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-----------+-----------+------+-----+---------+-------+ | pw_name | char(32) | | PRI | | | | pw_domain | char(64) | | PRI | | | | pw_passwd | char(40) | YES | | NULL | | | pw_uid | int(11) | YES | | NULL | | | pw_gid | int(11) | YES | | NULL | | | pw_gecos | char(48) | YES | | NULL | | | pw_dir | char(160) | YES | | NULL | | | pw_shell | char(20) | YES | | NULL | | +-----------+-----------+------+-----+---------+-------+

1 1

[Dovecot] Big problems after upgrade from beta8 (FC5) to rc21
by Jim Garrison 06 Feb '07

06 Feb '07

After upgrading a working system from FC5's beta8 to rc21, one vhost is failing with: > dovecot: Feb 05 09:54:30 Error: POP3(user(a)mydomain.com): > open(/var/mail/vhosts/mydomain.com/david/user, O_CREAT) failed: Not a > directory > dovecot: Feb 05 09:54:30 Error: POP3(user(a)mydomain.com): > mkdir_parents(/var/mail/vhosts/mydomain.com/user/.imap/INBOX) failed: > Not a directory > dovecot: Feb 05 09:54:30 Error: POP3(user(a)mydomain.com): Couldn't open > INBOX: Internal error occurred. Refer to server log for more > information. [2007-02-05 09:54:30] > dovecot: Feb 05 09:54:30 Info: POP3(user(a)mydomain.com): No INBOX for > user top=0/0, retr=0/0, del=0/0, size=0 > dovecot: Feb 05 09:54:30 Info: pop3-login: Login: > user=<user(a)mydomain.com>, method=PLAIN, rip=::ffff:4.242.120.235, > lip=::ffff:216.75.55.45, TLS Here's the entire dovecot.conf: protocols = pop3s ssl_cert_file = /etc/pki/dovecot/certs/popcert.pem ssl_key_file = /etc/pki/dovecot/private/popkey.pem ssl_ca_file = /etc/pki/tls/certs/ca-bundle.crt ssl_parameters_regenerate = 48 disable_plaintext_auth = yes log_path = /var/log/dovecot.log mail_extra_groups = mail mbox_read_locks = fcntl mbox_write_locks = fcntl dotlock protocol imap { } protocol pop3 { } auth_verbose = no auth_debug = no auth_debug_passwords = no auth default { mechanisms = plain passdb passwd-file { args = /etc/dovecot/%d.passwd } passdb pam { } userdb passwd { } userdb static { args = uid=502 gid=502 home=/var/mail/vhosts/%d mail=mbox:/var/mail/vhosts/%d/%n } user = root } plugin { } Did something changed between beta8 and rc21 in the index format and/or mbox location? If so, how do I recover so users can retrieve mail again? Thx.

2 4

[Dovecot] LDAP auth_bind=yes in rc21 = goodness
by Adrian Close 06 Feb '07

06 Feb '07

Hi all, I finally got a chance to try out the fix for the LDAP auth_bind=yes issues in production (using rc21). It seems to be working just fine even after 24 hours of severe abuse. Incidentally, I had switched to bsdauth+login_ldap (on OpenBSD) which was often OK, but occasionally I would see Postfix smtpd throttled with SASL errors due to "connection refused" on the Dovecot authentication socket. This may just have been due to increased LDAP server load (Dovecot's internal LDAP generates much less). Thanks Timo! (And thanks Brad for updating the port) Adrian.

2 1

[Dovecot] minor kerberized dovecot build issue
by Dan Price 05 Feb '07

05 Feb '07

Hi all, again, I'm not knowledgeable about Kerberos, but we have a few users who use it to make sure that it works well on Solaris... so, they would like a Kerberized IMAPd. I tried compiling dovecot --with-gssapi on Solaris Nevada (the current development release), and the configure script choked for a couple of reasons. First, script uses 'grep -qv' in the case that you specify --with-gssapi to run some test. Unfortunately, Solaris doesn't have 'grep -q'. There's a note in GNU grep's manpage about this problem and a note that portable scripts should just redirect to /dev/null. Once I fixed this problem, I got: configure: WARNING: gssapi/gssapi.h: present but cannot be compiled configure: WARNING: gssapi/gssapi.h: check for missing prerequisite headers? configure: WARNING: gssapi/gssapi.h: see the Autoconf documentation configure: WARNING: gssapi/gssapi.h: section "Present But Cannot Be +Compiled" configure: WARNING: gssapi/gssapi.h: proceeding with the preprocessor's result configure: WARNING: gssapi/gssapi.h: in the future, the compiler will take +precedence configure: WARNING: ## ---------------------------------- ## configure: WARNING: ## Report this to dovecot(a)dovecot.org ## configure: WARNING: ## ---------------------------------- ## For whatever reason, on Solaris /usr/bin/krb5-config does not like the syntax 'krb5-config --cflags gssapi' (ditto with --libs). I found that removing the 'gssapi' part from the configure script fixed the problem, but I imagine this must break some other OS. If need be, I can follow up with the Kerberos team about this. Once I made these minor changes, I also had to add -lgss to the src/auth makefile's compile line. After that, I had no problem getting this to compile, although we have not yet tested it. -dp -- Daniel Price - Solaris Kernel Engineering - dp(a)eng.sun.com - blogs.sun.com/dp

2 3