August 2006 - dovecot

[Dovecot] 1.0.rc2 released
by Timo Sirainen 11 Aug '06

11 Aug '06

Final 1.0 release is looking to get closer. There's nothing left in my TODO list for it, so please report any old and new bugs you see. This release also makes upgrades from 0.99 more transparent with automatic renaming of .subscriptions and .customflags files with maildir. http://dovecot.org/releases/dovecot-1.0.rc2.tar.gz http://dovecot.org/releases/dovecot-1.0.rc2.tar.gz.sig * disable_plaintext_auth=yes: Removed hardcoded 127.* and ::1 IP checks. Now we just assume that the connection is secure if the local IP matches the remote IP address. * SSL code rewrite which hopefully makes it work better than before. Seems to work correctly, but if you suddently have trouble with SSL connections this is likely the reason. + verbose_ssl=yes: Log also SSL alerts and BIO errors - If namespace's location field wasn't set, the default location was supposed to be used but it wasn't. - When copying ssl-parameters.dat file from /var/lib to /var/run its permissions went wrong if it couldn't be copied with hard linking. - Fixed filesystem quota plugin to work with BSDs. - Maildir: Saving mails didn't work if quota plugin was enabled (again) - Maildir: Messages' received time wasn't saved properly when saving/copying multiple messages at a time. Also if using quota plugin the S= size was only set for the first saved file, and even that was wrong. - passdb passwd-file: Don't require valid uid/gid fields if file isn't also being used as a userdb. - PostgreSQL: Handle failures better so that there won't be "invalid fd" errors in logs. - Don't try to expunge messages if the mailbox is read-only. It'll just cause our index files to go out of sync with the real mailbox and cause errors. - ANONYMOUS authentication mechanism couldn't work because anonymous_username setting wasn't passed from master process.

9 15

[Dovecot] imap-login (proxy) Connection queue full
by Bill Boebel 11 Aug '06

11 Aug '06

When running Dovecot (rc6 CVS dovecot-20060808) as a proxy, why would it log the following when it has not yet maxed out the allowed number of imap-login processes? Aug 10 16:23:34 director5 dovecot: imap-login: Disconnected: Connection queue full: I am seeing this logged several times per minute. From netstat, the server has 650 proxied IMAP connections and 450 proxied IMAPS connections, there are currently 10 imap-login processes running, and using the following config: login_process_per_connection = no login_processes_count = 3 login_max_processes_count = 20 login_max_connections = 225 I would think that under these conditions we should never see "Connection queue full" being logged. Thanks, Bill

2 1

[Dovecot] OpenLDAP only?
by ssoo＠siliconfile.com 10 Aug '06

10 Aug '06

I tried to compile Dovecot v1.0RC2 on Opeteron/Solaris 10 box. "db-ldap.c", line 343: undefined symbol: LDAP_SASL_QUIET "LDAP_SASL_QUIET" is defined as "2U" in OpenLDAP's ldap.h but not in /usr/include/ldap.h How can I compile Dovecot with Sun's LDAP implementation?

3 3

[Dovecot] maildir++ specification compliance.
by Apps Lists 10 Aug '06

10 Aug '06

Hi folks. I was doing some testing this morning and realized Dovecot RC2 isn't fully compliant with the maildir++ specification. Most notably, it appears that Dovecot isn't creating a "maildirfolder" file (0 byte length) in folders it creates. >From http://www.inter7.com/courierimap/README.maildirquota.html "Within each subdirectory there's an empty file, maildirfolder. Its existence tells the mail delivery agent that this Maildir is a really a folder underneath a parent Maildir++." and "Make sure to create the maildirfolder file in any new folders created within the Maildir++." Also, GETQUOTAROOT on a folder (with messages in it) will always return the entire usage, not the usage in the folder itself. For example: INBOX - really has 100 mb of messages. Sent - has a few messages totalling 1mb. Drafts - no messages A GETQUOTAROOT on INBOX will return 101 mb, 100mb for INBOX and 1mb for Sent. Drafts will show zero, because there's nothing. Unfortunately, Sent will also show 101 mb, not 1mb. While Dovecot shouldn't be creating maildirsize files in each folder (the maildirfolder file is there to tell clients to look in .. for a maildirsize) ... it should still be able to answer queries about the number of messages (count of files in the folder) as well as the total size of all messages in that folder (sum of S=xxxx or by stat() on individual message files). Steve

4 4

[Dovecot] Developing new Dspam Plugin
by Timothy White 10 Aug '06

10 Aug '06

This email /was/ going to be a bit of a bitch about trying to get mysql compiled in to a dovecot plugin, but I got it all working before I finished the email! :-D Ok, basically what I am doing (Johannes permission) is modifying the dspam plugin, so that it instead talks to a database (MySQL in this case), and updates a table, so that you can do the actual training at a time of lower load. It /does/ require MySQL >= 5 cause I am using stored PROCEDURE's, which gives you the advantage, that you can use the same plugin, without changing anything, regardless of what your database looks like, or how you want the database to work! :-D I have successfully modified Johannes plugin to call SPAM and HAM in the database, with the signatures, and also upgraded it to work with beta9. I also made the small change of a define for the Spam folder, so that if your Spam folder is named differently, you don't have to change the code in 2-3 places as well ;-) I am yet to write any 'batch' code for processing the database tables, but that is fairly trivial, and I could even write it in php if I wanted ;-) (If you want something to do, go ahead and write me a C client to do this! I'll get to it eventually, and you can reuse most of the code from the plugin!) See http://members.plug.org.au/~linuxalien/dokuwiki/projects:dovecot-mysql-dspa… for more details, and the code! Feed back, comments, and criticism are all welcome! Tim -- Linux Counter user #273956

5 12

[Dovecot] Complex MySQL query in dovecot-sql.conf
by Timo Neuvonen 10 Aug '06

10 Aug '06

Is there some limit for the maximum length of the line in dovecot-sql.conf? In particular, I would be writing a long MySQL password query. And further, can a long line be somehow split to several pieces? I tried the common unix-style way of putting backslash as the last character of the first line, but it results in an error immediatedly while restarting Dovecot (while parsing the conf, ie. not a MySQL error): (in conf:) password_query = SELECT user, password FROM users\ WHERE user='%u' AND active='Y' (results in log, line 88 refers to line starting with 'WHERE':) auth(default): Error in configuration file /etc/dovecot-mysql.conf line 88: Expecting '=' The example above is just the beginning, what comes to my question about a long line. I would be checking %r (remote ip) against user-specific allowed ip-address ranges given in the database. Actually, I'd like to use some MySQL user-defined variables within my query, for example to keep only the 3 highest octets of the IPv4 representation of %r (eg. '192.168.0') so I could easily check against allowed /24 networks stored in the database. That seems to run into another problem. A very simplified example what happens when I try to define a variable within the query: (in conf, split to two lines for news only:) password_query = SET @yes='Y'; SELECT user, password FROM users WHERE user='%u' AND active=@yes (results in log:) Password query failed: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '; SELECT user, password FROM users WHERE user='testi' AND active=@yes' at line 1 Anyway, a similar one-line construct works from the MySQL command line: SET @yes='Y'; SELECT user, password FROM users WHERE user='testi' AND active=@yes; (reults the following:) +-------+----------+ | user | password | +-------+----------+ | testi | 123 | +-------+----------+ Any ideas? Am I doing something in a completely wrong way? Of course, split lines & variables are not absolutely necessary, they just would improve readability & shorten the query. But if there is no limit on the length of the query, I propably can write everything as a one long SELECT command... I'm trying all this on Fedora Core 5, having - dovecot-1.0-0.beta8.2.fc5 - mysql-5.0.22-1.FC5.1 Regards, Timo Neuvonen

2 1

[Dovecot] My view on 1.0 release and version numbering
by Marc Perkel 10 Aug '06

10 Aug '06

Just throwing out some thoughts here. I know Timo has high standards and is trying to make 1.0 bug free before calling it 1.0. But I want to argue for an earlier less perfect 1.0. Dovecot is already way past the quality of most 1.0 releases. So it is good enough by common standards. No one really expects a 1.0 release to be perfect. But - some people don't consider a product ripe until it reaches 1.0. Once you cross the line then more people will pick it up and you will get more bug reports because of a larger base. This would actually accelerate the debugging process. And it will spur acceptance of the product and perhaps bring in more developer resources. Generally 1.0 is still buggy. It is usually followed quickly with 1.0.1, 1.0.2 ... and then stabilizes. Then as more people ask for new features it spawns 1.1.0 which is somewhat buggy and then 1.1.1 and by 1.1.2 it's rock solid. So - I'm not suggesting Timo go to 1.0 immediately. I'm just saying that I think it's pretty ripe and sooner is better than later. My 2 centz ...

14 20

[Dovecot] Dovecot under daemontools
by dale gallagher 10 Aug '06

10 Aug '06

Hi there I'm having difficulties in getting dovecot to run under daemontools. Also, from a sysadmin perspective, so far, I find dovecot's config rather complex - I suppose that's what happens when one attempts to appease all authentication methods... anyway, I digress. System details, to set the scene: OS: Slackware Linux Linux Kernel: 2.4.29 GCC: 3.2.2 Dovecot version: 1.0.rc6 Dovecot configure options: ./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var --with-checkpassword --without-passwd --without-passwd-file --without-shadow --without-pam --without-bsdauth --without-gssapi --without-ldap --without-vpopmail --with-prefetch-userdb --without-sql --without-pgsql --without-mysql --with-ssl=openssl --without-pop3d --with-storages=maildir --without-deliver MTA: qmail Storage: Maildirs under a single virtual user (vmail) Checkpassword: /var/qmail/bin/qmail-vpoplogin (modified cdb checkpassword) User DB: /var/qmail/users/vpasswd.cdb /etc/dovecot.conf base_dir = /var/run/dovecot protocols = imap listen = 127.0.0.1 ssl_disable = yes shutdown_clients = yes log_path = /var/log/dovecot login_dir = /var/run/dovecot/login maildir_copy_with_hardlinks = yes protocol imap { # only require access from webmail on same host listen = 127.0.0.1:143 } auth default { passdb checkpassword { args = /var/qmail/bin/qmail-vpoplogin } userdb prefetch { } # user who can access user/passwd db user = auth } Firstly, I cannot run dovecot as an unprivileged user, as the OS moans that the user doesn't have permission to write to /dev/stderr. I've had a similar issue running Clamav, but a patch to Clamav resolved this? This seems to be related to referencing /dev/stderr directly. Any ideas? Should I be running dovecot as the virtual mailbox user vmail? Before I ramble on about what I've tried, could someone explain to me what dovecot actually does, in terms of privilege separation? The online docs are somewhat convoluted. This would help shed light on the solution. I also noticed the posts re daemontools some time ago, yet they do not include any setup solutions etc - I'd appreciate details from those of you who've succeeded in this regard. Thanks Dale

9 29

[Dovecot] userdb prefetch problem..
by Mike 10 Aug '06

10 Aug '06

I have a setup that I'm working on migrating to Dovecot for thousands of users across a few dozen domains.. All users are authed from LDAP.. In LDAP, each user has a unique uidNumber, and varying gidNumber (for use in other applications) As far as mail storage, the entire Maildir structure is owned by UID/GID: vmail/vmail So, the uidNumber/gidNumber from LDAP is not needed. global uid/gid is set to 'vmail' When using "passdb ldap" and "userdb ldap" both pointing to dovecot-ldap.conf, and removing the "uidNumber=uid"..etc from the "_attrs" lines, this works just fine. I would like to use "userdb prefetch" to use one less ldap query, but the uid/gid stuff is giving me some problems. The appropriate line in - dovecot-ldap.conf: pass_attrs = mail=user,userPassword=password,homeDirectory=userdb_home I get the following erros in the log: .... .... dovecot: auth(default): prefetch(user(a)domain.com,127.0.0.1): userdb_uid not returned dovecot: auth(default): prefetch(user(a)domain.com,127.0.0.1): userdb_gid not returned dovecot: auth(default): userdb(user(a)domain.com,127.0.0.1): user not found from userdb dovecot: auth(default): master out: NOTFOUND 1 dovecot: pop3-login: Internal login failure: user=<user(a)domain.com>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Any help is appreciated! -Mike FreeBSD 6-1-RELEASE OpenLDAP 2.3 Dovecot 1.0rc2

3 2

[Dovecot] 1.0 RC1 status
by Timo Sirainen 10 Aug '06

10 Aug '06

Today I finally caught up with my mail. I've now read all the mails from the Dovecot list from this year and all my private mails from last few years (totaling around 500 unread mails). If you think there's still something I should have replied to, please re-send it because I've either missed it or just skipped it for some reason. (Except there are still a couple of mails from this week I've yet to reply to.) Besides reading mails I've also fixed a lot of bugs today. What's left from my original 1.0 roadmap is: - Solaris sendfile is broken? Is it? Can someone try? You can test it with maildir by setting mail_save_crlf=yes, saving a somewhat large mail and FETCHing it (and truss that it really uses sendfile) - LDA: empty mail gives an error. - quota code should probably be always doing some quota_set_critical() instead of using mail_storage_set_critical(), so that quota_last_error() would work properly - rfc2557 support for BODYSTRUCTURE, as specified by RFC3501 - keywords: add some limits to how many there can be - don't return \* in PERMANENTFLAGS when we're full - mbox: how well does dirty sync + status work? it reads the last mail every time? not very good.. - mbox: dirty state should be stored to index (with mbox_very_dirty_syncs) I think 1.0 could be released without any of those changes. But what about the connection hangs that some people have had? Do they still occur? Is it only with SSL? I guess I should look at my previous ssl-proxy rewrite, figure out what went wrong with it and try again. I'm thinking about releasing 1.0 RC1 within a few days. Would be nice if that release actually worked, so testing would be appreciated :) You can try it by getting the dovecot-latest from http://dovecot.org/nightly/ The current changes include: * PAM: If user's password is expired, give "Password expired" error message to the user. Now actually working thanks to Vaidas Pilkauskas * Relicensed dovecot-auth, lib-sql and lib-ntlm to MIT license. See COPYING file for more information. * Abuse prevention: When creating a mailbox, limit the number of hierarchies (up to 20) and the length of the mailbox name within a hierarchy (up to 200 characters). * mbox: If saved mail doesn't end with LF, add it ourself so that the mails always have one empty line before the next From-line. + Added --with-statedir configure option which defaults to $localstatedir/lib/dovecot. ssl-parameters.dat is permanently stored in that directory and is copied to login_dirs from there. + IMAP: Support SASL-IR extension (SASL initial response) + Support initial SASL response with LOGIN mechanism. Patch by Anders Karlsson + Added PLAIN-MD4 password scheme. Patch by Andrey Panin. + Added support for XFS disk quotas. Patch by Pawel Jarosz + If another process deletes the opened mailbox, try to handle it without writing errors to log file. Handles the most common cases. + Added TLS support for LDAP if the library supports it. - Dovecot corrupted mbox files which had CR+LF linefeeds in headers - MySQL code could have crashed while escaping strings - MD4 code with NTLM authentication was broken with 64bit systems. Patch by Andrey Panin - Plugin loading was broken in some OSes (eg. FreeBSD) - Several fixes to handling empty values in configuration file - Several fixes to dictionary quota backend and dict server. Also changed how they're configured. - deliver: Fixed plugin handling settings - mbox_min_index_size handling was somewhat broken - passdb passwd-file: extra_args field wasn't read unless the file was also used as userdb.

13 36