July 2006 - dovecot - dovecot.org

[Dovecot] Developing new Dspam Plugin
by Timothy White 10 Aug '06

10 Aug '06

This email /was/ going to be a bit of a bitch about trying to get mysql compiled in to a dovecot plugin, but I got it all working before I finished the email! :-D Ok, basically what I am doing (Johannes permission) is modifying the dspam plugin, so that it instead talks to a database (MySQL in this case), and updates a table, so that you can do the actual training at a time of lower load. It /does/ require MySQL >= 5 cause I am using stored PROCEDURE's, which gives you the advantage, that you can use the same plugin, without changing anything, regardless of what your database looks like, or how you want the database to work! :-D I have successfully modified Johannes plugin to call SPAM and HAM in the database, with the signatures, and also upgraded it to work with beta9. I also made the small change of a define for the Spam folder, so that if your Spam folder is named differently, you don't have to change the code in 2-3 places as well ;-) I am yet to write any 'batch' code for processing the database tables, but that is fairly trivial, and I could even write it in php if I wanted ;-) (If you want something to do, go ahead and write me a C client to do this! I'll get to it eventually, and you can reuse most of the code from the plugin!) See http://members.plug.org.au/~linuxalien/dokuwiki/projects:dovecot-mysql-dspa… for more details, and the code! Feed back, comments, and criticism are all welcome! Tim -- Linux Counter user #273956

5 12

[Dovecot] Complex MySQL query in dovecot-sql.conf
by Timo Neuvonen 10 Aug '06

10 Aug '06

Is there some limit for the maximum length of the line in dovecot-sql.conf? In particular, I would be writing a long MySQL password query. And further, can a long line be somehow split to several pieces? I tried the common unix-style way of putting backslash as the last character of the first line, but it results in an error immediatedly while restarting Dovecot (while parsing the conf, ie. not a MySQL error): (in conf:) password_query = SELECT user, password FROM users\ WHERE user='%u' AND active='Y' (results in log, line 88 refers to line starting with 'WHERE':) auth(default): Error in configuration file /etc/dovecot-mysql.conf line 88: Expecting '=' The example above is just the beginning, what comes to my question about a long line. I would be checking %r (remote ip) against user-specific allowed ip-address ranges given in the database. Actually, I'd like to use some MySQL user-defined variables within my query, for example to keep only the 3 highest octets of the IPv4 representation of %r (eg. '192.168.0') so I could easily check against allowed /24 networks stored in the database. That seems to run into another problem. A very simplified example what happens when I try to define a variable within the query: (in conf, split to two lines for news only:) password_query = SET @yes='Y'; SELECT user, password FROM users WHERE user='%u' AND active=@yes (results in log:) Password query failed: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '; SELECT user, password FROM users WHERE user='testi' AND active=@yes' at line 1 Anyway, a similar one-line construct works from the MySQL command line: SET @yes='Y'; SELECT user, password FROM users WHERE user='testi' AND active=@yes; (reults the following:) +-------+----------+ | user | password | +-------+----------+ | testi | 123 | +-------+----------+ Any ideas? Am I doing something in a completely wrong way? Of course, split lines & variables are not absolutely necessary, they just would improve readability & shorten the query. But if there is no limit on the length of the query, I propably can write everything as a one long SELECT command... I'm trying all this on Fedora Core 5, having - dovecot-1.0-0.beta8.2.fc5 - mysql-5.0.22-1.FC5.1 Regards, Timo Neuvonen

2 1

[Dovecot] My view on 1.0 release and version numbering
by Marc Perkel 10 Aug '06

10 Aug '06

Just throwing out some thoughts here. I know Timo has high standards and is trying to make 1.0 bug free before calling it 1.0. But I want to argue for an earlier less perfect 1.0. Dovecot is already way past the quality of most 1.0 releases. So it is good enough by common standards. No one really expects a 1.0 release to be perfect. But - some people don't consider a product ripe until it reaches 1.0. Once you cross the line then more people will pick it up and you will get more bug reports because of a larger base. This would actually accelerate the debugging process. And it will spur acceptance of the product and perhaps bring in more developer resources. Generally 1.0 is still buggy. It is usually followed quickly with 1.0.1, 1.0.2 ... and then stabilizes. Then as more people ask for new features it spawns 1.1.0 which is somewhat buggy and then 1.1.1 and by 1.1.2 it's rock solid. So - I'm not suggesting Timo go to 1.0 immediately. I'm just saying that I think it's pretty ripe and sooner is better than later. My 2 centz ...

14 20

[Dovecot] userdb prefetch problem..
by Mike 10 Aug '06

10 Aug '06

I have a setup that I'm working on migrating to Dovecot for thousands of users across a few dozen domains.. All users are authed from LDAP.. In LDAP, each user has a unique uidNumber, and varying gidNumber (for use in other applications) As far as mail storage, the entire Maildir structure is owned by UID/GID: vmail/vmail So, the uidNumber/gidNumber from LDAP is not needed. global uid/gid is set to 'vmail' When using "passdb ldap" and "userdb ldap" both pointing to dovecot-ldap.conf, and removing the "uidNumber=uid"..etc from the "_attrs" lines, this works just fine. I would like to use "userdb prefetch" to use one less ldap query, but the uid/gid stuff is giving me some problems. The appropriate line in - dovecot-ldap.conf: pass_attrs = mail=user,userPassword=password,homeDirectory=userdb_home I get the following erros in the log: .... .... dovecot: auth(default): prefetch(user(a)domain.com,127.0.0.1): userdb_uid not returned dovecot: auth(default): prefetch(user(a)domain.com,127.0.0.1): userdb_gid not returned dovecot: auth(default): userdb(user(a)domain.com,127.0.0.1): user not found from userdb dovecot: auth(default): master out: NOTFOUND 1 dovecot: pop3-login: Internal login failure: user=<user(a)domain.com>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Any help is appreciated! -Mike FreeBSD 6-1-RELEASE OpenLDAP 2.3 Dovecot 1.0rc2

3 2

[Dovecot] 1.0 RC1 status
by Timo Sirainen 10 Aug '06

10 Aug '06

Today I finally caught up with my mail. I've now read all the mails from the Dovecot list from this year and all my private mails from last few years (totaling around 500 unread mails). If you think there's still something I should have replied to, please re-send it because I've either missed it or just skipped it for some reason. (Except there are still a couple of mails from this week I've yet to reply to.) Besides reading mails I've also fixed a lot of bugs today. What's left from my original 1.0 roadmap is: - Solaris sendfile is broken? Is it? Can someone try? You can test it with maildir by setting mail_save_crlf=yes, saving a somewhat large mail and FETCHing it (and truss that it really uses sendfile) - LDA: empty mail gives an error. - quota code should probably be always doing some quota_set_critical() instead of using mail_storage_set_critical(), so that quota_last_error() would work properly - rfc2557 support for BODYSTRUCTURE, as specified by RFC3501 - keywords: add some limits to how many there can be - don't return \* in PERMANENTFLAGS when we're full - mbox: how well does dirty sync + status work? it reads the last mail every time? not very good.. - mbox: dirty state should be stored to index (with mbox_very_dirty_syncs) I think 1.0 could be released without any of those changes. But what about the connection hangs that some people have had? Do they still occur? Is it only with SSL? I guess I should look at my previous ssl-proxy rewrite, figure out what went wrong with it and try again. I'm thinking about releasing 1.0 RC1 within a few days. Would be nice if that release actually worked, so testing would be appreciated :) You can try it by getting the dovecot-latest from http://dovecot.org/nightly/ The current changes include: * PAM: If user's password is expired, give "Password expired" error message to the user. Now actually working thanks to Vaidas Pilkauskas * Relicensed dovecot-auth, lib-sql and lib-ntlm to MIT license. See COPYING file for more information. * Abuse prevention: When creating a mailbox, limit the number of hierarchies (up to 20) and the length of the mailbox name within a hierarchy (up to 200 characters). * mbox: If saved mail doesn't end with LF, add it ourself so that the mails always have one empty line before the next From-line. + Added --with-statedir configure option which defaults to $localstatedir/lib/dovecot. ssl-parameters.dat is permanently stored in that directory and is copied to login_dirs from there. + IMAP: Support SASL-IR extension (SASL initial response) + Support initial SASL response with LOGIN mechanism. Patch by Anders Karlsson + Added PLAIN-MD4 password scheme. Patch by Andrey Panin. + Added support for XFS disk quotas. Patch by Pawel Jarosz + If another process deletes the opened mailbox, try to handle it without writing errors to log file. Handles the most common cases. + Added TLS support for LDAP if the library supports it. - Dovecot corrupted mbox files which had CR+LF linefeeds in headers - MySQL code could have crashed while escaping strings - MD4 code with NTLM authentication was broken with 64bit systems. Patch by Andrey Panin - Plugin loading was broken in some OSes (eg. FreeBSD) - Several fixes to handling empty values in configuration file - Several fixes to dictionary quota backend and dict server. Also changed how they're configured. - deliver: Fixed plugin handling settings - mbox_min_index_size handling was somewhat broken - passdb passwd-file: extra_args field wasn't read unless the file was also used as userdb.

13 36

[Dovecot] log error file ioloop.c assertion failed
by Dominique Feyer 10 Aug '06

10 Aug '06

pop3-login process have some error like that log: dovecot: pop3-login: file ioloop.c: line 22 (io_add): assertion failed: (fd >= 0) dovecot: child 4648 (login) killed with signal 6 Any idea ? Thanks -- Net4all S.A. Dominique Feyer Administrateur Système Ch. de la Colline 5bis CH-1007 Lausanne dfeyer(a)net4all.ch

2 1

[Dovecot] Dovecot and MySQL auth - initial setup issues
by Andrew Hearn 10 Aug '06

10 Aug '06

Hello, I'm just setting up a new server and wanting to use MySQL for user authentication. Here is what I have: Fedora Core 5: 2.6.17-1.2145_FC5 Dovecot v1.0.rc2 (RPM install from atrpms.net) MySQL v5.0.22 SquirrelMail v 1.4.6-7.fc5 ...Running on a Dual XEON machine I've added the following to dovecot.conf: auth default { passdb sql { args = /etc/dovecot-mysql.conf } userdb sql { args = /etc/dovecot-mysql.conf } The rest of dovecot.conf is standard. /etc/dovecot-mysql.conf has: driver=mysql connect = host=127.0.0.1 port=3306 dbname=maildb user=root client_flags=0 password_query = SELECT password FROM users WHERE id = '%u' user_query = SELECT maildir FROM users WHERE id = '%u' (I've run the SQL queries within mysql and they give the right results) When I start Dovecot the log shows: dovecot: Jul 11 12:47:25 Info: Dovecot v1.0.rc2 starting up dovecot: Jul 11 12:47:27 Info: auth-worker(default): mysql: Connected to 127.0.0.1 (maildb) When I try to log in using SquirrelMail, it takes about 3 seconds, then I get dovecot: Jul 11 12:48:00 Info: imap-login: Aborted login: user=<andrew(a)mail-test.aaisp.net.uk>, method=PLAIN, rip=::ffff:127.0.0.1, lip=::ffff:127.0.0.1, secured ...So, no log of it trying MySQL... And SquirrelMail then says: Unknown user or password incorrect Any ideas of what I'm missing? Thanks! Andrew. -- Andrew Hearn, Senior Support Engineer Andrews & Arnold Ltd 01344 400999 For efficient support, please email: support(a)aaisp.net.uk

4 3

[Dovecot] Corrupted Indexes
by Joseph W. Breu 10 Aug '06

10 Aug '06

What are the reprecussions of a corrupted index? Jul 11 08:47:23 pop1.cfu.net dovecot: IMAP(breu(a)cfu.net): Corrupted index cache file /var/spool/mail/filer/storage//cfu.net/b/r/breu/.lists.horde-imp/dovecot.index.cache: field header names corrupted Jul 11 13:22:06 pop6.cfu.net dovecot: IMAP(breu(a)cfu.net): Corrupted index cache file /var/spool/mail/filer/storage//cfu.net/b/r/breu/dovecot.index.cache: invalid record size Thanks, --------------------------------------------------------- Joseph W. Breu, CCNA phone : +1.319.268.5228 Senior Network Administrator fax : +1.319.266.8158 Cedar Falls Utilities cell : +1.319.493.1686 support: +1.319.268.5221 url : http://www.cfu.net

3 2

[Dovecot] no protocols inhibits dovecot --exec-mail
by Johannes Berg 10 Aug '06

10 Aug '06

Hi, Running the current debian unstable version (1.0.beta2) dovecot tells me: $ /usr/sbin/dovecot --exec-mail imap Error: No protocols given in configuration file Could this be changed? I don't need dovecot listening on any ports but would still like to allow this usage... Or should I configure protocols but no listen addresses instead? [if so the debian package needs an update because it filters out the case of no protocols and doesn't start dovecot in that case] johannes

5 11

[Dovecot] Bug#377840: dovecot: imap segfaults on small mbox files (2 bytes)
by Roland Stigge 10 Aug '06

10 Aug '06

Package: dovecot Severity: important Tags: patch Hi, a recent addition in src/lib-storage/index/mbox/istream-raw-mbox.c: ======================================================================================== [...] if (pos == 2) { /* we're at the end of file with CR+LF linefeeds? need more data to verify it. */ rstream->input_peak_offset = stream->istream.v_offset + pos; return _read(stream); } [...] ======================================================================================== makes my imap process crash, resulting in repeatedly spawned (and killed) processes in my log file: ======================================================================================== [...] Jul 11 15:31:04 localhost dovecot: imap-login: Login: user=<ernie>, method=plain, rip=127.0.0.1, lip=127.0.0.1, secured Jul 11 15:31:04 localhost dovecot: child 31284 (imap) killed with signal 11 Jul 11 15:31:04 localhost dovecot: child 31287 (imap) killed with signal 11 Jul 11 15:31:04 localhost dovecot: imap-login: Login: user=<ernie>, method=plain, rip=127.0.0.1, lip=127.0.0.1, secured Jul 11 15:31:05 localhost dovecot: imap-login: Login: user=<ernie>, method=plain, rip=127.0.0.1, lip=127.0.0.1, secured Jul 11 15:31:05 localhost dovecot: child 31293 (imap) killed with signal 11 Jul 11 15:31:05 localhost dovecot: imap-login: Login: user=<ernie>, method=plain, rip=127.0.0.1, lip=127.0.0.1, secured Jul 11 15:31:05 localhost dovecot: child 31296 (imap) killed with signal 11 Jul 11 15:31:05 localhost dovecot: child 31299 (imap) killed with signal 11 Jul 11 15:31:05 localhost dovecot: imap-login: Login: user=<ernie>, method=plain, rip=127.0.0.1, lip=127.0.0.1, secured Jul 11 15:31:06 localhost dovecot: child 31310 (imap) killed with signal 11 Jul 11 15:31:07 localhost dovecot: imap-login: Login: user=<ernie>, method=plain, rip=127.0.0.1, lip=127.0.0.1, secured Jul 11 15:31:07 localhost dovecot: imap-login: Login: user=<ernie>, method=plain, rip=127.0.0.1, lip=127.0.0.1, secured Jul 11 15:31:07 localhost dovecot: child 31313 (imap) killed with signal 11 Jul 11 15:31:07 localhost dovecot: child 31316 (imap) killed with signal 11 Jul 11 15:31:07 localhost dovecot: imap-login: Login: user=<ernie>, method=plain, rip=127.0.0.1, lip=127.0.0.1, secured [...] ======================================================================================== This is caused by the infinite recursive loop made by the "return _read(stream);" The problem is triggered in my mbox directories because I have .svn/format files lying around (mail under svn revision control). In this case, the file is 2 bytes long: ======================================================================================== ernie@dent:~/Mail$ hexdump -C .svn/format 00000000 34 0a |4.| 00000002 ======================================================================================== The attached patch (guarding against eof) is a reasonable workaround. bye, Roland -- System Information: Debian Release: testing/unstable APT prefers unstable APT policy: (500, 'unstable'), (1, 'experimental') Architecture: i386 (i686) Shell: /bin/sh linked to /bin/bash Kernel: Linux 2.6.17-1-686 Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8) (ignored: LC_ALL set to en_GB.UTF-8)

2 2