- dovecot - dovecot.org

Re: [Dovecot] Dovecot deliver logging problem and procmail
by Gerhard Wiesinger 11 Aug '06

11 Aug '06

Hello Timo! Any ideas why it may take so long time to open big mbox files with indexes and delivered through dovecot deliver? So it looks like that something is wrong with the indexes or the algorithm. For details have a look below. BTW: procmail filtering through deliver works well now for me in production environment. I'll make some code cleanups and configuration options. Maybe I can release the patch today. Ciao, Gerhard On Mon, 3 Jul 2006, Timo Sirainen wrote: > On Jul 3, 2006, at 11:07 PM, Gerhard Wiesinger wrote: > >> As far as I saw dovecot's deliver process updates some header fields and the >> index files: >> Status: >> X-Keywords: (with a lot of spaces, why?) >> Content-Length: 6 > > The extra spaces are there so that if Dovecot needs to update some headers it > can take the needed space from there so it doesn't have to move the whole > mbox. > Yes, I saw it after I wrote the mail ... >> After access with dovecot the following header fields change: >> Status: O >> X-IMAPbase: 115195111 0000000009 >> X-UID: 2 >> >> So basically this isn't an advantage over the procmail or any other >> delivering process top mbox. The problem is that after the first access the >> X-UID and X-IMAPbase header must be added, so on large mbox files this might >> take a long time. > > Well, first of all Dovecot should take the space it needs from the X-Keywords > spaces, so it doesn't need to write all that much data. > > Second, the X-UID etc. headers are added by deliver if the mbox file is fully > synced against the indexes. So after you had opened it once with IMAP, after > that deliver should write the X-UID headers also as long as you don't use > something else than Dovecot to modify the mbox. Ok, that worked well. I tried to deliver some large mails (~50MB) which worked well. After sending around 500MB (10 Mails) I opened the mailbox with pine over imap. It still took a lot of time (>20s) to open the mbox. Indexes should be up2date, so why does it take so long time? After opening a second time everything is fine. Any ideas? Ciao, Gerhard

2 2

Re: [Dovecot] imap-login (proxy) Connection queue full
by Bill Boebel 11 Aug '06

11 Aug '06

On Thu, August 10, 2006 5:22 pm, Timo Sirainen <tss(a)iki.fi> said: > On Thu, 2006-08-10 at 16:28 -0400, Bill Boebel wrote: >> When running Dovecot (rc6 CVS dovecot-20060808) as a proxy, why would it log the >> following when it has not yet maxed out the allowed number of imap-login >> processes? >> >> Aug 10 16:23:34 director5 dovecot: imap-login: Disconnected: Connection queue >> full: >> >> I am seeing this logged several times per minute. From netstat, the server has >> 650 proxied IMAP connections and 450 proxied IMAPS connections, there are >> currently 10 imap-login processes running, and using the following config: > > I did some changes to CVS which hopefully should fix this? Cool, thanks. I will try this tonight. Bill

1 0

[Dovecot] 1.0.rc2 released
by Timo Sirainen 11 Aug '06

11 Aug '06

Final 1.0 release is looking to get closer. There's nothing left in my TODO list for it, so please report any old and new bugs you see. This release also makes upgrades from 0.99 more transparent with automatic renaming of .subscriptions and .customflags files with maildir. http://dovecot.org/releases/dovecot-1.0.rc2.tar.gz http://dovecot.org/releases/dovecot-1.0.rc2.tar.gz.sig * disable_plaintext_auth=yes: Removed hardcoded 127.* and ::1 IP checks. Now we just assume that the connection is secure if the local IP matches the remote IP address. * SSL code rewrite which hopefully makes it work better than before. Seems to work correctly, but if you suddently have trouble with SSL connections this is likely the reason. + verbose_ssl=yes: Log also SSL alerts and BIO errors - If namespace's location field wasn't set, the default location was supposed to be used but it wasn't. - When copying ssl-parameters.dat file from /var/lib to /var/run its permissions went wrong if it couldn't be copied with hard linking. - Fixed filesystem quota plugin to work with BSDs. - Maildir: Saving mails didn't work if quota plugin was enabled (again) - Maildir: Messages' received time wasn't saved properly when saving/copying multiple messages at a time. Also if using quota plugin the S= size was only set for the first saved file, and even that was wrong. - passdb passwd-file: Don't require valid uid/gid fields if file isn't also being used as a userdb. - PostgreSQL: Handle failures better so that there won't be "invalid fd" errors in logs. - Don't try to expunge messages if the mailbox is read-only. It'll just cause our index files to go out of sync with the real mailbox and cause errors. - ANONYMOUS authentication mechanism couldn't work because anonymous_username setting wasn't passed from master process.

9 15

[Dovecot] imap-login (proxy) Connection queue full
by Bill Boebel 11 Aug '06

11 Aug '06

When running Dovecot (rc6 CVS dovecot-20060808) as a proxy, why would it log the following when it has not yet maxed out the allowed number of imap-login processes? Aug 10 16:23:34 director5 dovecot: imap-login: Disconnected: Connection queue full: I am seeing this logged several times per minute. From netstat, the server has 650 proxied IMAP connections and 450 proxied IMAPS connections, there are currently 10 imap-login processes running, and using the following config: login_process_per_connection = no login_processes_count = 3 login_max_processes_count = 20 login_max_connections = 225 I would think that under these conditions we should never see "Connection queue full" being logged. Thanks, Bill

2 1

[Dovecot] OpenLDAP only?
by ssoo＠siliconfile.com 10 Aug '06

10 Aug '06

I tried to compile Dovecot v1.0RC2 on Opeteron/Solaris 10 box. "db-ldap.c", line 343: undefined symbol: LDAP_SASL_QUIET "LDAP_SASL_QUIET" is defined as "2U" in OpenLDAP's ldap.h but not in /usr/include/ldap.h How can I compile Dovecot with Sun's LDAP implementation?

3 3

[Dovecot] maildir++ specification compliance.
by Apps Lists 10 Aug '06

10 Aug '06

Hi folks. I was doing some testing this morning and realized Dovecot RC2 isn't fully compliant with the maildir++ specification. Most notably, it appears that Dovecot isn't creating a "maildirfolder" file (0 byte length) in folders it creates. >From http://www.inter7.com/courierimap/README.maildirquota.html "Within each subdirectory there's an empty file, maildirfolder. Its existence tells the mail delivery agent that this Maildir is a really a folder underneath a parent Maildir++." and "Make sure to create the maildirfolder file in any new folders created within the Maildir++." Also, GETQUOTAROOT on a folder (with messages in it) will always return the entire usage, not the usage in the folder itself. For example: INBOX - really has 100 mb of messages. Sent - has a few messages totalling 1mb. Drafts - no messages A GETQUOTAROOT on INBOX will return 101 mb, 100mb for INBOX and 1mb for Sent. Drafts will show zero, because there's nothing. Unfortunately, Sent will also show 101 mb, not 1mb. While Dovecot shouldn't be creating maildirsize files in each folder (the maildirfolder file is there to tell clients to look in .. for a maildirsize) ... it should still be able to answer queries about the number of messages (count of files in the folder) as well as the total size of all messages in that folder (sum of S=xxxx or by stat() on individual message files). Steve

4 4

[Dovecot] Developing new Dspam Plugin
by Timothy White 10 Aug '06

10 Aug '06

This email /was/ going to be a bit of a bitch about trying to get mysql compiled in to a dovecot plugin, but I got it all working before I finished the email! :-D Ok, basically what I am doing (Johannes permission) is modifying the dspam plugin, so that it instead talks to a database (MySQL in this case), and updates a table, so that you can do the actual training at a time of lower load. It /does/ require MySQL >= 5 cause I am using stored PROCEDURE's, which gives you the advantage, that you can use the same plugin, without changing anything, regardless of what your database looks like, or how you want the database to work! :-D I have successfully modified Johannes plugin to call SPAM and HAM in the database, with the signatures, and also upgraded it to work with beta9. I also made the small change of a define for the Spam folder, so that if your Spam folder is named differently, you don't have to change the code in 2-3 places as well ;-) I am yet to write any 'batch' code for processing the database tables, but that is fairly trivial, and I could even write it in php if I wanted ;-) (If you want something to do, go ahead and write me a C client to do this! I'll get to it eventually, and you can reuse most of the code from the plugin!) See http://members.plug.org.au/~linuxalien/dokuwiki/projects:dovecot-mysql-dspa… for more details, and the code! Feed back, comments, and criticism are all welcome! Tim -- Linux Counter user #273956

5 12

[Dovecot] Complex MySQL query in dovecot-sql.conf
by Timo Neuvonen 10 Aug '06

10 Aug '06

Is there some limit for the maximum length of the line in dovecot-sql.conf? In particular, I would be writing a long MySQL password query. And further, can a long line be somehow split to several pieces? I tried the common unix-style way of putting backslash as the last character of the first line, but it results in an error immediatedly while restarting Dovecot (while parsing the conf, ie. not a MySQL error): (in conf:) password_query = SELECT user, password FROM users\ WHERE user='%u' AND active='Y' (results in log, line 88 refers to line starting with 'WHERE':) auth(default): Error in configuration file /etc/dovecot-mysql.conf line 88: Expecting '=' The example above is just the beginning, what comes to my question about a long line. I would be checking %r (remote ip) against user-specific allowed ip-address ranges given in the database. Actually, I'd like to use some MySQL user-defined variables within my query, for example to keep only the 3 highest octets of the IPv4 representation of %r (eg. '192.168.0') so I could easily check against allowed /24 networks stored in the database. That seems to run into another problem. A very simplified example what happens when I try to define a variable within the query: (in conf, split to two lines for news only:) password_query = SET @yes='Y'; SELECT user, password FROM users WHERE user='%u' AND active=@yes (results in log:) Password query failed: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '; SELECT user, password FROM users WHERE user='testi' AND active=@yes' at line 1 Anyway, a similar one-line construct works from the MySQL command line: SET @yes='Y'; SELECT user, password FROM users WHERE user='testi' AND active=@yes; (reults the following:) +-------+----------+ | user | password | +-------+----------+ | testi | 123 | +-------+----------+ Any ideas? Am I doing something in a completely wrong way? Of course, split lines & variables are not absolutely necessary, they just would improve readability & shorten the query. But if there is no limit on the length of the query, I propably can write everything as a one long SELECT command... I'm trying all this on Fedora Core 5, having - dovecot-1.0-0.beta8.2.fc5 - mysql-5.0.22-1.FC5.1 Regards, Timo Neuvonen

2 1

[Dovecot] My view on 1.0 release and version numbering
by Marc Perkel 10 Aug '06

10 Aug '06

Just throwing out some thoughts here. I know Timo has high standards and is trying to make 1.0 bug free before calling it 1.0. But I want to argue for an earlier less perfect 1.0. Dovecot is already way past the quality of most 1.0 releases. So it is good enough by common standards. No one really expects a 1.0 release to be perfect. But - some people don't consider a product ripe until it reaches 1.0. Once you cross the line then more people will pick it up and you will get more bug reports because of a larger base. This would actually accelerate the debugging process. And it will spur acceptance of the product and perhaps bring in more developer resources. Generally 1.0 is still buggy. It is usually followed quickly with 1.0.1, 1.0.2 ... and then stabilizes. Then as more people ask for new features it spawns 1.1.0 which is somewhat buggy and then 1.1.1 and by 1.1.2 it's rock solid. So - I'm not suggesting Timo go to 1.0 immediately. I'm just saying that I think it's pretty ripe and sooner is better than later. My 2 centz ...

14 20

[Dovecot] Dovecot under daemontools
by dale gallagher 10 Aug '06

10 Aug '06

Hi there I'm having difficulties in getting dovecot to run under daemontools. Also, from a sysadmin perspective, so far, I find dovecot's config rather complex - I suppose that's what happens when one attempts to appease all authentication methods... anyway, I digress. System details, to set the scene: OS: Slackware Linux Linux Kernel: 2.4.29 GCC: 3.2.2 Dovecot version: 1.0.rc6 Dovecot configure options: ./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var --with-checkpassword --without-passwd --without-passwd-file --without-shadow --without-pam --without-bsdauth --without-gssapi --without-ldap --without-vpopmail --with-prefetch-userdb --without-sql --without-pgsql --without-mysql --with-ssl=openssl --without-pop3d --with-storages=maildir --without-deliver MTA: qmail Storage: Maildirs under a single virtual user (vmail) Checkpassword: /var/qmail/bin/qmail-vpoplogin (modified cdb checkpassword) User DB: /var/qmail/users/vpasswd.cdb /etc/dovecot.conf base_dir = /var/run/dovecot protocols = imap listen = 127.0.0.1 ssl_disable = yes shutdown_clients = yes log_path = /var/log/dovecot login_dir = /var/run/dovecot/login maildir_copy_with_hardlinks = yes protocol imap { # only require access from webmail on same host listen = 127.0.0.1:143 } auth default { passdb checkpassword { args = /var/qmail/bin/qmail-vpoplogin } userdb prefetch { } # user who can access user/passwd db user = auth } Firstly, I cannot run dovecot as an unprivileged user, as the OS moans that the user doesn't have permission to write to /dev/stderr. I've had a similar issue running Clamav, but a patch to Clamav resolved this? This seems to be related to referencing /dev/stderr directly. Any ideas? Should I be running dovecot as the virtual mailbox user vmail? Before I ramble on about what I've tried, could someone explain to me what dovecot actually does, in terms of privilege separation? The online docs are somewhat convoluted. This would help shed light on the solution. I also noticed the posts re daemontools some time ago, yet they do not include any setup solutions etc - I'd appreciate details from those of you who've succeeded in this regard. Thanks Dale

9 29