- dovecot - dovecot.org

[Dovecot] Dovecot v1.2rc6 dict-server socket path problem & pgsql crash
by Steffen Kaiser 28 Jun '09

28 Jun '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello, I find this in the logs: 2009-06-25 14:10:37 IMAP(dvtest) [29566]: Error: net_connect_unix(/usr/local/dovecot-1.2.rc6acl-hook2/var/run/dovecot/dict-server) failed: Permission denied 2009-06-25 14:10:37 IMAP(dvtest) [29566]: Error: acl: dict iteration failed, can't update dict although base_dir is just /var/run/dovecot/ . Dovecot v1.2 changeset: 9157:d14e8b8df025 Conf is attached. I am trying to get ACL's shared dictionary running: acl_shared_dict: proxy::acl dict: acl: pgsql:/usr/local/dovecot-1.2.rc6acl-hook2/etc/dovecot-dict-acl-sql.conf ============ I created the symlink to get map /var/run/dovecot under $install_dir, then I get a crash: 2009-06-25 14:26:53 dict: Error: pgsql: Connect failed to dovecot: could not open certificate file "/root/.postgresql/postgresql.crt": No such file or directory ==> /var/log/postgresql/postgresql-8.1-main.log <== 2009-06-25 14:26:53 CEST LOG: could not accept SSL connection: peer did not return a certificate ==> /var/log/dovecot/dovecot-1.2.log <== 2009-06-25 14:26:58 dict: Error: pgsql: Connect failed to dovecot: could not open certificate file "/root/.postgresql/postgresql.crt": No such file or directory ==> /var/log/postgresql/postgresql-8.1-main.log <== 2009-06-25 14:26:58 CEST LOG: could not accept SSL connection: peer did not return a certificate ==> /var/log/dovecot/dovecot-1.2.log <== 2009-06-25 14:27:03 dict: Error: dict sql iterate failed: Not connected to database 2009-06-25 14:27:03 IMAP(dvtest) [29801]: Error: read(/usr/local/dovecot-1.2.rc6acl-hook2/var/run/dovecot/dict-server) failed: Remote disconnected 2009-06-25 14:27:03 IMAP(dvtest) [29801]: Error: acl: dict commit failed 2009-06-25 14:27:03 dovecot: Error: child 29802 (dict) killed with signal 11 (core dumps disabled) My sql-dict.conf is: connect = host=localhost dbname=dovecot user=dovecot # password=pass [CUT samples for quota & expire] # CREATE TABLE user_shares ( # from_user varchar(100) not null, # to_user varchar(100) not null, # primary key (from_user, to_user) # ); map { pattern = shared/shared-boxes/user/$to/$from table = user_shares fields { from_user = $from to_user = $to } } ================== So I changed the connect line into: connect = dbname=dovecot user=dovecot into forcing to use the local socket, which seems to work: 2009-06-25 14:37:24 dict: Info: pgsql: Connected to dovecot but: 2009-06-25 14:37:24 dovecot: Error: child 29986 (dict) killed with signal 11 (core dumped) 2009-06-25 14:37:24 IMAP(dvtest) [29985]: Error: read(/usr/local/dovecot-1.2.rc6acl-hook2/var/run/dovecot/dict-server) failed: Remote disconnected 2009-06-25 14:37:24 IMAP(dvtest) [29985]: Error: acl: dict commit failed ==> /var/log/postgresql/postgresql-8.1-main.log <== 2009-06-25 14:37:24 CEST LOG: unexpected EOF on client connection The core's bt full of this latter crash is: Core was generated by `dict'. Program terminated with signal 11, Segmentation fault. #0 0xb7ebe463 in strlen () from /lib/tls/i686/cmov/libc.so.6 (gdb) bt full #0 0xb7ebe463 in strlen () from /lib/tls/i686/cmov/libc.so.6 No symbol table info available. #1 0x08065072 in str_append (str=0x806c168, cstr=0x0) at str.c:91 No locals. #2 0x080509ef in sql_dict_set_query (build=0xbffe8c34) at dict-sql.c:536 dict = (struct sql_dict *) 0x8076e08 sql_fields = <value optimized out> extra_values = <value optimized out> i = 0 prefix = (string_t *) 0x806c168 suffix = (string_t *) 0x806c1d8 __PRETTY_FUNCTION__ = "sql_dict_set_query" #3 0x080518a4 in sql_dict_set (_ctx=0x8080028, key=0x806c032 "shared/shared-boxes/user/dvtest2/dvtest", value=0x806c05a "1") at dict-sql.c:648 build = {dict = 0x8076e08, fields = {arr = {buffer = 0x806c130, element_size = 8}, v = 0x806c130, v_modifiable = 0x806c130}, extra_values = 0xbffe8c50, key1 = 115 's', inc = false} field = {map = 0x8076ed8, value = 0x806c05a "1"} query = <value optimized out> _data_stack_cur_id = 4 ctx = <value optimized out> dict = <value optimized out> map = <value optimized out> values = {arr = {buffer = 0x806c0f0, element_size = 4}, v = 0x806c0f0, v_modifiable = 0x806c0f0} #4 0x08052460 in dict_set (ctx=0x8080028, key=0x806c032 "shared/shared-boxes/user/dvtest2/dvtest", value=0x806c05a "1") at dict.c:168 __PRETTY_FUNCTION__ = "dict_set" #5 0x0804f355 in cmd_set (conn=0x8075bc8, line=0x8075df8 "2\tshared/shared-boxes/user/dvtest2/dvtest\t1") at dict-server.c:288 trans = (struct dict_server_transaction *) 0x8080118 #6 0x0804fbf9 in dict_client_connection_input (conn=0x8075bc8) at dict-server.c:458 _data_stack_cur_id = 3 line = 0x8075df7 "S2\tshared/shared-boxes/user/dvtest2/dvtest\t1" i = 6 ret = <value optimized out> #7 0x0805e2d0 in io_loop_handler_run (ioloop=0x8074a38) at ioloop-epoll.c:208 ctx = <value optimized out> event = (const struct epoll_event *) 0x8074b80 list = (struct io_list *) 0x8075da8 io = (struct io_file *) 0x8075d80 tv = {tv_sec = 2147483, tv_usec = 0} t_id = 2 msecs = <value optimized out> ret = 1 i = 0 j = 0 call = <value optimized out> #8 0x0805d398 in io_loop_run (ioloop=0x8074a38) at ioloop.c:338 No locals. #9 0x0805002f in main () at main.c:129 No locals. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iQEVAwUBSkNxwXWSIuGy1ktrAQKuyAgAn8NU8Y59qGV75VStW2MZQH/7HWLF2z9y 2L41g+NzX5LEw9o/B2ZjlAVEsNhVbqbew2A57uuvOAgV5K8iGgx1yPUjmQT9ud6X uthcsBNQIVGu6ymCN2ufeh3XdNEkrOu1pLJaWkiVQjvc8FOczMp432Jhytc9Hpbo jtMbnFuSbZmg0GX6bqJ50kOsjwme7B7ak/ubVeOwGXHU4MQ8PRXl+TXdVTlnFwKw IUZj8xPB+Gi7nRdsJlkQItp//TgeAJ9lm+i/OWzH420qljADDs9NfrfKd/DDOo6i u6W6I+5FUsJ5N7CHDAvHeDnTHPomPC+bnYAWpRyXF/kMVPaccaFxQw== =O2WZ -----END PGP SIGNATURE-----

2 2

[Dovecot] [PATCH] allow disabling libcap support although it's installed
by Daniel Mierswa 28 Jun '09

28 Jun '09

see subject :) patch is in hg export format -- Mierswa, Daniel If you still don't like it, that's ok: that's why I'm boss. I simply know better than you do. --- Linus Torvalds, comp.os.linux.advocacy, 1996/07/22 # HG changeset patch # User Daniel Mierswa <impulze(a)impulze.org> # Date 1245757676 -7200 # Branch HEAD # Node ID 343390a32894c654c8646650eba2cf130ca480d2 # Parent d77225a6412873528b3c94abe8268436df5dbe82 do not automagically depend on libcap if it's installed diff --git a/configure.in b/configure.in --- a/configure.in +++ b/configure.in @@ -196,6 +196,11 @@ TEST_WITH(bzlib, $withval), want_bzlib=auto) +AC_ARG_WITH(libcap, +[ --with-libcap Build with libcap support (Dropping capabilities).], + TEST_WITH(libcap, $withval), + want_libcap=auto) + AC_ARG_WITH(ssl, [ --with-ssl=gnutls|openssl Build with GNUTLS or OpenSSL (default)], if test x$withval = xno; then @@ -358,11 +363,13 @@ AC_DEFINE(HAVE_FDATASYNC,, Define if you have fdatasync()) ]) -AC_CHECK_LIB(cap, cap_init, [ - AC_DEFINE(HAVE_LIBCAP,, libcap is installed for cap_init()) - LIBCAP="-lcap" -]) -AC_SUBST(LIBCAP) +if test $want_libcap != no; then + AC_CHECK_LIB(cap, cap_init, [ + AC_DEFINE(HAVE_LIBCAP,, libcap is installed for cap_init()) + LIBCAP="-lcap" + ]) + AC_SUBST(LIBCAP) +fi AC_DEFINE(PACKAGE_WEBPAGE, "http://www.dovecot.org/", Support URL)

2 2

[Dovecot] strange quota behaviour with dovecot 1.1.7
by listacc＠gmx.de 28 Jun '09

28 Jun '09

Hello! I´m running dovecot 1.1.7 (which is the most recent binary of openSuSE 11.1 repository) with Postfix, MySQL and Postfixadmin. Now I´m trying for some days to get quota working, but I get some strange behaviour and I was not yet able to figure out where the error in my configuration is. Quota information is inserted by postfixadmin into a MySQL database. It seems that dovecot is reading this information correctly - when a mailbox is filled up and gets over-quota, further emails are rejected. But then, if I want to increase the quota for this filled-up mailbox in postfixadmin, postfixadmin correctly changes the quota information in MySQL table. After growing quota, mails to the account are accepted again. But my Thunderbird with Quota plugin continues showing the old quota setting. When more mails are sent to this account, it shows a quota > 100%. If I look into the "mailboxsize" file of this account, it still shows the old quota value before I raised it. If I delete the entry in mailboxsize, dovecot writes the old value in again. Seems that dovecot isn´t looking in the MySQL table (but it did after creating the account). But why are mails accepted again if dovecot is not recognizing the raised quota?? If I try now, as a user, to delete mails from the "overfilled" Inbox, it is denied with a "quota exceeded" message. But I´m able to send further mails to this account. The only way to get the account working properly again is to delete the messages directly on the server´s filesystem. I´m very grateful for every hint! regards, Andreas ------------- dovecot -n: # 1.1.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.27.23-0.1-default x86_64 openSUSE 11.1 (x86_64) ext3 base_dir: /drbd/mail/var/run/dovecot/ log_path: /var/log/dovecot.err info_log_path: /var/log/dovecot.info protocols: imaps managesieve listen(default): * listen(imap): * listen(managesieve): 192.168.1.8:2000 ssl_listen(default): *:993 ssl_listen(imap): *:993 ssl_listen(managesieve): ssl_cert_file: /etc/ssl/certs/imap.domain.crt ssl_key_file: /etc/ssl/private/imap.domain.key verbose_ssl: yes login_dir: /drbd/mail/var/run/dovecot//login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(managesieve): /usr/lib/dovecot/managesieve-login max_mail_processes: 2000 mail_max_userip_connections(default): 30 mail_max_userip_connections(imap): 30 mail_max_userip_connections(managesieve): 10 first_valid_uid: 5001 last_valid_uid: 5001 mail_location: maildir:/drbd/mail/vmail/%d/%n mail_debug: yes mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(managesieve): /usr/lib/dovecot/managesieve mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(managesieve): mail_plugin_dir(default): /usr/lib64/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib64/dovecot/modules/imap mail_plugin_dir(managesieve): /usr/lib64/dovecot/modules/managesieve managesieve_implementation_string(default): dovecot managesieve_implementation_string(imap): dovecot managesieve_implementation_string(managesieve): Cyrus timsieved v2.2.13 sieve_storage(default): sieve_storage(imap): sieve_storage(managesieve): /drbd/mail/vmail/%d/%n/sieve sieve(default): sieve(imap): sieve(managesieve): /drbd/mail/vmail/%d/%n/dovecot.sieve auth default: mechanisms: plain login cram-md5 user: nobody verbose: yes debug: yes debug_passwords: yes passdb: driver: pam passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: passwd userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 432 user: vmail group: vmail plugin: sieve: /drbd/mail/vmail/%d/%n/dovecot.sieve quota: maildir --- dovecot-sql.conf: (...) password_query = SELECT username AS user, password, '/drbd/mail/vmail/%d/%n' AS userdb_home, 'maildir:/drbd/mail/vmail/%d/%n' AS userdb_mail, 5001 AS userdb_uid, 5001 AS userdb_gid FROM mailbox WHERE username = '%u' AND active = '1' (...) user_query = SELECT '/drbd/mail/vmail/%d/%n' AS home, 'maildir:/drbd/mail/vmail/%d/%n' AS mail, 5001 AS uid, 5001 AS gid, concat('*:storage=', quota, 'B') AS quota_rule FROM mailbox WHERE username = '%u' AND active = '1' --- mysql> select quota from mailbox; +-----------+ | quota | +-----------+ | 512000000 | | 512000000 | | 512000000 | | 512000000 | | 512000000 | | 512000000 | | 512000000 | | 512000000 | | 512000000 | | 30720000 | | 30720000 | +-----------+ 11 rows in set (0.00 sec) -- GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT! Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01

2 1

[Dovecot] POP3 flag updates
by Timo Sirainen 28 Jun '09

28 Jun '09

Why does Dovecot add \Seen flag for mails that have been RETRed? Does anyone really want to use this feature? I can't remember why I implemented it originally, perhaps it was just "because other servers do that too". I'm thinking about at least changing pop3_no_flag_updates=yes to be default. There's also of course pop3_enable_last setting to enable LAST command which was removed from POP3 specs. That requires knowing the last RETRed mail, so I guess I should add a check that it can't be enabled without pop3_no_flag_updates=no.

2 2

[Dovecot] Spliting Folders for Efficiency
by Daniel Watts 28 Jun '09

28 Jun '09

Dear Timo, Would there be any sense in giving Dovecot the option to split folders into multiple subfolders when they reached a specified size (probably message count) limit? Dovecot would monitor folders and when they reached, say, 10,000 messages, silently split the folder on the filesystem to ensure that access remains fast. I know that Dovecot scales very well but this would give practically unlimited storage capability and also keep things fast. You could even have it so that the latest 100 messages are kept in their own folder for fast access. .Folder.new .Folder.cur .Folder.tmp could become: .Folder__1.new .Folder__1.cur .Folder__1.tmp and .Folder__2.new .Folder__2.cur .Folder__2.tmp with Dovecot merging them before display as just "Folder" within the mail client. This could be further extended so that Dovecot could be configured to store 'old' message folders in a separate location. We could then have slower+cheaper+larger storage mounted so that 'old mail' does not take up the expensive local SCSI disks on the machine. Mail from 2 years ago is much less likely to be accessed than mail from the last week. This would provide very neat behind-the-scenes archiving functionality. Looking forward to hearing your thoughts. Best, Daniel -- Squirrelmail Stable 1.4.8 (and developing on 1.5.2) PHP 5.x Hardened with Eaccelerator Apache 2.x Mysql 5.0.x Imapproxy over Dovecot 1.0.rc27 with Maildir all running on Gentoo Linux for ~5,000 users.

7 11

[Dovecot] Sieve/ManageSieve integration into pkgsrc
by Edgar Fuß 28 Jun '09

28 Jun '09

I'm integrating Sieve (the new one) and ManageSieve into wip/dovecot. Currently, this works as dovecot options because dovecot must be built before sieve can be configured and sieve must be built before managesieve can be configured/built. Now, the question arose what the long-term solution (in pkgsrc) should be. To my understanding, with dovecot 2.0, ManageSieve will no longer need to patch dovecot. But what about both Sieve and ManageSieve depending on doevcot sources in order to build (or on libraries dovecot only builds internally)? The preferred way (for pkgsrc) would be if both Sieve and ManageSieve could be built as stand-alone packages and not needing a dovecot source tree to build. What's the long-term plan for Sieve/ManageSieve in this respect? The pkgsrc infrastructure (intentionally) doesn't like a package depending on anothers package working directory in order to build. So with these cross-dependencies, the only pkgsrc ways to go would be either to build it all as one package with options (that's what I currently do) or extract, patch, configure and build dovecot inside a sieve package.

3 2

[Dovecot] dovecot and vmailmgr/qmail mailboxes
by Mij 26 Jun '09

26 Jun '09

Hello folks, Time ago I worked on a patch for dovecot to work with vmailmgr http://mij.oltrelinux.com/net/dovecot-qmail-vmailmgr/ The patch was required because, despite vmailmgr complies with the checkpassword interface supported by dovecot, dovecot snips the mailbox directory returned by the authentication module at colon characters ':'. Incidentally, vmailmgr maps any "name.surname@domain" to "~domain/users/name:surname" (a standard practice in the qmail world, see eg http://www.qmail.org/man/man5/dot-qmail.html at "For security") This path dovecot would then parse as "~domain/users/name". An authentication module can't do anything about it, as this is due to dovecot collecting accounting information in one common buffer, tokenized by ':' separators. I soon stopped updating the patch because I didn't see much point in running indefinitely after releases. However, a bunch of people wrote me over the last years asking for details or updates. Partially for having a similar requirement myself, partially drafted by such feedback, I recently wrote a "compromise" wrapper solution (attached). It does not require patching dovecot (external module) and it's thus (supposedly) durable across dovecot releases; however, it cannot help with the colon problem. The wrapper works as follows: 1) compile and install: # cc -O2 -Wall -o checkvpw-dovecot-wrapper checkvpw-dovecot-wrapper.c # install -s -o root checkvpw-dovecot-wrapper /usr/local/bin 2) edit dovecot.conf, setup the auth section as follows: ============ auth default { mechanism = plain passdb checkpassword { args = /usr/local/bin/checkvpw-dovecot-wrapper } userdb prefetch { } user = root } ============ If "name.surname@domain" wants to be created, the administrator shall instead create the user "namesurname@domain" and an alias "name.surname@domain" pointing to it. This may be conceivable for new installations, but it is not for scenarios where dovecot is to replace another server on top of an existing user base. Of course, it's very desirable to support "." users too. I see that it would require several changes in the code, but I'd like to hear your opinion on this. What about - using a different separator than ":", less common and safer. Could '\0' be one? If not, what about '*'? - as a quicker & dirtier solution, what about allowing escaping? One can modify the auth module to replace ":" with "\:", if dovecot is instructed to avoid tokenizing when a "\" precedes a ":". - as an alternative, dovecot may require the module to chdir() to the mailbox directory instead of returning its path as a string, and subsequently operate locally on mailbox-relative paths. I can't guess how compatible this is with the current design. In the next days, I will publish such wrapper to the webpage. For your convenience, I will invite interested users to show up on this thread; this should help you in assessing how this is interesting for people. bye

4 7

Re: [Dovecot] Lots of pop3-logins
by V S Rao 26 Jun '09

26 Jun '09

> > Doing a "ps aux" on my Slackware box, I have approx 100 PID's of "pop3-login's going on. This is a production mail server, but it is getting VERY low traffic. In fact, only 3 people can "pop3" into it. I've check their e-mail clients, and they are not checking mail any more often than every 5 minutes. > > > > This is a new installation and I've had the server up and running since Sunday. If it matters, I'm using Postfix for the MTA and using the Dovecot SASL library to AUTH SMTP. > > > > Is this a cause for concern? Why does Dovecot need this many processes? > > > > >> Because dovecot preforks the *-login processes to speed-up the login. > > >> No need to worry. > > 100 login sessions for just 3 connections? That is not right, no matter what. >> No, login_processes_count matters. How? If my understanding is correct, you have extra 3 login processes created to cater to new connections. So with only 3 POP3 users, why should so many login processes be spawned? I can understand 10-15. But 100 definitely indicates either the processes are not dying or something else happening on the system which is causing such high number of login processes. The system definitely needs to be checked for some kind of attack, a rogue process running on the system or something else. Regards --Rao

7 9

[Dovecot] dovecot-acl-list not honouring owner / permissions that are set on dovecot-shared
by Stuart Rowan 25 Jun '09

25 Jun '09

Hi, I've just upgraded our dovecot installation from 1.0.15 to 1.1.13. This went smoothly and just worked -- thank you very much for writing such a high performance and robust piece of software. However, we have a shared mailbox that people can subscribe to. After upgrading to 1.1.13 ... dovecot tries to create a dovecot-acl-list file in the top level of this ... but it creates it chown'd to the first logged in user with a umask of -rw- --- --- A. -rw------- 1 userA shared 17179 2009-06-25 11:10 /home/imapshared/SharedMaildir/dovecot-acl-list Everyone else who tried to access the shared folder (and hence dovecot tried to read / update the dovecot-acl-list file) caused an error to be logged to the error.log I changed the permissions (once I realised what had gone wrong) to match those on the dovecot-shared file and all was well as everyone can now read/update it. B. -rw-rw---- 1 imapshared shared 17179 2009-06-25 11:19 /home/imapshared/SharedMaildir/dovecot-acl-list -rw-rw---- 1 imapshared shared 0 2008-04-25 13:20 /home/imapshared/SharedMaildir/dovecot-shared Also from time to time the permissions seem to get changed back on dovecot-acl-list inexplicably to those in situation A -- usually following a restart of dovecot but I haven't been able to reproduce this reliably. This seems like a bug to me ... do you agree / is there a sensible work-around? The declaration in dovecot.conf for the shared namespace sets user specific CONTROL and INDEX locations so that users can independently subscribe to folders and mark messages as read on an individual basis. Many thanks, Stu.

2 4

Re: [Dovecot] Lots of pop3-logins
by V S Rao 25 Jun '09

25 Jun '09

> Hello, > > Doing a "ps aux" on my Slackware box, I have approx 100 PID's of "pop3-login's going on. This is a production mail server, but it is getting VERY low traffic. In fact, only 3 people can "pop3" into it. I've check their e-mail clients, and they are not checking mail any more often than every 5 minutes. > > This is a new installation and I've had the server up and running since Sunday. If it matters, I'm using Postfix for the MTA and using the Dovecot SASL library to AUTH SMTP. > > Is this a cause for concern? Why does Dovecot need this many processes? > >> Because dovecot preforks the *-login processes to speed-up the login. >> No need to worry. 100 login sessions for just 3 connections? That is not right, no matter what. There is definitely some issue. Once the load increases the system will start timing out on POP3 connections or other network connections, such as IMAP, SSH etc. Better check out the system logs, utilization etc. for any abnormal values. Regards Rao

2 1