February 2010 - dovecot

[Dovecot] Testing 2.0b3 - problem with pop3
by Odhiambo Washington 07 Apr '10

07 Apr '10

I am testing the version on FreeBSD 7.3-PRELEASE (I doubt it's the issue) and I am having a problem with pop3. Here is what I see in the log when I attempt to connect to port 110: Feb 22 18:15:03 master: Warning: SIGHUP received - reloading configuration Feb 22 18:15:08 pop3-login: Panic: file login-settings.c: line 216 (login_settings_read): assertion failed: (input.roots[count] == NULL) Feb 22 18:15:08 master: Error: service(pop3-login): child 70115 killed with signal 6 (core dumped) Feb 22 18:15:08 master: Error: service(pop3-login): command startup failed, throttling And the output of doveconf can be seen at http://62.8.64.102/~wash/2.0b3.txt I've basically made my best effort to reflect the settings I had on 2.0b1 into this version. What could the problem be? -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ "If you have nothing good to say about someone, just shut up!." -- Lucky Dube

7 10

[Dovecot] simple steps with sieve
by Gavin Hamill 03 Apr '10

03 Apr '10

Today is my first day with sieve, so be gentle :) I'm trying to set up a pretty webmail interface to our Dovecot 1.2.4 server using roundcube. The managesieve config + roundcube 'managesieve' plugin work fine, and I'm able to use roundcube's UI to generate .dovecot.sieve files. We use winbind + LDAP lookups to do some exotic mail rewriting... ultimately user.name(a)domain.com gets rewritten to 'ar-cloh@localhost', and in /home/AD/ar-cloh/.dovecot.sieve there is only this: require ["vacation"]; # rule:[gdh_test] if anyof (header :contains "Subject" "test") { vacation "test away"; } When I send an email to the user, I see this in the mail.log Oct 19 15:39:43 ar-smtp-1 dovecot: deliver(ar-cloh): sieve: msgid=<1255963176.14420.31.camel@gdh-work>: discarding vacation response for message implicitly delivered to <ar-cloh@ar-smtp-1> I don't understand; sending a vacation for implicitly-delivered messages sounds exactly like what I want - is there any way to say 'do this anyway' ? gdh

8 18

[Dovecot] dovecot-2.0.beta3 tcpwrapper support in Solaris
by Tomi Vainio 22 Mar '10

22 Mar '10

Hi, 2.0 compiles fine in Solaris but and I've found only one glitch so far. Tcpwapper support needs some tweaks. I need to add CPPFLAGS=/usr/sfw/include because tcpd.h is in there. Then also LDFLAGS='-R/usr/sfw/lib -L/usr/sfw/lib' is needed. It would be nice to have --with-tcpwrap-dir or something. After this linking gives an error Undefined first referenced symbol in file deny_severity /usr/sfw/lib//libwrap.so allow_severity /usr/sfw/lib//libwrap.so These are not defined but application itself should define these globally so I've added these to configure and src/util/tcpwrap.c #include <syslog.h> int allow_severity = LOG_INFO; int deny_severity = LOG_WARNING; Tomppa

2 2

[Dovecot] Dovecot v2.0.beta3 quota using userdb
by e-frog 20 Mar '10

20 Mar '10

Hello Timo, I'm playing around with the new Dovecot v2.0.beta3 and don't seem to be able to get quotas working from userdb. I didn't change the SQL query which was working for 1.2.10. Feb 20 19:34:06 ubuntu-test dovecot: auth: sql(test,192.168.156.99): SELECT home, mail, uid, gid, concat('*:storage=', quota_limit, 'MB') AS quota_rule FROM users WHERE username = 'test' ... Feb 20 19:34:06 ubuntu-test dovecot: imap(test): Quota root: name=user backend=dict args=:proxy::dictionary Feb 20 19:34:06 ubuntu-test dovecot: imap(test): dict quota: user=test, uri=proxy::dictionary, noenforcing=0 ... Note there is no "Quota rule". However it seems to work by adding "quota_rule = *:storage=1024MB" directly to plugin.conf: ... Feb 20 19:39:15 ubuntu-test dovecot: imap(test): Quota root: name=user backend=dict args=:proxy::dictionary Feb 20 19:39:15 ubuntu-test dovecot: imap(test): Quota rule: root=user mailbox=* bytes=1073741824 messages=0 Feb 20 19:39:15 ubuntu-test dovecot: imap(test): dict quota: user=test, uri=proxy::dictionary, noenforcing=0 ... Has there anything changed in dovecot 2.0 in this perspective? Thanks, e-frog ------------------------------------------------------------------------------------------ # 2.0.beta3: /etc/dovecot/dovecot.conf # OS: Linux 2.6.31-19-generic-pae i686 Ubuntu 9.10 auth_debug = yes auth_socket_path = /var/run/dovecot/auth-userdb deliver_log_format = msgid=%m: from=%f: %$ dict { dictionary = mysql:/etc/dovecot/dovecot-dict-sql.conf } disable_plaintext_auth = no lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_debug = yes namespace { hidden = no inbox = yes list = yes location = prefix = separator = / subscriptions = yes type = private } namespace { location = virtual:/etc/dovecot/virtual:INDEX=~/.maildir/control prefix = virtual/ separator = / subscriptions = no type = private } passdb { args = /etc/dovecot/dovecot-sql.conf deny = no driver = sql master = no pass = no } plugin { fts = squat fts_squat = partial=4 full=10 quota = dict:user::proxy::dictionary sieve = ~/.dovecot.sieve sieve_before = /etc/dovecot/sieve-global/ sieve_dir = ~/.sieve } postmaster_address = postmaster@ubuntu-test protocols = imap lmtp managesieve quota_full_tempfail = yes service managesieve-login { inet_listener sieve_deprecated { port = 2000 } } service dict { unix_listener dict { mode = 0666 } } service auth { unix_listener auth-userdb { mode = 0666 } unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service lmtp { unix_listener /var/spool/postfix/lmtp-server { group = postfix mode = 0660 user = postfix } } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } verbose_proctitle = yes

3 7

[Dovecot] problem with master db and dovecot-2.0.beta3
by Oliver Eales 19 Mar '10

19 Mar '10

Hello, i am currenty testing dovecot 2.0.beta3 and ran into a problem with the master-db setting. For our webmail access we are sucessfully using the following settings with dovecot-1.2.10 auth_master_user_separator =* auth default { mechanisms = plain passdb passwd-file { args = /etc/dovecot/passwd.masterusers master = yes pass = yes } passdb ldap { args = /etc/dovecot/dovecot-ldap.conf } } when "converting" the settings to version 2.0 i tried the following: auth_mechanisms = plain auth_master_user_separator = * passdb { driver = passwd-file args = /etc/dovecot/passwd.masterusers master = yes pass = yes } passdb { driver = ldap args = /etc/dovecot/dovecot-ldap.conf } But when i try to login, the auth process gets stalled with the following log entry: auth: Fatal: Master passdb can't have pass=yes if there are no passdbs master: Error: service(auth): command startup failed, throttling Am i doing something wrong or is it problem with the 2.0-beta ? Regards, Oliver Eales

2 3

[Dovecot] GlusterFs - Any new progress reports?
by Ed W 18 Mar '10

18 Mar '10

GlusterFs always strikes me as being "the solution" (one day...). It's had a lot of growing pains, but there have been a few on the list had success using it already. Given some time has gone by since I last asked - has anyone got any more recent experience with it and how has it worked out with particular emphasis on Dovecot maildir storage? How has version 3 worked out for you? Anyone had success using some other clustered/HA filestore with dovecot who can share their experience? (OCFS/GFS over DRBD, etc?) My interest is more in bootstrapping a more highly available system from lower quality (commodity) components than very high end use Thanks Ed W

8 18

Re: [Dovecot] sieve vacation
by Martin F. Foster 18 Mar '10

18 Mar '10

Following from Steffen Kaiser's response, RFC5230 explains the conditions necessary for the Sieve Vacation extension to trigger a response. This says: "Vacation" MUST NOT respond to a message unless the recipient user's email address is in a "To", "Cc", "Bcc", "Resent-To", "Resent-Cc", or "Resent-Bcc" line of the original message. So your scenario 4 happens when the sender is only in the envelope, and not in the headers. For example: this transaction will get a response because the mailbox address is in the To: header ehlo there.com mail from: <dude(a)there.com> rcpt to: <user(a)domain.tld> << envelope recipient, mailsystem routes with this information data To: "Some User" <user(a)domain.tld> << header recipient must be correlated for Sieve Vacation to send a response From: "Dude" <dude(a)there.com> Subject: message Hi there . This transaction will not get a response, and will trigger the "discarding vacation response for message implicitly delivered to <user(a)domain.tld> " log: ehlo there.com mail from: <dude(a)there.com> rcpt to: <user(a)domain.tld> data Subject: message Hi there . Cheers, -Martin Foster Le 12-Feb-10 21:00, dovecot-request(a)dovecot.org a écrit : > Message: 9 > Date: Fri, 12 Feb 2010 09:18:23 +0100 > From: Lampa<lampacz(a)gmail.com> > Subject: [Dovecot] sieve vacation > To: Dovecot Mailing List<dovecot(a)dovecot.org> > Message-ID: > <9aef75b11002120018v69f185b5hb0a9e0f7dad1e5c5(a)mail.gmail.com> > Content-Type: text/plain; charset=ISO-8859-1 > > Hello, > > scenario: > > OK 1. created vacation script (see below) > OK 2. send message fromuser(a)anotherdomain.tld touser(a)domain.tld => > got auto vacation response (2010-02-11T19:42:13) > OK 3. send message fromuser(a)anotherdomain.tld touser(a)domain.tld => > no response (is ok because ":days 1" settings) (2010-02-11T19:47:46) > BAD 4. send message fromanotheruser(a)anotherdomain.tld to > user(a)domain.tld => no response (2010-02-11T19:48:04) > > > Why is discarding vacation response for message implicitly delivered > to<user(a)domain.tld> ? > > log: > 2010-02-11T19:42:13.669321+01:00 hermes dovecot: > deliver(user(a)domain.tld): sieve: > msgid=<052a01caab49$ec4752e0$c4d5f8a0$@tld>: sent vacation response to > <user(a)anotherdomain.tld> > 2010-02-11T19:42:13.732541+01:00 hermes dovecot: > deliver(user(a)domain.tld): sieve: > msgid=<052a01caab49$ec4752e0$c4d5f8a0$@tld>: stored mail into mailbox > 'INBOX' > 2010-02-11T19:47:46.271534+01:00 hermes dovecot: > deliver(user(a)domain.tld): sieve: > msgid=<053201caab4a$b295ece0$17c1c6a0$@tld>: discarded duplicate > vacation response to<user(a)anotherdomain.tld> > 2010-02-11T19:47:46.275891+01:00 hermes dovecot: > deliver(user(a)domain.tld): sieve: > msgid=<053201caab4a$b295ece0$17c1c6a0$@tld>: stored mail into mailbox > 'INBOX' > 2010-02-11T19:48:04.958622+01:00 hermes dovecot: > deliver(user(a)domain.tld): sieve: > msgid=<053701caab4a$bda90220$38fb0660$@tld>: discarding vacation > response for message implicitly delivered to<user(a)domain.tld> > 2010-02-11T19:48:04.965875+01:00 hermes dovecot: > deliver(user(a)domain.tld): sieve: > msgid=<053701caab4a$bda90220$38fb0660$@tld>: stored mail into mailbox > 'INBOX' > > vacation script: > ----------- > require "vacation"; > > vacation > :days 1 > :subject "Auto reply" > > "auto reply message"; > -------------- > > Thank you > -- Lampa ------------------------------

3 4

[Dovecot] qmail-secretary plugin for dovecot deliver
by Rajkumar S 17 Mar '10

17 Mar '10

Hi, I have been using qmail-ldap for quite some time and now moved to postfix/dovecot. One feature that I miss is that provided by qmail-secretary. qmail-secretary basically is a mail list manager with following features: 1 no limit, just explode to all members 2 members only, as the name says; only members are allowed (based on envelope sender, so not very secure, everybody can fake that) 3 sender confirm, sender has to confirm submission and return address like DJBs qmail-secretary 4 moderated, moderator(s) have to accept every message; there are DN and RFC822 moderators All configuration (name, members, moderated etc...) are taken from LDAP. I had gone through the code and from what I understand for each 4 delivery types it works as follows: 1. Just deliver to all recipients (say, inject it via sendmail). 2. verify if envelope sender is in allowed list and proceed as above. 3. 1. create a random hash 2. save the mail in new/ with hash as filename 3. create a new mail with listname-confirm-hash(a)domain.com and send 4. If that mail comes back, deliver the mail to all recipients. 4. 1. as above but using listname-approve-hash and list-name-reject-hash 2. move the msg to cur/ 3. if appropriate reply comes retrieve the mail from cur/ and send to all recipient. qmail-secretary is invoked by qmail-local and thus this is a nice candidate for a deliver plugin. Before I start coding I just want to check if there is any gotchas that I am missing? I assume all these operations can be performed as a deliver plugin and that deliver plugin is the correct place for such functionality. It goes without saying that I would be contributing this plugin back to dovecot. with regards, raj

4 9

[Dovecot] segfault with vpopmail 5.5.0
by Giuseppe Iuculano 16 Mar '10

16 Mar '10

Hi, version: 1.2.10 OS: Debian unstable/x86_64 [118905.528497] dovecot-auth[3856]: segfault at 0 ip (null) sp 00007fff6666ad98 error 14 in dovecot-auth[400000+61000] 2010-02-26 22:17:45 dovecot: Error: child 3856 (auth) killed with signal 11 (core dumped) (gdb) bt full #0 0x0000000000000000 in ?? () No symbol table info available. #1 0x0000000000423b0e in vpopmail_lookup_vqp (request=0x726f68, vpop_user=0x7fff6666ae20 "postmaster", vpop_domain=0x7fff6666adc0 "test.bogus") at userdb-vpopmail.c:43 vpw = <value optimized out> #2 0x0000000000421376 in vpopmail_password_lookup (auth_request=0x7fff6666ae20, cleartext=false, result_r=0x7fff6666aebc) at passdb-vpopmail.c:65 vpop_user = "postmaster", '\000' <repeats 70 times> vpop_domain = "test.bogus", '\000' <repeats 70 times> vpw = <value optimized out> password = <value optimized out> #3 0x00000000004215ad in vpopmail_verify_plain (request=0x7fff6666ae20, password=0x704360 "test", callback=0x415500 <auth_request_verify_plain_callback>) at passdb-vpopmail.c:119 result = <value optimized out> scheme = <value optimized out> tmp_pass = <value optimized out> crypted_pass = <value optimized out> ret = <value optimized out> #4 0x0000000000415391 in auth_request_verify_plain (request=0x726f68, password=0x704360 "test", callback=0x41eb90 <plain_verify_callback>) at auth-request.c:509 passdb = 0x70ca40 result = <value optimized out> cache_key = <value optimized out> __PRETTY_FUNCTION__ = "auth_request_verify_plain" #5 0x000000000041a6f7 in mech_plain_auth_continue (request=0x726f68, data=<value optimized out>, data_size=<value optimized out>) at mech-plain.c:58 authid = 0x704340 "" authenid = 0x704341 "postmaster(a)test.bogus" error = <value optimized out> pass = 0x704360 "test" count = 2 #6 0x0000000000415d41 in auth_request_handler_auth_begin (handler=0x7192f8, args=<value optimized out>) at auth-request-handler.c:388 mech = <value optimized out> request = 0x726f68 list = 0x24 name = 0x704308 "@Cp" arg = 0x704195 "AHBvc3RtYXN0ZXJAdGVzdC5ib2d1cwB0ZXN0" initial_resp = 0x704195 "AHBvc3RtYXN0ZXJAdGVzdC5ib2d1cwB0ZXN0" initial_resp_len = 2 id = <value optimized out> buf = 0x704308 #7 0x0000000000411549 in auth_client_handle_line (conn=0x717f70) at auth-client-connection.c:174 No locals. #8 auth_client_input (conn=0x717f70) at auth-client-connection.c:250 _data_stack_cur_id = 3 line = 0x7182b6 "AUTH\t1\tPLAIN\tservice=pop3\tlip=XXX.XXX.XXX.XXXcensored\trip=XXX.XXX.XXX.XXXcensored\tlport=110\trport=41899\tresp=AHBvc3RtYXN0ZXJAdGVzdC5ib2d1cwB0ZXN0" ret = <value optimized out> #9 0x000000000043643d in io_loop_handler_run (ioloop=<value optimized out>) at ioloop-epoll.c:208 ctx = 0x70f3e0 event = 0x70f450 list = 0x718280 io = 0x718240 ---Type <return> to continue, or q <return> to quit--- tv = {tv_sec = 4, tv_usec = 999573} t_id = 2 msecs = <value optimized out> ret = 1 i = 0 call = <value optimized out> #10 0x00000000004358b8 in io_loop_run (ioloop=0x70c380) at ioloop.c:335 No locals. #11 0x0000000000419efe in main (argc=<value optimized out>, argv=<value optimized out>) at main.c:347 foreground = false Cheers, Giuseppe

2 4

[Dovecot] Scalability plans: Abstract out filesystem and make it someone else's problem
by Timo Sirainen 12 Mar '10

12 Mar '10

This is something I figured out a few months ago, mainly because this one guy at work (hi, Stu) kept telling me my multi-master replication plan sucked and we should use some existing scalable database. (I guess it didn't go exactly like that, but that's the result anyway.) So, my current plan is based on a couple of observations: * Index files are really more like memory dumps. They're already in an optimal format for keeping them in memory, so they can be just mmap()ed and used. Doing some kind of translation to another format would just make it more complex and slower. * I can change all indexing and dbox code to not require any locks or overwriting files. I just need very few filesystem operations, primarily the ability to atomically append to a file. * Index and mail data is very different. Index data is accessed constantly and it must be very low latency or performance will be horrible. It practically should be in memory in local machine and there shouldn't normally be any network lookups when accessing it. * Mail data on the other hand is just written once and usually read maybe once or a couple of times. Caching mail data in memory probably doesn't help all that much. Latency isn't such a horrible issue as long as multiple mails can be fetched at once / in parallel, so there's only a single latency wait. So the high level plan is: 1. Change the index/cache/log file formats in a way that allows lockless writes. 2. Abstract out filesystem accessing in index and dbox code and implement a regular POSIX filesystem support. 3. Make lib-storage able to access mails in parallel and send multiple "get mail" requests in advance. (3.5. Implement async I/O filesystem backend.) 4. Implement a multi-master filesystem backend for index files. The idea would be that all servers accessing the same mailbox must be talking to each others via network and every time something is changed, push the change to other servers. This is actually very similar to my previous multi-master plan. One of the servers accessing the mailbox would still act as a master and handle conflict resolution and writing indexes to disk more or less often. 5. Implement filesystem backend for dbox and permanent index storage using some scalable distributed database, such as maybe Cassandra. This is the part I've thought the least about, but it's also the part I hope to (mostly) outsource to someone else. I'm not going to write a distributed database from scratch.. This actually should solve several issues: * Scalability, of course. It'll be as scalable as the distributed database being used to store mails. * NFS reliability! Even if you don't care about any of these alternative databases, this still solves NFS caching problems. You'd keep using the regular POSIX FS API (or async FS api) but with the in-memory index "cache", so only a single server is writing to mailbox indexes at the same time. * Shared mailboxes. The filesystem API is abstracted, so it should be possible to easily add another layer to handle accessing other users' mails from both local and remote servers. This should finally make it possible to easily support shared mailboxes with system users.

17 82