December 2012 - dovecot

[Dovecot] Released Pigeonhole v0.2.4 for Dovecot v2.0.14
by Stephan Bosch 26 Mar '13

26 Mar '13

Hello Dovecot users, I release a new version of Pigeonhole tonight. This release extends the configuration capabilities of the include and vacation extensions, adds the ihave extension (RFC 5463), and fixes a few small other things. Judging by Timo's plans, this will be one of the last feature releases for Dovecot v2.0, since I'll be switching active development to Dovecot v2.1 once that matures a little more. Changelog v0.2.4: + Vacation extension: finally added support for using the original recipient in vacation address check. It is also possible to disable the recipient address check entirely. Check doc/vacation.txt for configuration information. + Include extension: made limits on the include depth and the total number of included scripts configurable. Check doc/include.txt for configuration information. + Implemented ihave extension. This allows checking for the availability of Sieve language extensions at 'runtime'. Actually, this is checked at compile time. At runtime the interpreter checks whether extensions that were not previously available are still unavailable. If the situation changed, the script is re-compiled and the ihave tests are evaluated again. + Sieve: optimized compilation of tests that yield constant results (i.e. known at compile tme), such as 'true' and 'false'. No code is produced anymore for script sections that are never executed. Also, semantics are not verified anymore in uncompiled script sections. + Made vnd.dovecot.debug extension available to the LDA plugin instead of only the command line tools. + Sieve: redirect action now adds X-Sieve-Redirected-From header (mainly for people using SPF/SRS). - Sieve: fixed bug in handling flags and keywords; in case of error an assertion was triggered. - Script storage: improved handling of unconfigured user home directory. Originally this would produce an unhelpful error message. - Imap4flags extension: prevent forcibly enabling imap4flags when imapflags is enabled. - Fixed various -Wunused-but-set-variable compiler warnings. - Include extension: forgot to check variable identifier syntax for 'global' command. - Sieve: fixed debug mode; no messages were logged in some situations. - sievec tool: forgot to enable -D (debug) parameter. The release is available as follows: http://www.rename-it.nl/dovecot/2.0/dovecot-2.0-pigeonhole-0.2.4.tar.gz http://www.rename-it.nl/dovecot/2.0/dovecot-2.0-pigeonhole-0.2.4.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.0 wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan(a)rename-it.nl

2 1

[Dovecot] doveadm with multiple commands
by Timo Sirainen 25 Mar '13

25 Mar '13

This has been in my TODO for a while and today I implemented it (but didn't dare to test it yet :) So the problem has been that you want to run for example: doveadm expunge -A mailbox Trash savedbefore 7d doveadm purge -A This is annoying because it scans through the users' mailboxes twice, wasting disk IO and CPU. So how to make it do the scan only once? The syntax I thought of today would be: doveadm multi [-A | -u wildcards] <separator string> <comand 1> [<separator string> <command 2> [...]] So the first two commands could be for example: doveadm multi -A : expunge mailbox Trash savedbefore 7d : purge The ':' character is the separator. Any other string could be used as well, for example "?SEP?" or whatever. Thoughts? Any better name for the command than "multi"?

6 13

[Dovecot] virtual folder - crash while searching
by Matthias Rieber 10 Mar '13

10 Mar '13

Hi, I've upgraded from dovecot 2.0 to 2.1. When I perform a header search in a virtual folder dovecot crashes. Here's the stacktrace: http://pastebin.com/urSDDzku Regards, Matthias

4 11

[Dovecot] 2.1.12: Panic: file solr-connection.c: line 547 (solr_connection_post_more)
by Marc Schiffbauer 26 Feb '13

26 Feb '13

Hi all, I am having a problem indexing one of my mailboxes using the solr fts backend in dovecot 2.1.12 For many mailboxes it works just fine, but on one mailbox I currently always get a panic. solr setup: Java: icedtea 6.1.11.5 Solr: 3.6.2 running in tomcat 7.0.32 Command to reproduce error: doveadm index -u my@user badmailbox I already noticed that there have been some solr backend fixes just before 2.1.21 release... Is this another bug in the backend? Any idea? IMO relevant logs line parts: java.io.CharConversionException: Invalid UTF-8 start byte 0xf8 (at char #483989, byte #532382) complete dovecot log: doveadm(my@user): Error: fts_solr: Invalid XML input at 85:198: mismatched tag (near: <html><head><title>Apache Tomcat/7.0.32-gentoo - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:wh) doveadm(my@user): Panic: file solr-connection.c: line 547 (solr_connection_post_more): assertion failed: (maxfd >= 0) doveadm(my@user): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x4bcdb) [0x34fbd53ecdb] -> /usr/lib64/dovecot/libdovecot.so.0(default_fatal_handler+0x3a) [0x34fbd53edb8] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x34fbd50eeac] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(solr_connection_post_more+0x2a4) [0x34fbba81072] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(+0x40a5) [0x34fbba7d0a5] -> /usr/lib64/dovecot/lib20_fts_plugin.so(fts_build_mail+0x462) [0x34fbbc8d6dc] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0xd8c2) [0x34fbbc928c2] -> doveadm(+0x17a30) [0x264ecb7a30] -> doveadm(+0x13a45) [0x264ecb3a45] -> doveadm(+0x14762) [0x264ecb4762] -> doveadm(doveadm_mail_try_run+0x173) [0x264ecb4cfd] -> doveadm(main+0x3e9) [0x264ecbd0d5] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x34fbd16a48d] -> doveadm(+0x13809) [0x264ecb3809] Aborted complete java exception: Dec 31, 2012 4:12:58 AM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr-web path=/update params={} status=500 QTime=118 Dec 31, 2012 4:12:58 AM org.apache.solr.common.SolrException log SEVERE: java.lang.RuntimeException: [was class java.io.CharConversionException] Invalid UTF-8 start byte 0xf8 (at char #483989, byte #532382) at com.ctc.wstx.util.ExceptionUtil.throwRuntimeException(ExceptionUtil.java:18) at com.ctc.wstx.sr.StreamScanner.throwLazyError(StreamScanner.java:731) at com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3657) at com.ctc.wstx.sr.BasicStreamReader.getText(BasicStreamReader.java:809) at org.apache.solr.handler.XMLLoader.readDoc(XMLLoader.java:315) at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:156) at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:79) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:58) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:365) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:260) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Unknown Source) at org.apache.catalina.core.ApplicationFilterChain.doFilter(Unknown Source) at org.apache.catalina.core.StandardWrapperValve.invoke(Unknown Source) at org.apache.catalina.core.StandardContextValve.invoke(Unknown Source) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(Unknown Source) at org.apache.catalina.core.StandardHostValve.invoke(Unknown Source) at org.apache.catalina.valves.ErrorReportValve.invoke(Unknown Source) at org.apache.catalina.valves.AccessLogValve.invoke(Unknown Source) at org.apache.catalina.core.StandardEngineValve.invoke(Unknown Source) at org.apache.catalina.connector.CoyoteAdapter.service(Unknown Source) at org.apache.coyote.http11.AbstractHttp11Processor.process(Unknown Source) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(Unknown Source) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) Caused by: java.io.CharConversionException: Invalid UTF-8 start byte 0xf8 (at char #483989, byte #532382) at com.ctc.wstx.io.UTF8Reader.reportInvalidInitial(UTF8Reader.java:302) at com.ctc.wstx.io.UTF8Reader.read(UTF8Reader.java:188) at com.ctc.wstx.io.ReaderSource.readInto(ReaderSource.java:84) at com.ctc.wstx.io.BranchingReaderSource.readInto(BranchingReaderSource.java:57) at com.ctc.wstx.sr.StreamScanner.loadMore(StreamScanner.java:992) at com.ctc.wstx.sr.BasicStreamReader.readTextSecondary(BasicStreamReader.java:4628) at com.ctc.wstx.sr.BasicStreamReader.readCoalescedText(BasicStreamReader.java:4126) at com.ctc.wstx.sr.BasicStreamReader.finishToken(BasicStreamReader.java:3701) at com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3649) ... 25 more TIA -Marc -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich

4 9

[Dovecot] POLL: v2.2 to allow one mail over quota?
by Timo Sirainen 12 Feb '13

12 Feb '13

Currently if user is 1MB under quota and someone tries to deliver mail that is over 1MB, Dovecot rejects the mail. But smaller mails aren't rejected probably for days. So user might not even realize that they didn't receive one of the mails. Also having a user "almost over quota" is a rather strange state I think. So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? Except add a limit that if the message size is as much as the user's entire quota limit it wouldn't be added (or 50% or ..?). Also IMAP wouldn't allow this, since user would get an error anyway. I could make this also optional, but if nobody really wants to keep the old behavior there's really no point in adding the option.

20 27

[Dovecot] passdb to add extra fields?
by Timo Sirainen 30 Jan '13

30 Jan '13

Some passdbs like PAM can't really return any extra fields. Also some people have wanted to combine users' data from different passdb/userdbs so that for example you'd have userdb passwd give the uid/gid/home, but then you'd also have some other userdb give quota limits. So I was thinking something like this: passdb { driver = pam } passdb { driver = sql include = yes } or: userdb { driver = passwd } userdb { driver = passwd-file include = yes } I'm not sure about two things: 1) Should there be a way to replace all of the existing fields instead of just adding new ones? 2) Any thoughts of a better name than "include"? With passdb it would mean that it's included only when the authentication failed for some other passdb. With userdb it means it's included only if a previous userdb lookup succeeded. Also there are already "deny" and "pass" settings. Interaction with them can be somewhat confusing.. Maybe all of these should be replaced with: type=deny: Same as old deny=yes (deny auth if user is in list) type=precondition(?): Same as pass=yes (require another passdb to match) type=postcondition(?): Require user to exist in this passdb/userdb as well, adding any extra fields in it. type=add: Add any extra fields, if the user exists at all. (Better ideas for the names here? Is even "type" a good name?) Then maybe a new setting to delete existing extra fields .. or perhaps just extend passdb { override_fields } so that having "-field" would delete the field if it already existed..

3 6

[Dovecot] From Sendmail to Dovecot mdbox, what is missing.
by Torpey List 27 Jan '13

27 Jan '13

Sendmail 8.14.4 dovecot 2.0.9 I have sendmail working and it is sending mail to /var/mail/%u. I have dovecot working in that I can move emails into IMAP folders and I can send email through IMAP. I have set up dovecot to use mdbox based on the following: mail_location = mdbox:~/mail However, I seem to be lacking a key piece of information. Sendmail is sending the mail to /var/mail/%u as a mbox (single file for all emails) format. Dovecot wants to read the mail in mdbox (Multiple messages per file, but unlike mbox multiple files per mailbox.) So the two programs are not working together. So, I cannot get dovecot to read new emails at /var/mail/%u. So I tried changing to the following: mail_location = mdbox:~/mail:INBOX=/var/mail/%u However, dovecot complains that it is NOT a directory. That is because sendmail is sending as mbox format. I have tried two lines of “mail_location” but that did not work. example mail_location = mdbox:~/mail ----> for dovecot mail_location = mbox:INBOX=/var/mail/%u -----> for sendmail I have tried LMTP and dovecot-lda. LMTP – I could not see any difference with this added or not. Dovecot-lda – I have had issues getting it configured. Thanks for any help! Sendmail changes FEATURE(`local_procmail', `/usr/libexec/dovecot/dovecot-lda',`/usr/libexec/dovecot/dovecot-lda -d $u') MODIFY_MAILER_FLAGS(`LOCAL', `-f') MAILER(procmail)dnl Here is dovecot configuration [root@nala mail]# dovecot -n # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.14.1.el6.x86_64 x86_64 Scientific Linux release 6.3 (Carbon) auth_mechanisms = plain login mail_gid = mail mail_location = mdbox:~/mail mail_uid = mail mbox_write_locks = fcntl passdb { driver = pam } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_group_events = yes } service auth { unix_listener auth-userdb { mode = 0600 user = mail } } service lmtp { inet_listener lmtp { address = 192.168.1.185 127.0.0.1 ::1 port = 24 } user = mail } ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { driver = passwd } protocol lda { info_log_path = /var/log/maillog log_path = /var/log/maillog postmaster_address = postmaster(a)torpey.org }

5 11

[Dovecot] question on "doveadm user" and pass/userdb
by Gábor Lénárt 22 Jan '13

22 Jan '13

Hi, I'm about getting to know dovecot in details, I am about the command "doveadm user". The man page shows this example: userdb: jane uid : 8001 gid : 8001 home : /home/jane mail : sdbox:~/sdbox plugins : sieve quota_rule: *:storage=150M It seems to be nice, however I never got this only: ~# doveadm user lgb@office userdb: lgb@office That's all. The dovecot test configuration works otherwise via IMAP/POP3/LMTP nicely. What I guess that the lack of extra information (which would be needed by some scripts of mine, eg for getting user's home easily from shell scripts, as it's hash based, etc) is caused because I don't pass home (and/or other settings) back from userdb and/or passdb. However I do have these in global scope: mail_home = /mailstorage/%Ld/%Ln (yes, this is not hashed at all at the momement but it will be soon) mail_location = maildir:~/Maildir mail_uid = vmail mail_gid = vmail (... and also some global quota stuffs configured in plugin section) Now, I am a bit confused: even with passdb/userdb does not return with user home or GID/UID, dovecot knows these in case of pop3/imap/lmtp access, since these information can be "composed" from those settings. But then, wouldn't it possible so "doveadm user" shows those as well, like lmtp/pop3/imap can resolve those too? Honestly, it's a bit redundant to put needed information into user and pass attrs in every dbs (I have some) when dovecot knows those otherwise ... If I interpret "doveadm user"'s goal wrong (it's just for userdb/passdb query tool nothing more), is there any tool which works in this config, displaying extra information (at least user's home), or should I create some script which just repeats the functionality of dovecot's configured mail_home resolution? Thanks.

3 5

[Dovecot] 2 node dovecot
by Lazy 08 Jan '13

08 Jan '13

Hi, We run dovecot 2.1 on 2 nodes with shared nfs storage. Index files are kept on local disk. Second node is usually used only for email snapshots access. If primary node fails it is configured to take place of the primary imap server. Snapshots are additional namespaces with mail from various points in time. We need a way to easly copy multiple messages from snapshots to current mailboxes. It would be best to make current mailbox available as another namespace so one would simply copy messages from one folder to another. Is it safe ? There will be times when given user is writing to the same nfs based Maildir mailbox from 2 nodes both of them having separate index files. What will happen if already present message gets copied back from the snapshot ? Regards, Michał Grzędzicki

2 2

[Dovecot] lda crash?
by kjonca＠o2.pl 08 Jan '13

08 Jan '13

When I try to refilter some messages lda crash: Dec 29 16:07:34 LDA(kjonca): Info: sieve: msgid=<003a01bf6d8f$c9d69f20$9c2dcdc3(a)tiger.com.pl>: stored mail into mailbox 'scott/Sent' Dec 29 16:08:02 lda: Debug: Loading modules from directory: /usr/lib/dovecot/modules Dec 29 16:08:02 lda: Debug: Module loaded: /usr/lib/dovecot/modules/lib90_sieve_plugin.so Dec 29 16:08:02 LDA(kjonca): Debug: Effective uid=1000, gid=1000, home=/home/kjonca Dec 29 16:08:02 LDA(kjonca): Debug: Namespace : type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Mail/0:LAYOUT=fs:INBOX=~/Mail/0/INBOX Dec 29 16:08:02 LDA(kjonca): Debug: fs: root=/home/kjonca/Mail/0, index=, control=, inbox=/home/kjonca/Mail/0/INBOX, alt= Dec 29 16:08:02 LDA(kjonca): Debug: userdb lookup skipped, username taken from USER environment Dec 29 16:08:02 LDA(kjonca): Debug: none: root=, index=, control=, inbox=, alt= Dec 29 16:08:02 LDA(kjonca): Debug: Destination address: kjonca(a)alfa.kjonca (source: user@hostname) Dec 29 16:08:02 LDA(kjonca): Debug: sieve: include: sieve_global_dir is not set; it is currently not possible to include `:global' scripts. Dec 29 16:08:02 LDA(kjonca): Debug: sieve: Pigeonhole Sieve Extprograms plugin version 0.1.0 loaded Dec 29 16:08:02 LDA(kjonca): Debug: sieve: using the following location for user's Sieve script: /home/kjonca/.dovecot.sieve Dec 29 16:08:02 LDA(kjonca): Debug: sieve: opening script /home/kjonca/.dovecot.sieve Dec 29 16:08:02 LDA(kjonca): Debug: sieve: script binary /home/kjonca/.dovecot.svbin successfully loaded Dec 29 16:08:02 LDA(kjonca): Debug: sieve: binary save: not saving binary /home/kjonca/.dovecot.svbin, because it is already stored Dec 29 16:08:02 LDA(kjonca): Debug: sieve: executing script from /home/kjonca/.dovecot.svbin Dec 29 16:08:02 LDA(kjonca): Info: sieve: msgid=<001b01bf7227$18649840$9c2dcdc3(a)tiger.com.pl>: stored mail into mailbox 'scott/Sent' doveconf -n output KJ -- http://sporothrix.wordpress.com/2011/01/16/usa-sie-krztusza-kto-nastepny/ Little known fact about Middle Earth: The Hobbits had a very sophisticated computer network! It was a Tolkien Ring...

3 6