- dovecot - dovecot.org

[Dovecot] dovecot fts solr plugin
by Thomas Baumann 05 Aug '13

05 Aug '13

Hello list, * apache-solr-3.6.2 * dovecot-2.1.7 I configured fts_solr for dovecot, but I have two issues When doing a search request within telnet 127.0.0.1 143 it connects to solr. Jul 21, 2013 4:38:15 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr path=/select params={rows=1&q=box:10361d30f8c4ea51ad7c00002ecaff3b+user:" tom(a)tirism.support.tiri.li"&sort=uid+desc&fl=uid} hits=0 status=0 QTime=10 Jul 21, 2013 4:38:15 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr path=/select params={rows=3&q=(hdr:"test"+OR+body:"test")&fq=%2Bbox:10361d30f8c4ea51ad7c00002ecaff3b+%2Buser:" tom(a)tirism.support.tiri.li"&sort=uid+asc&fl=uid,score} hits=0 status=0 QTime=7 But (1) doveadm fts fails: $ doveadm fts rescan -u tom(a)domain.com Error message: doveadm(tom(a)tirism.support.tiri.li): Error: fts not enabled for user's namespace (null) (2) solr schema has errors Jul 21, 2013 4:35:46 PM org.apache.solr.common.SolrException log SEVERE: org.apache.solr.common.SolrException: undefined field text at org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1330) at org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getAnalyzer(IndexSchema.java:408) at org.apache.solr.schema.IndexSchema$SolrIndexAnalyzer.reusableTokenStream(IndexSchema.java:383) at org.apache.lucene.queryParser.QueryParser.getFieldQuery(QueryParser.java:574) at org.apache.solr.search.SolrQueryParser.getFieldQuery(SolrQueryParser.java:206) at org.apache.lucene.queryParser.QueryParser.Term(QueryParser.java:1436) at org.apache.lucene.queryParser.QueryParser.Clause(QueryParser.java:1319) at org.apache.lucene.queryParser.QueryParser.Query(QueryParser.java:1245) at org.apache.lucene.queryParser.QueryParser.TopLevelQuery(QueryParser.java:1234) at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:206) at org.apache.solr.search.LuceneQParser.parse(LuceneQParserPlugin.java:79) at org.apache.solr.search.QParser.getQuery(QParser.java:143) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:105) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:165) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376) at org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:59) at org.apache.solr.core.SolrCore$3.call(SolrCore.java:1182) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314) at java.util.concurrent.FutureTask.run(FutureTask.java:149) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) at java.lang.Thread.run(Thread.java:736) Hope somebody can help! Best regards, Thomas # dovecot --version 2.1.7 # dovecot -n # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.0.74-0.6.10-default x86_64 SUSE Linux Enterprise Server 11 (x86_64) xfs base_dir = /var/run/dovecot/ dict { expire = sqlite:/etc/dovecot/dovecot-dict-sql.conf.ext quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } first_valid_uid = 5000 listen = * login_greeting = tiri Mail ready. login_trusted_networks = 127.0.0.1 mail_location = maildir:/var/vmail/%u/ mail_plugins = fts fts_solr managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } ssl = no userdb { driver = passwd } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol imap { plugin { fts = solr fts_solr = break-imap-search url=http://localhost:8983/solr/ } } protocol pop3 { plugin { fts = solr fts_solr = break-imap-search url=http://localhost:8983/solr/ } }

3 4

[Dovecot] Design: Adding checksums to index files
by Timo Sirainen 05 Aug '13

05 Aug '13

I've been planning on adding these for years. Maybe it's about time soon. I guess they could be added already to v2.2, but enabled only by a new setting because it requires file format changes that old Dovecots can't then read. I could probably patch v2.1 also so it is able to at least read the new format without failing. For v2.3 this new format could then be made the default. And what would the checksums be exactly? Would the standard CRC32 and CRC8 work fine, or are there any better ones? 1. dovecot.index v2.1+ always only fully recreates this file, never overwrites data to it. So the checksums could be written only when the dovecot.index is being recreated. There are 3 possible things to checksum: - header (32bit checksum) - all of the mail records (32bit checksum) - each mail record independently (8bit checksum per mail) The header's checksum could be verified every time the index is opened. The full mail record checksum could be verified when something appears to be wrong, but it's probably a waste of time to check it in normal operation. I'm not really sure about the per-mail checksums. It would be easy to create them while dovecot.index is being created, but after reading the file into memory the records are updated in many ways in many places. It's probably not worth the complexity and extra slowness to verify and/or update the checksums in all the different places. So is it worth it to even have them? In error conditions when fixing up indexes it could be useful to skip over records with broken checksums (and check if the mail is in dovecot.index.backup with correct checksum). Maybe that's enough to be worth 1 byte per message?.. 2. dovecot.index.log This file is only appended to. Each committed transaction could be prefixed in the new format with <transaction size><transaction 32bit checksum>. With the new format this wouldn't actually increase the log file size much, because there is already some space wasted for a compatibility "boundary" record that could be removed now. 3. dovecot.index.cache Cache file is the most complex file. Its headers get overwritten once in a while. Probably not worth the trouble to checksum the header itself, and there's not a lot that could be done even if a broken checksum was found. But each mail_cache_record could have its own checksum. A 8bit checksum could be added without increasing the file's size. Maybe that would be enough? 4. dovecot.index.thread This is a rather simple file and a 32bit checksum could be added to its header, and verified every time the file is read (because it's fully read anyway). 5. dovecot.mailbox.log This file doesn't even have a header. There are 3 unused bytes in each record currently. One of them could be used for a new "flags" parameter, with the only flag being "checksum added". There would still be space left for 8bit or 16bit checksum. 6. Other files There are also some text files, like dovecot-acl, subscriptions, quota usage and Sieve scripts. They probably have to be without checksums for now.

2 2

[Dovecot] How to configure sendmail using dovecot lmtp
by alfdc 05 Aug '13

05 Aug '13

Hi, I can't seems to find any posting on how to configure sendmail to use dovecot as lmtp. All I see is using it as LDA. I am particularly interested on how to tell sendmail to ask dovecot to deliver the message to the user. In the wiki for lda it says to add the following in the sendmail config: FEATURE(`local_procmail', `/usr/local/libexec/dovecot/deliver',`/usr/local/libexec/dovecot/deliver -d $u') MODIFY_MAILER_FLAGS(`LOCAL', `-f') MAILER(procmail) Is this the same for lmtp? If so how does it know when to use lda or lmtp? Beside, is procmail playing any role in any of these? Can the I just replace the whole thing as: FEATURE(`local_lmtp',`/usr/local/libexec/dovecot/deliver',`/usr/local/libexec/dovecot/deliver -d $u) MAILER(`local') Thanks. -- View this message in context: http://dovecot.2317879.n4.nabble.com/How-to-configure-sendmail-using-doveco… Sent from the Dovecot mailing list archive at Nabble.com.

3 2

[Dovecot] Maildir Synchronization warnings
by Kavish Karkera 05 Aug '13

05 Aug '13

Hi, We are repeatedly getting these below warnings for some of our users, al though we have no complaints from them yet, we need to know why these warning occurs. So it would be help full if some one explain these warning msg in detail. ----------------------------------------------------------------------------------------------------------------------------------------------------- Aug 2 12:52:55 blade8 dovecot: imap(kavish.karkera(a)example.com): Warning: Maildir: Scanning /mail/v3store/example.com/kavish.karkera(a)example.com/Maildir/cur took 94 seconds (23191 readdir()s, 0 rename()s to cur/, why=0x1) Aug 2 12:52:55 blade8 dovecot: imap(kavish.karkera(a)example.com): Warning: Maildir /mail/v3store/example.com/kavish.karkera(a)example.com/Maildir: Synchronization took 94 seconds (0 new msgs, 0 flag change attempts, 0 expunge attempts) Aug 2 12:53:54 blade6 dovecot: imap(kavish.karkera(a)example.com): Warning: Maildir: Scanning /mail/v3store/example.com/kavish.karkera(a)example.com/Maildir/cur took 154 seconds (43129 readdir()s, 0 rename()s to cur/, why=0x1) Aug 2 12:53:54 blade6 dovecot: imap(kavish.karkera(a)example.com): Warning: Maildir: Scanning /mail/v3store/example.com/kavish.karkera(a)example.com/Maildir/cur took 162 seconds (43129 readdir()s, 0 rename()s to cur/, why=0xc) Aug 2 12:53:54 blade6 dovecot: imap(kavish.karkera(a)example.com): Warning: Maildir /mail/v3store/example.com/kavish.karkera(a)example.com/Maildir: Synchronization took 162 seconds (0 new msgs, 0 flag change attempts, 0 expunge attempts) Aug 2 12:53:54 blade6 dovecot: imap(kavish.karkera(a)example.com): Warning: Locking transaction log file /indexes//mail/v3store/example.com/kavish.karkera(a)example.com/.INBOX/dovecot.index.log took 50 seconds ----------------------------------------------------------------------------------------------------------------------------------------------------- Thanks In advance, Regards, Kavish Karkera

3 3

[Dovecot] Antispam folder names
by Jost Krieger 05 Aug '13

05 Aug '13

Is there anywhere a collection of folder names for the antispam_trash and antispam_spam configuration? Our problem is that in academia, you have a vast amount of different clients, and some, like Microsoft, use language-specific folder names *on the server*. So we're now collecting folder names to better handle the rare cases. Two more problems: 1. We needed to fix UTF-8 handling in the plugin, so that we kow canb say antispam_trash_pattern_ignorecase = gelöscht* in place of :set sp antispam_trash_pattern_ignorecase = gel&APY-scht* 2. Some clients allow prefixing the IMAP folders with a fixed name, often "INBOX". So we would like to specify something like antispam_trash_pattern_ignorecase = Junk*;*/Junk* which is not supported. Working on a patch ... Yours Jost -- | Helft Spam ausrotten! HTML in Mail ist unhöflich. | | Postmaster, JAPH, manchmal Wahrsager am RZ der RUB | | Wahre Worte sind nicht gefällig, gefällige Worte sind nicht wahr.| | Lao Tse, Tao Te King 81 |

2 1

[Dovecot] Fileoperations in Maildir – problematic or okay?
by Martin Burgraf 05 Aug '13

05 Aug '13

Hi, I'm wondering: Is it a problem to move and delete files inside a dovecot-managed maildir? For example: I have a folder ~/.maildir/.Sub1.Start/cur and a folder ~/.maildir/.Sub1.Start.old/cur and like to regularly move old mails from the first to the second one. Can you create a cron job saying something like: find ~/.maildir/.Sub1.Start/cur -mtime +5 -exec mv {} ~/.maildir/.Sub1.Start.old/cur \; ? I guess this would be some sort of problem, wouldn't it? IIRC dovecot keeps indexes of the files inside a folder and moving them around like this may confuse the software, am I right? bye Martin

3 2

[Dovecot] ./configure dovecot
by LuKreme 03 Aug '13

03 Aug '13

When I do a ./configure in dovecot (2.2.1) I get this notice (first line): configure: WARNING: you should use --build, --host, --target Then, much later on: checking sys/ucred.h presence... yes configure: WARNING: sys/ucred.h: present but cannot be compiled configure: WARNING: sys/ucred.h: check for missing prerequisite headers? configure: WARNING: sys/ucred.h: see the Autoconf documentation configure: WARNING: sys/ucred.h: section "Present But Cannot Be Compiled" configure: WARNING: sys/ucred.h: proceeding with the compiler's result configure: WARNING: ## ---------------------------------- ## configure: WARNING: ## Report this to dovecot(a)dovecot.org ## configure: WARNING: ## ---------------------------------- ## checking for sys/ucred.h... no at the end I get: configure: creating ./config.status Install prefix . : /usr/local File offsets ... : 64bit I/O polling .... : kqueue I/O notifys .... : kqueue SSL ............ : yes (OpenSSL) GSSAPI ......... : no passdbs ........ : static passwd passwd-file pam checkpassword : -shadow -bsdauth -sia -ldap -sql -vpopmail userdbs ........ : static prefetch passwd passwd-file checkpassword nss : -ldap -sql -vpopmail SQL drivers .... : : -pgsql -mysql -sqlite Full text search : squat : -lucene -solr but there is no makefile created. ./configure --help just runs the configure script again, and none of --build --host or --target have any effect on the errors shown, including the first line error. So I think, ok, fine, I'll grab the latest build (2.2.4). Same result. OK, fine. Let's look at 1.2.17. Same thing, only without the cured.h warning. Still the same error on the first line after ./configure and still no make file. There's no big flashing warning anywhere that I can see that says, "Hey, dummy, this isn't going to work." config.log even ends with a "configure: exit 0" (The system already has dovecot 1.2.17 installed, but I was looking at installing $LATEST). -- May you live in interesting times

2 9

[Dovecot] Mail being delivered but MUA showing empty inbox
by Nigel Smith 03 Aug '13

03 Aug '13

Dovecot : 2.0.19 (see config at http://pastebin.com/raw.php?i=KMaQ9Ccc) Postfix : 2.9.6 (see config at http://pastebin.com/raw.php?i=EEeevHfk) I'm trying to get Dovecot and Postfix working on Ubuntu 12.04LTS but am not having much luck. The problem is that although mail gets delivered to the correct place, however pointing a MUA of any description at Dovecot just yields a purportedly empty inbox ! I will first demonstrate correct delivery of mail, then I will show MUA - domain names changed to protect the innocent ! (1) Mail Delivery Aug 2 08:57:17 ukc-vm02-mx01 postfix/smtpd[2640]: connect from unknown[192.168.199.198] Aug 2 08:57:39 ukc-vm02-mx01 postfix/smtpd[2640]: 76AB04470E: client=unknown[192.168.199.198] Aug 2 08:57:47 ukc-vm02-mx01 postfix/cleanup[2644]: 76AB04470E: message-id=<> Aug 2 08:57:47 ukc-vm02-mx01 postfix/qmgr[2093]: 76AB04470E: from=<an(a)example.com>, size=208, nrcpt=1 (queue active) Aug 2 08:57:47 ukc-vm02-mx01 dovecot: lmtp(2192): Debug: none: root=, index=, control=, inbox=, alt= Aug 2 08:57:47 ukc-vm02-mx01 dovecot: lmtp(2192): Connect from local Aug 2 08:57:47 ukc-vm02-mx01 dovecot: lmtp(2192): Debug: auth input: test(a)ops.example.com uid=1001 gid=1001 home=/srv/mail/ops.example.com/test/./ Aug 2 08:57:47 ukc-vm02-mx01 dovecot: lmtp(2192, test(a)ops.example.com) Debug: Effective uid=1001, gid=1001, home=/srv/mail/ops.example.com/test Aug 2 08:57:47 ukc-vm02-mx01 dovecot: lmtp(2192, test(a)ops.example.com) Debug: maildir++: root=/srv/mail/ops.example.com/test, index=, control=, inbox=/srv/mail/ops.example.com/test, alt= Aug 2 08:57:47 ukc-vm02-mx01 dovecot: lmtp(2192, test(a)ops.example.com) ZH9VM3tm+1GQCAAAwf4bjw: msgid=unspecified: saved mail to INBOX Aug 2 08:57:47 ukc-vm02-mx01 postfix/lmtp[2645]: 76AB04470E: to=<test(a)ops.example.com>, relay=ukc-vm02-mx01.ops.example.com[private/dovecot-lmtp], delay=24, delays=24/0.01/0/0.09, dsn=2.0.0, status=sent (250 2.0.0 <test(a)ops.example.com> ZH9VM3tm+1GQCAAAwf4bjw Saved) Aug 2 08:57:47 ukc-vm02-mx01 dovecot: lmtp(2192): Disconnect from local: Client quit (in reset) Aug 2 08:57:47 ukc-vm02-mx01 postfix/qmgr[2093]: 76AB04470E: removed Aug 2 08:57:48 ukc-vm02-mx01 postfix/smtpd[2640]: disconnect from unknown[192.168.199.198] The mail is correctly saved in the right place….. # cat /srv/mail/ops.ops.example.com/test/new/1375430267.M868397P2192.ukc-vm02-mx01\,S\=477\,W\=486 Return-Path: <an(a)example.com> Delivered-To: <test(a)ops.ops.example.com> Received: from ukc-vm02-mx01.ops.ops.example.com by ukc-vm02-mx01.ops.ops.example.com (Dovecot) with LMTP id ZH9VM3tm+1GQCAAAwf4bjw for <test(a)ops.ops.example.com>; Fri, 02 Aug 2013 08:57:47 +0100 Received: from x (unknown [192.168.199.198]) by ukc-vm02-mx01.ops.ops.example.com (Postfix) with ESMTP id 76AB04470E for <test(a)ops.ops.example.com>; Fri, 2 Aug 2013 08:57:24 +0100 (BST) Subject: test (2) MUA * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. a1 LOGIN test(a)ops.example.com test a1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in a2 EXAMINE INBOX * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS ()] Read-only mailbox. * 0 EXISTS * 0 RECENT * OK [UIDVALIDITY 1375424854] UIDs valid * OK [UIDNEXT 1] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest a2 OK [READ-ONLY] Select completed. a3 LOGOUT * BYE Logging out a3 OK Logout completed. closed

5 29

[Dovecot] problems with doveadm
by listserv 03 Aug '13

03 Aug '13

Hello, since few weeks i have problems with the command: doveadm expunge -A mailbox Trash savedbefore 7d => kernel: doveadm[11609]: segfault at 10 ip b75f146a sp bf856ad0 error 4 in libdovecot.so.0.0.0[b7585000+c8000] but for a single user it's works fine: doveadm expunge -u test(a)vtlx.de mailbox Trash savedbefore 30h I hope there is a solution for this. with regards Mario -- 中華人民共和國

1 0

[Dovecot] system_groups_user syntax especially in LDAP
by ray klassen 02 Aug '13

02 Aug '13

so if possible, I'd like an example of how to include system_groups_user in the userdb setup. I'm using ldap, but I could revert to using pam on ldap. There is a ldap query (gleaned from smbldap-tools) that will return a list of groups for a user (&(objectclass=posixGroup)(memberuid=%u)) but I don't know if the ldap driver will handle it but above all I can't figure out how it fits into the dovecot configuration! I'm trying to get Dovecot 2.1.7 on debian wheezy to recognize shared folders that have group ownership.

1 1