[Dovecot] dovecot fts solr plugin
Hello list,
- apache-solr-3.6.2
- dovecot-2.1.7
I configured fts_solr for dovecot, but I have two issues
When doing a search request within telnet 127.0.0.1 143 it connects to solr.
Jul 21, 2013 4:38:15 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr path=/select params={rows=1&q=box:10361d30f8c4ea51ad7c00002ecaff3b+user:" tom@tirism.support.tiri.li"&sort=uid+desc&fl=uid} hits=0 status=0 QTime=10 Jul 21, 2013 4:38:15 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr path=/select params={rows=3&q=(hdr:"test"+OR+body:"test")&fq=%2Bbox:10361d30f8c4ea51ad7c00002ecaff3b+%2Buser:" tom@tirism.support.tiri.li"&sort=uid+asc&fl=uid,score} hits=0 status=0 QTime=7
But
(1) doveadm fts fails:
$ doveadm fts rescan -u tom@domain.com
Error message: doveadm(tom@tirism.support.tiri.li): Error: fts not enabled for user's namespace (null)
(2) solr schema has errors
Jul 21, 2013 4:35:46 PM org.apache.solr.common.SolrException log SEVERE: org.apache.solr.common.SolrException: undefined field text at org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1330) at org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getAnalyzer(IndexSchema.java:408) at org.apache.solr.schema.IndexSchema$SolrIndexAnalyzer.reusableTokenStream(IndexSchema.java:383) at org.apache.lucene.queryParser.QueryParser.getFieldQuery(QueryParser.java:574) at org.apache.solr.search.SolrQueryParser.getFieldQuery(SolrQueryParser.java:206) at org.apache.lucene.queryParser.QueryParser.Term(QueryParser.java:1436) at org.apache.lucene.queryParser.QueryParser.Clause(QueryParser.java:1319) at org.apache.lucene.queryParser.QueryParser.Query(QueryParser.java:1245) at org.apache.lucene.queryParser.QueryParser.TopLevelQuery(QueryParser.java:1234) at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:206) at org.apache.solr.search.LuceneQParser.parse(LuceneQParserPlugin.java:79) at org.apache.solr.search.QParser.getQuery(QParser.java:143) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:105) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:165) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376) at org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:59) at org.apache.solr.core.SolrCore$3.call(SolrCore.java:1182) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314) at java.util.concurrent.FutureTask.run(FutureTask.java:149) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) at java.lang.Thread.run(Thread.java:736)
Hope somebody can help!
Best regards, Thomas
# dovecot --version 2.1.7
# dovecot -n # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.0.74-0.6.10-default x86_64 SUSE Linux Enterprise Server 11 (x86_64) xfs base_dir = /var/run/dovecot/ dict { expire = sqlite:/etc/dovecot/dovecot-dict-sql.conf.ext quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } first_valid_uid = 5000 listen = * login_greeting = tiri Mail ready. login_trusted_networks = 127.0.0.1 mail_location = maildir:/var/vmail/%u/ mail_plugins = fts fts_solr managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } ssl = no userdb { driver = passwd } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol imap { plugin { fts = solr fts_solr = break-imap-search url=http://localhost:8983/solr/ } } protocol pop3 { plugin { fts = solr fts_solr = break-imap-search url=http://localhost:8983/solr/ } }
On 21.7.2013, at 23.11, Thomas Baumann <list.dovecot@tiri.li> wrote:
(1) doveadm fts fails:
Error message: doveadm(tom@tirism.support.tiri.li): Error: fts not enabled for user's namespace (null) .. protocol imap { plugin { fts = solr fts_solr = break-imap-search url=http://localhost:8983/solr/ } } protocol pop3 { plugin { fts = solr fts_solr = break-imap-search url=http://localhost:8983/solr/ } }
Don't put these inside protocol sections. They need to be enabled globally.
(2) solr schema has errors
Jul 21, 2013 4:35:46 PM org.apache.solr.common.SolrException log SEVERE: org.apache.solr.common.SolrException: undefined field text
I did some Solr schema fixes to v2.2 recently..
Hello Timo,
I did following (using apache-solr-3.6.2):
wget "http://hg.dovecot.org/dovecot-2.0/raw-file/3fbfdda3e5d3/src/plugins/fts-s olr/schema.xml" -O solr/conf/schema.xml
Then I started solr
java -jar start.jar
And and SEVERE Error is thrown, but searching seems to work.
05.08.2013 14:22:29 org.apache.solr.common.SolrException log SEVERE: org.apache.solr.common.SolrException: undefined field text at org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:13 30) at org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getAnalyzer(IndexSche ma.java:408) at org.apache.solr.schema.IndexSchema$SolrIndexAnalyzer.reusableTokenStream(I ndexSchema.java:383) at org.apache.lucene.queryParser.QueryParser.getFieldQuery(QueryParser.java:5 74) at org.apache.solr.search.SolrQueryParser.getFieldQuery(SolrQueryParser.java: 206) at org.apache.lucene.queryParser.QueryParser.Term(QueryParser.java:1436) at org.apache.lucene.queryParser.QueryParser.Clause(QueryParser.java:1319) at org.apache.lucene.queryParser.QueryParser.Query(QueryParser.java:1245) at org.apache.lucene.queryParser.QueryParser.TopLevelQuery(QueryParser.java:1 234) at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:206) at org.apache.solr.search.LuceneQParser.parse(LuceneQParserPlugin.java:79) at org.apache.solr.search.QParser.getQuery(QParser.java:143) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.ja va:105) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHa ndler.java:165) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBas e.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376) at org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.j ava:59) at org.apache.solr.core.SolrCore$3.call(SolrCore.java:1182) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314) at java.util.concurrent.FutureTask.run(FutureTask.java:149) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor. java:897) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java :919) at java.lang.Thread.run(Thread.java:736)
And for solr-4.3.1 the schema is not useable. "Caused by: org.apache.solr.common.SolrException: Plugin init failure for [schema.xml] analyzer/filter: Error loading class 'solr.EnglishPorterFilterFactory'"
**Do you have a Schema for 4.3.1 as well?**
10361d30f8c4ea51ad7c00002ecaff3b 1 10361d30f8c4ea51ad7c00002ecaff3b 2 10361d30f8c4ea51ad7c00002ecaff3b 3
Thanks for your help in advance, Thomas.
-----Ursprüngliche Nachricht----- Von: Timo Sirainen [mailto:tss@iki.fi] Gesendet: Freitag, 2. August 2013 15:03 An: Thomas Baumann Cc: dovecot@dovecot.org Betreff: Re: [Dovecot] dovecot fts solr plugin
On 21.7.2013, at 23.11, Thomas Baumann <list.dovecot@tiri.li> wrote:
(1) doveadm fts fails:
Error message: doveadm(tom@tirism.support.tiri.li): Error: fts not enabled for user's namespace (null) .. protocol imap { plugin { fts = solr fts_solr = break-imap-search url=http://localhost:8983/solr/ } } protocol pop3 { plugin { fts = solr fts_solr = break-imap-search url=http://localhost:8983/solr/ } }
Don't put these inside protocol sections. They need to be enabled globally.
(2) solr schema has errors
Jul 21, 2013 4:35:46 PM org.apache.solr.common.SolrException log SEVERE: org.apache.solr.common.SolrException: undefined field text
I did some Solr schema fixes to v2.2 recently..
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Thomas Baumann wrote the following on 05.08.2013 19:27:
Regarding: "org.apache.solr.common.SolrException log SEVERE: org.apache.solr.common.SolrException: undefined field text"
As for Solr 4.4.0 this went away after editing solr-schema.xml (from Dovecot 2.2.4), Changing the call to: fieldType name="text" to fieldType name="text_general" Changing all calls to field name type="text" to type="text_general" and by adding: <field name="text" type="text_general" indexed="true" stored="false" multiValued="true"/> before: <field name="_version_" type="long" indexed="true" stored="true"/>
Although fts solr searches are working fine even with the above errors in the logs. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (MingW32)
iQEcBAEBAgAGBQJR/8DCAAoJEFJuhDv/UPjwuFoIALV/W4Az52nTEImAPPu6cE6l H95rVMFpQ2bvIyyjY9I0AZrPB/Fk2RudVGVJTj3AXwCFph/nKEJ2Rc1AZsio5HQz evJA1KR+U7mXh2SEw0mADUM83JTQPCiBipu5g3w91XWX8D9kvYBRmSmDwuYkDp94 byLqxwEbJ5f8AeNDhYQ5QNmBYd/j7r8e5bYEzxcjHr/ICXkCIWiO/mvV1/NlHYE6 QS3SVEx9FLDOZoAb0c/UDvQyF/khUDZ028b1Afo1b4qHVU0r41KccoTvHgBl6sl5 x1+4f+qQLP4I5NZohfOxBHvU0jaSQk+jBDXuQU+FNuqkfwlkbhLTQdwgBcx22o8= =7PcO -----END PGP SIGNATURE-----
On 5.8.2013, at 18.12, Tamsy <dovecot-list@mohtex.net> wrote:
Regarding: "org.apache.solr.common.SolrException log SEVERE: org.apache.solr.common.SolrException: undefined field text"
I wonder what tries to access such "text" field .. I don't see any such code in Dovecot. Or does Solr assume that there always is a field named "text"?
As for Solr 4.4.0 this went away after editing solr-schema.xml (from Dovecot 2.2.4), Changing the call to: fieldType name="text" to fieldType name="text_general" Changing all calls to field name type="text" to type="text_general" and by adding: <field name="text" type="text_general" indexed="true" stored="false" multiValued="true"/> before: <field name="_version_" type="long" indexed="true" stored="true"/>
So basically you're adding a new field named "text" that is always empty (because Dovecot Solr plugin doesn't add anything to it)? I'm not sure if the text_general changes did anything good.
participants (3)
-
Tamsy
-
Thomas Baumann
-
Timo Sirainen