Hello Everyone,
I have a small base of users (30), but a lot of emails.
I have an error again when I am indexing a virtual folder with a large number of folders. I appreciate this is a special case, but I am using dovecot and solr as it is, according to the documentation, the favourite way.
One user is using a large number of archives sub folders, by years, months and subfolders, since 5 years. I am convincing this user to give up this organisation, but in the mean time, I hope to solve this problem.
The virtual folder, called Archives, references all the sub folders in the Archive folder. There is about 380 folders, and worst of all, 55k characters with their names concatenated.
On solr, I have increased the value of headerBufferSize to a huge value, but I have still this error.
Anyway, this is the error I have when I try to index this virtual folder:
---8<--------------------------------------------------------------------------------------------------------------------------------------- root@lapetus:/mnt/emails/indiefield.co.uk/jsmith/Mails/Virtual# doveadm -v index -u 'john smith' 'Search/Archives' doveadm(john smith): Info: Search/Archives: Caching mails seq=109580..127362 1800/17783doveadm(john smith): Panic: file solr-connection.c: line 416 (solr_connection_select): assertion failed: (!conn->posting) doveadm(john smith): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x4414a) [0x7fcb7b5b014a] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x2a) [0x7fcb7b5b020a] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fcb7b58782a] -> /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so(+0x754a) [0x7fcb790fa54a] -> /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so(+0x4037) [0x7fcb790f7037] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_backend_get_last_uid+0x5a) [0x7fcb79b2f66a] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xc1fd) [0x7fcb79b341fd] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xc689) [0x7fcb79b34689] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xc51e) [0x7fcb79b3451e] -> doveadm(+0x16719) [0x7fcb7bf71719] -> doveadm(+0x13346) [0x7fcb7bf6e346] -> doveadm(+0x13e21) [0x7fcb7bf6ee21] -> doveadm(doveadm_mail_try_run+0x161) [0x7fcb7bf6f2e1] -> doveadm(main+0x3d1) [0x7fcb7bf6def1] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7fcb7b200ead] -> doveadm(+0x13125) [0x7fcb7bf6e125] Aborted ---8<---------------------------------------------------------------------------------------------------------------------------------------
On the solr side:
---8<--------------------------------------------------------------------------------------------------------------------------------------- 28-Apr-2013 17:13:33 org.apache.solr.update.processor.LogUpdateProcessor finish INFO: {add=[1/70af772eafc17c51fb350000812a8a77/john smith@indiefield.co.uk, 2/70af772eafc17c51fb350000812a8a77/john smith@indiefield.co.uk, 3/70af772eafc17c51fb350000812a8a 77/john smith@indiefield.co.uk, 4/70af772eafc17c51fb350000812a8a77/john smith@indiefield.co.uk, 5/70af772eafc17c51fb350000812a8a77/john smith@indiefield.co.uk, 6/70af772ea fc17c51fb350000812a8a77/john smith@indiefield.co.uk, 7/70af772eafc17c51fb350000812a8a77/john smith@indiefield.co.uk, 8/70af772eafc17c51fb350000812a8a77/john smith@indiefie ld.co.uk, ... (8 added)]} 0 19141 28-Apr-2013 17:13:33 org.apache.solr.common.SolrException log SEVERE: org.apache.solr.common.SolrException: ParseError at [row,col]:[1188418,131] Message: Premature end of file. at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:72) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:54) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1317) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:943) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:843) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) Caused by: javax.xml.stream.XMLStreamException: ParseError at [row,col]:[1188418,131] Message: Premature end of file. at com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl.next(XMLStreamReaderImpl.java:592) at org.apache.solr.handler.XMLLoader.readDoc(XMLLoader.java:273) at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:138) at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:69) ... 22 more ---8<---------------------------------------------------------------------------------------------------------------------------------------
Kind regards, Andre Rodier