doveadm index: can't index a different namespace?
Timo Sirainen
tss at iki.fi
Wed Nov 2 20:14:20 UTC 2016
On 02 Nov 2016, at 22:02, Larry Rosenman <larryrtx at gmail.com> wrote:
>
> why wouldn't it be able to use the ~/MAILARCHIVE/ filesystem space?
>
> The other indexes etc seem to get built there.
It's not about adding an index there, but about trying to open "MAILARCHIVE" itself as a folder. fts-lucene attempts to use an existing folder's indexes for tracking some (rather useless) settings information. I guess that tracking could be removed entirely as well.
>
> I'll see if the current FreeBSD solr port is easier to set up than the last time I tried.
>
>
> On Wed, Nov 2, 2016 at 2:58 PM, Timo Sirainen <tss at iki.fi <mailto:tss at iki.fi>> wrote:
> I think it's simply not going to work with mbox. It's trying to open the "#ARCHIVE" namespace prefix itself, which won't work and it's too much work to try to fix it. You could try if this happens to make it work, but I'm not sure: http://wiki2.dovecot.org/MboxChildFolders <http://wiki2.dovecot.org/MboxChildFolders>
>
> Also instead of fts-lucene, you should switch to fts-solr. That would also fix it and in general works better.
>
>
>> On 02 Nov 2016, at 21:40, Larry Rosenman <larryrtx at gmail.com <mailto:larryrtx at gmail.com>> wrote:
>>
>> Is there anything else I can provide?
>>
>>
>> On Wed, Nov 2, 2016 at 1:30 PM, Larry Rosenman <larryrtx at gmail.com <mailto:larryrtx at gmail.com>> wrote:
>> nope -- No difference
>>
>> thebighonker.lerctr.org <http://thebighonker.lerctr.org/> ~ $ ls -l ~/MAILARCHIVE/
>> total 176
>> drwx------ 6 ler ler 7 May 25 2014 2004
>> drwx------ 8 ler ler 8 Mar 3 2013 2005
>> drwx------ 11 ler ler 11 Mar 3 2013 2006
>> drwx------ 11 ler ler 11 May 3 2013 2007
>> drwx------ 14 ler ler 14 Mar 3 2013 2008
>> drwx------ 14 ler ler 14 Mar 3 2013 2009
>> drwx------ 14 ler ler 14 Mar 3 2013 2010
>> drwx------ 15 ler ler 15 Mar 3 2013 2011
>> drwx------ 15 ler ler 15 Mar 3 2013 2012
>> drwx------ 14 ler ler 14 Dec 31 2013 2013
>> drwxr-xr-x 14 ler ler 14 Jan 1 2015 2014
>> drwxr-xr-x 14 ler ler 14 Jan 1 2016 2015
>> drwxr-xr-x 12 ler ler 12 Nov 1 00:05 2016
>> -rw-r--r-- 1 ler ler 0 Nov 2 13:26 inbox
>> thebighonker.lerctr.org <http://thebighonker.lerctr.org/> ~ $
>>
>> doveadm(ler): Error: Mailbox #ARCHIVE/2013/04/Amplify_Mail: Status lookup failed: Internal error occurred. Refer to server log for more information. [2016-11-02 13:28:28]
>> doveadm(ler): Error: lucene: Failed to sync mailbox INBOX: Mailbox isn't selectable
>> doveadm(ler): Error: Mailbox #ARCHIVE/2013/04/clamav-rules: Status lookup failed: Internal error occurred. Refer to server log for more information. [2016-11-02 13:28:28]
>> doveadm(ler): Error: lucene: Failed to sync mailbox INBOX: Mailbox isn't selectable
>> doveadm(ler): Error: Mailbox #ARCHIVE/2013/04/SA/FN: Status lookup failed: Internal error occurred. Refer to server log for more information. [2016-11-02 13:28:28]
>> thebighonker.lerctr.org <http://thebighonker.lerctr.org/> ~ $
>>
>> thebighonker.lerctr.org <http://thebighonker.lerctr.org/> ~ $ doveadm -D -vvvvv index \#ARCHIVE/\*
>> Debug: Loading modules from directory: /usr/local/lib/dovecot
>> Debug: Module loaded: /usr/local/lib/dovecot/lib15_notify_plugin.so
>> Debug: Module loaded: /usr/local/lib/dovecot/lib20_fts_plugin.so
>> Debug: Module loaded: /usr/local/lib/dovecot/lib21_fts_lucene_plugin.so
>> Debug: Module loaded: /usr/local/lib/dovecot/lib90_stats_plugin.so
>> Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm
>> Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: Undefined symbol "acl_user_module" (this is usually intentional, so just ignore this message)
>> Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so: Undefined symbol "expire_set_lookup" (this is usually intentional, so just ignore this message)
>> Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so: Undefined symbol "quota_user_module" (this is usually intentional, so just ignore this message)
>> Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib10_doveadm_sieve_plugin.so
>> Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_lucene_plugin.so
>> Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so
>> doveadm(ler): Debug: Effective uid=1001, gid=1001, home=/home/ler
>> doveadm(ler): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail:INBOX=~/mail/INBOX
>> doveadm(ler): Debug: fs: root=/home/ler/mail, index=, indexpvt=, control=, inbox=/home/ler/mail/INBOX, alt=
>> doveadm(ler): Debug: Namespace archive: type=private, prefix=#ARCHIVE/, sep=/, inbox=no, hidden=no, list=no, subscriptions=yes location=mbox:~/MAILARCHIVE
>> doveadm(ler): Debug: mbox: INBOX defaulted to /home/ler/MAILARCHIVE/inbox
>> doveadm(ler): Debug: fs: root=/home/ler/MAILARCHIVE, index=, indexpvt=, control=, inbox=/home/ler/MAILARCHIVE/inbox, alt=
>> doveadm(ler): Error: lucene: Failed to sync mailbox INBOX: Mailbox isn't selectable
>> doveadm(ler): Error: Mailbox #ARCHIVE/2011/07/satelliteguys: Status lookup failed: Internal error occurred. Refer to server log for more information. [2016-11-02 13:27:19]
>> doveadm(ler): Error: lucene: Failed to sync mailbox INBOX: Mailbox isn't selectable
>> doveadm(ler): Error: Mailbox #ARCHIVE/2011/07/SA/FN: Status lookup failed: Internal error occurred. Refer to server log for more information. [2016-11-02 13:27:19]
>> doveadm(ler): Error: lucene: Failed to sync mailbox INBOX: Mailbox isn't selectable
>> doveadm(ler): Error: Mailbox #ARCHIVE/2011/07/SENT: Status lookup failed: Internal error occurred. Refer to server log for more information. [2016-11-02 13:27:19]
>> doveadm(ler): Error: lucene: Failed to sync mailbox INBOX: Mailbox isn't selectable
>> doveadm(ler): Error: Mailbox #ARCHIVE/2011/07/INBOX: Status lookup failed: Internal error occurred. Refer to server log for more information. [2016-11-02 13:27:19]
>> doveadm(ler): Error: lucene: Failed to sync mailbox INBOX: Mailbox isn't selectable
>> doveadm(ler): Error: Mailbox #ARCHIVE/2011/07/SPAM: Status lookup failed: Internal error occurred. Refer to server log for more information. [2016-11-02 13:27:19]
>> doveadm(ler): Error: lucene: Failed to sync mailbox INBOX: Mailbox isn't selectable
>> doveadm(ler): Error: Mailbox #ARCHIVE/2011/07/root-mail: Status lookup failed: Internal error occurred. Refer to server log for more information. [2016-11-02 13:27:19]
>> doveadm(ler): Error: lucene: Failed to sync mailbox INBOX: Mailbox isn't selectable
>> doveadm(ler): Error: Mailbox #ARCHIVE/2011/07/bacula: Status lookup failed: Internal error occurred. Refer to server log for more information. [2016-11-02 13:27:19]
>> doveadm(ler): Error: lucene: Failed to sync mailbox INBOX: Mailbox isn't selectable
>> doveadm(ler): Error: Mailbox #ARCHIVE/2011/07/Junk E-mail: Status lookup failed: Internal error occurred. Refer to server log for more information. [2016-11-02 13:27:19]
>> doveadm(ler): Error: lucene: Failed to sync mailbox INBOX: Mailbox isn't selectable
>> doveadm(ler): Error: Mailbox #ARCHIVE/2011/07/Amplify_Mail: Status lookup failed: Internal error occurred. Refer to server log for more information. [2016-11-02 13:27:19]
>> doveadm(ler): Error: lucene: Failed to sync mailbox INBOX: Mailbox isn't selectable
>>
>> On Wed, Nov 2, 2016 at 1:25 PM, Timo Sirainen <tss at iki.fi <mailto:tss at iki.fi>> wrote:
>> It looks like fts-lucene wants to write some information into INBOX's index files. But your archive namespace doesn't have a working INBOX. Maybe it would work if you create "#ARCHIVE/inbox"?
>>
>>> On 02 Nov 2016, at 20:13, Larry Rosenman <larryrtx at gmail.com <mailto:larryrtx at gmail.com>> wrote:
>>>
>>> one other thing, fts_autoindex does NOT appear to be working on the
>>> #ARCHIVE/ NAMESPACE.
>>>
>>> I wonder if these are all interrelated?
>>>
>>> On Wed, Nov 2, 2016 at 1:10 PM, Larry Rosenman <larryrtx at gmail.com <mailto:larryrtx at gmail.com>> wrote:
>>> FWIW it DOES work on my default NAMESPACE, just not the #ARCHIVE/ NAMESPACE.
>>>
>>>
>>> On Wed, Nov 2, 2016 at 12:32 PM, Larry Rosenman <larryrtx at gmail.com <mailto:larryrtx at gmail.com>> wrote:
>>> It's ALWAYS failed on the other NAMESPACE.
>>>
>>> Some of the logging looks weird as well for the 2nd namespace:
>>>
>>> Nov 2 12:32:03 thebighonker dovecot: imap-login: Login: user=<mrm>, method=PLAIN, rip=207.231.91.36, lip=192.147.25.65, mpid=76712, TLS, session=<HC+00VRAWczP51sk>
>>> Nov 2 12:32:03 thebighonker dovecot: imap(mrm): Debug: Loading modules from directory: /usr/local/lib/dovecot
>>> Nov 2 12:32:03 thebighonker dovecot: imap(mrm): Debug: Module loaded: /usr/local/lib/dovecot/lib15_notify_plugin.so
>>> Nov 2 12:32:03 thebighonker dovecot: imap(mrm): Debug: Module loaded: /usr/local/lib/dovecot/lib20_fts_plugin.so
>>> Nov 2 12:32:03 thebighonker dovecot: imap(mrm): Debug: Module loaded: /usr/local/lib/dovecot/lib20_mail_log_plugin.so
>>> Nov 2 12:32:03 thebighonker dovecot: imap(mrm): Debug: Module loaded: /usr/local/lib/dovecot/lib21_fts_lucene_plugin.so
>>> Nov 2 12:32:03 thebighonker dovecot: imap(mrm): Debug: Module loaded: /usr/local/lib/dovecot/lib90_stats_plugin.so
>>> Nov 2 12:32:03 thebighonker dovecot: imap(mrm): Debug: Module loaded: /usr/local/lib/dovecot/lib95_imap_stats_plugin.so
>>> Nov 2 12:32:03 thebighonker dovecot: imap(mrm): Debug: Effective uid=1004, gid=1004, home=/home/mrm
>>> Nov 2 12:32:03 thebighonker dovecot: imap(mrm): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail:INBOX=~/mail/INBOX
>>> Nov 2 12:32:03 thebighonker dovecot: imap(mrm): Debug: fs: root=/home/mrm/mail, index=, indexpvt=, control=, inbox=/home/mrm/mail/INBOX, alt=
>>> Nov 2 12:32:03 thebighonker dovecot: imap(mrm): Debug: Namespace archive: type=private, prefix=#ARCHIVE/, sep=/, inbox=no, hidden=no, list=no, subscriptions=yes location=mbox:~/MAILARCHIVE
>>> Nov 2 12:32:03 thebighonker dovecot: imap(mrm): Debug: mbox: INBOX defaulted to /home/mrm/MAILARCHIVE/inbox
>>> Nov 2 12:32:03 thebighonker dovecot: imap(mrm): Debug: fs: root=/home/mrm/MAILARCHIVE, index=, indexpvt=, control=, inbox=/home/mrm/MAILARCHIVE/inbox, alt=
>>>
>>>
>>>
>>> On Wed, Nov 2, 2016 at 12:30 PM, Timo Sirainen <tss at iki.fi <mailto:tss at iki.fi>> wrote:
>>> It's getting a bit strange. You were running 2.2.25 previously and it was working then? Does the doveadm index always fail with the same errors?
>>>
>>> Also I just noticed that it's saying INBOX isn't selectable: "lucene: Failed to sync mailbox INBOX: Mailbox isn't selectable". Which I guess isn't true. It seems to be failing somewhat randomly..
>>>
>>>> On 02 Nov 2016, at 19:25, Larry Rosenman <larryrtx at gmail.com <mailto:larryrtx at gmail.com>> wrote:
>>>>
>>>> thebighonker.lerctr.org <http://thebighonker.lerctr.org/> ~ $ doveadm mailbox status -u ler at lerctr.org <mailto:ler at lerctr.org> vsize \#ARCHIVE/2016/10/clamav-rules
>>>> #ARCHIVE/2016/10/clamav-rules vsize=3840752
>>>> thebighonker.lerctr.org <http://thebighonker.lerctr.org/> ~ $
>>>>
>>>>
>>>>
>>>> On Wed, Nov 2, 2016 at 12:22 PM, Timo Sirainen <tss at iki.fi <mailto:tss at iki.fi>> wrote:
>>>> On 02 Nov 2016, at 19:19, Larry Rosenman <larryrtx at gmail.com <mailto:larryrtx at gmail.com>> wrote:
>>>> >
>>>> > clamav-rules is NOT a directory, nor are any of the others:
>>>>
>>>> Can you access these mboxes at all with Dovecot? e.g.
>>>>
>>>> doveadm mailbox status -u user at domain clamav-rules
>>>>
>>>> > thebighonker.lerctr.org <http://thebighonker.lerctr.org/> ~/MAILARCHIVE $ cd 2016/10
>>>> > thebighonker.lerctr.org <http://thebighonker.lerctr.org/> ~/MAILARCHIVE/2016/10 $ ls -l
>>>> > total 72106
>>>> > -rw-r--r-- 1 ler ler 440461 Nov 1 00:05 Amplify_Mail
>>>> > -rw-r--r-- 1 ler ler 104102478 Nov 1 14:53 INBOX
>>>> > drwxr-xr-x 3 ler ler 4 Nov 1 00:05 SA
>>>> > -rw-r--r-- 1 ler ler 21083767 Nov 1 00:05 SENT
>>>> > -rw-r--r-- 1 ler ler 1850677 Nov 1 00:05 SPAM
>>>> > -rw-r--r-- 1 ler ler 36483 Nov 1 00:05 UFCU_Mail
>>>> > -rw-r--r-- 1 ler ler 460073 Nov 1 00:05 bacula
>>>> > -rw-r--r-- 1 ler ler 1107098 Nov 1 00:05 bayes-update
>>>> > -rw-r--r-- 1 ler ler 3884041 Nov 1 00:05 clamav-rules
>>>> > -rw-r--r-- 1 ler ler 652132 Nov 1 00:05 firewall
>>>> > drwxr-xr-x 11 ler ler 20 Nov 1 00:05 lists
>>>> > -rw-r--r-- 1 ler ler 25122465 Nov 1 00:05 root-mail
>>>> > -rw-r--r-- 1 ler ler 44553 Nov 1 00:05 satelliteguys
>>>> > -rw-r--r-- 1 ler ler 625628 Nov 1 00:05 wview
>>>> > thebighonker.lerctr.org <http://thebighonker.lerctr.org/> ~/MAILARCHIVE/2016/10 $
>>>> >
>>>> > On Wed, Nov 2, 2016 at 12:17 PM, Timo Sirainen <tss at iki.fi <mailto:tss at iki.fi>> wrote:
>>>> >
>>>> >> On 01 Nov 2016, at 20:30, Larry Rosenman <larryrtx at gmail.com <mailto:larryrtx at gmail.com>> wrote:
>>>> >>>
>>>> >>> doveadm -D -vvvvvv index \#ARCHIVE/\* garners the below for ALL
>>>> >> mailboxes
>>>> >>> in the
>>>> >>> namespace:
>>>> >>> doveadm(ler): Error: Mailbox #ARCHIVE/2013/04/clamav-rules: Status lookup
>>>> >>> failed: Internal error occurred. Refer to server log for more
>>>> >> information.
>>>> >>> [2016-11-01 13:25:21]
>>>> >>> doveadm(ler): Error: lucene: Failed to sync mailbox INBOX: Mailbox isn't
>>>> >>> selectable
>>>> >>
>>>> >> You're using mbox format and "clamav-rules" is a directory. I guess
>>>> >> doveadm should be ignoring non-selectable mailboxes.
>>>> >>
>>>> >>
>>>> >
>>>> >
>>>> > --
>>>> > Larry Rosenman http://www.lerctr.org/~ler <http://www.lerctr.org/~ler>
>>>> > Phone: +1 214-642-9640 <tel:%2B1%20214-642-9640> (c) E-Mail: larryrtx at gmail.com <mailto:larryrtx at gmail.com>
>>>> > US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Larry Rosenman http://www.lerctr.org/~ler <http://www.lerctr.org/~ler>
>>>> Phone: +1 214-642-9640 <tel:%2B1%20214-642-9640> (c) E-Mail: larryrtx at gmail.com <mailto:larryrtx at gmail.com>
>>>> US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281
>>>
>>>
>>>
>>>
>>> --
>>> Larry Rosenman http://www.lerctr.org/~ler <http://www.lerctr.org/~ler>
>>> Phone: +1 214-642-9640 <tel:%2B1%20214-642-9640> (c) E-Mail: larryrtx at gmail.com <mailto:larryrtx at gmail.com>
>>> US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281
>>>
>>>
>>>
>>> --
>>> Larry Rosenman http://www.lerctr.org/~ler <http://www.lerctr.org/~ler>
>>> Phone: +1 214-642-9640 <tel:%2B1%20214-642-9640> (c) E-Mail: larryrtx at gmail.com <mailto:larryrtx at gmail.com>
>>> US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281
>>>
>>>
>>>
>>> --
>>> Larry Rosenman http://www.lerctr.org/~ler <http://www.lerctr.org/~ler>
>>> Phone: +1 214-642-9640 <tel:%2B1%20214-642-9640> (c) E-Mail: larryrtx at gmail.com <mailto:larryrtx at gmail.com>
>>> US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281
>>
>>
>>
>>
>> --
>> Larry Rosenman http://www.lerctr.org/~ler <http://www.lerctr.org/~ler>
>> Phone: +1 214-642-9640 <tel:%2B1%20214-642-9640> (c) E-Mail: larryrtx at gmail.com <mailto:larryrtx at gmail.com>
>> US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281
>>
>>
>>
>> --
>> Larry Rosenman http://www.lerctr.org/~ler <http://www.lerctr.org/~ler>
>> Phone: +1 214-642-9640 <tel:%2B1%20214-642-9640> (c) E-Mail: larryrtx at gmail.com <mailto:larryrtx at gmail.com>
>> US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281
>
>
>
>
> --
> Larry Rosenman http://www.lerctr.org/~ler <http://www.lerctr.org/~ler>
> Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com <mailto:larryrtx at gmail.com>
> US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281
More information about the dovecot
mailing list