fts_solr not working
Jeff Gamsby
jeffgamsby at merlock.com
Sun Jun 26 18:56:15 UTC 2016
On 2016-06-23 11:24, aki.tuomi at dovecot.fi wrote:
>> On June 23, 2016 at 9:06 PM Jeff Gamsby <jeffgamsby at merlock.com>
>> wrote:
>>
>>
>> I am running ISPConfig 3 on Debian and have managed to install the
>> dovecot-solr and dovecot-fts plugins.
>>
>> I have solr running undet tomcat at http://localhost:8880 but the
>> indexing is not working.
>>
>> I am using Dovecot 2.17
>>
>> I do not understand namespaces and why fts_solr needs them, I just
>> want
>> to index the entire users Maildir.
>>
>> I am trying to index a users mailbox but am getting the following
>> error:
>>
>> (changed user name)
>> doveadm fts rescan -u user at user.com inbox
>> doveadm(user at user.com): Error: Namespace prefix not found: inbox
>>
>> running that in debig mode gives:
>>
>> doveadm(root): Debug: Loading modules from directory:
>> /usr/lib/dovecot/modules
>> doveadm(root): Debug: Module loaded:
>> /usr/lib/dovecot/modules/lib20_fts_plugin.so
>> doveadm(root): Debug: Module loaded:
>> /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so
>> doveadm(root): Debug: Loading modules from directory:
>> /usr/lib/dovecot/modules/doveadm
>> doveadm(root): Debug: Skipping module doveadm_acl_plugin, because
>> dlopen() failed:
>> /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so:
>> undefined
>> symbol: acl_user_module (this is usually intentional, so just ignore
>> this message)
>> doveadm(root): Debug: Skipping module doveadm_expire_plugin, because
>> dlopen() failed:
>> /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so:
>> undefined symbol: expire_set_deinit (this is usually intentional, so
>> just ignore this message)
>> doveadm(root): Debug: Skipping module doveadm_quota_plugin, because
>> dlopen() failed:
>> /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so:
>> undefined symbol: quota_user_module (this is usually intentional, so
>> just ignore this message)
>> doveadm(root): Debug: Skipping module doveadm_zlib_plugin, because
>> dlopen() failed:
>> /usr/lib/dovecot/modules/doveadm/lib10_doveadm_zlib_plugin.so:
>> undefined
>> symbol: i_stream_create_deflate (this is usually intentional, so just
>> ignore this message)
>> doveadm(root): Debug: Module loaded:
>> /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so
>> doveadm(user at user.com): Debug: Added userdb setting:
>> mail=maildir:/var/vmail/user.com/user/Maildir
>> doveadm(user at user.com): Debug: Added userdb setting:
>> plugin/quota_rule=*:storage=0B
>> doveadm(user at user.com): Debug: Added userdb setting:
>> plugin/sieve=/var/vmail/user.com/user/.sieve
>> doveadm(user at user.com): Debug: Effective uid=5000, gid=5000,
>> home=/var/vmail/user.com/user
>> doveadm(user at user.com): Debug: Namespace inbox: type=private, prefix=,
>> sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes
>> location=maildir:/var/vmail/user.com/user/Maildir
>> doveadm(user at user.com): Debug: maildir++:
>> root=/var/vmail/user.com/user/Maildir, index=, control=,
>> inbox=/var/vmail/user.com/user/Maildir, alt=
>> doveadm(user at user.com): Debug: fts: No fts setting - plugin disabled
>> doveadm(user at user.com): Error: Namespace prefix not found: inbox
>>
>> in conf.d/10-mail.conf I have this namespace defined:
>>
>> namespace inbox {
>> # Namespace type: private, shared or public
>> #type = private
>>
>> # Hierarchy separator to use. You should use the same separator for
>> all
>> # namespaces or some clients get confused. '/' is usually a good
>> one.
>> # The default however depends on the underlying mail storage
>> format.
>> #separator =
>>
>> # Prefix required to access this namespace. This needs to be
>> different
>> for
>> # all namespaces. For example "Public/".
>> #prefix =
>>
>> # Physical location of the mailbox. This is in same format as
>> # mail_location, which is also the default for it.
>> #location =
>>
>> # There can be only one INBOX, and this setting defines which
>> namespace
>> # has it.
>> inbox = yes
>>
>> # If namespace is hidden, it's not advertised to clients via
>> NAMESPACE
>> # extension. You'll most likely also want to set list=no. This is
>> mostly
>> # useful when converting from another server with different
>> namespaces
>> which
>> # you want to deprecate but still keep working. For example you can
>> create
>> # hidden namespaces with prefixes "~/mail/", "~%u/mail/" and
>> "mail/".
>> #hidden = no
>> protocol imap {
>> plugin {
>> fts = solr
>> fts_solr = break-imap-search url=http://localhost:8880/solr/
>> }
>> }
>> protocol pop3 {
>> plugin {
>> fts = solr
>> fts_solr = break-imap-search url=http://localhost:8880/solr/
>> }
>>
>> # Show the mailboxes under this namespace with LIST command. This
>> makes the
>> # namespace visible for clients that don't support NAMESPACE
>> extension.
>> # "children" value lists child mailboxes, but hides the namespace
>> prefix.
>> #list = yes
>>
>> # Namespace handles its own subscriptions. If set to "no", the
>> parent
>> # namespace handles them (empty prefix should always have this as
>> "yes")
>> #subscriptions = yes
>> }
>>
>>
>> I had to put this in dovecot.conf in order for the plugin to be
>> enabled:
>>
>> mail_plugins = fts fts_solr
>>
>> solr is reachable at localhost:8880/solr and appears to be working.
>>
>> Please help, any suggestions are welcome
>>
>> Thanks
>
> Can you please send doveconf -n?
> ---
> Aki Tuomi
I managed to get fts_slor working and now I can index mailboxes, but I
am getting a solr error and cannot use the indexes that were created
I get:
Error: fts_solr: Lookup failed: Internal Server Error
dovecot -n
# 2.2.13: /etc/dovecot/dovecot.conf
# OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.1
auth_mechanisms = plain login
default_vsz_limit = 2 G
disable_plaintext_auth = no
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
listen = *,[::]
log_timestamp = "%Y-%m-%d %H:%M:%S "
mail_plugins = fts fts_solr
mail_privileged_group = vmail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date ihave
namespace inbox {
inbox = yes
location =
prefix =
}
passdb {
args = /etc/dovecot/dovecot-sql.conf
driver = sql
}
plugin {
fts = solr
fts_autoindex = yes
fts_solr = break-imap-search url=http://localhost:8880/solr/
quota = dict:user::file:/var/vmail/%d/%n/.quotausage
sieve = ~/.dovecot.sieve
sieve_after = /etc/sieve/after
sieve_before = /etc/sieve/before
sieve_default = /var/vmail/sieve/default.sieve
sieve_dir = ~/sieve
}
protocols = imap pop3
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
}
unix_listener auth-master {
group = vmail
mode = 0660
user = vmail
}
unix_listener auth-userdb {
group = vmail
mode = 0600
user = vmail
}
user = root
}
service imap-login {
client_limit = 1000
process_limit = 500
}
service imap {
vsz_limit = 2 G
}
ssl_cert = </etc/postfix/smtpd.cert
ssl_key = </etc/postfix/smtpd.key
userdb {
args = /etc/dovecot/dovecot-sql.conf
driver = sql
}
protocol pop3 {
pop3_uidl_format = %08Xu%08Xv
}
protocol lda {
auth_socket_path = /var/run/dovecot/auth-master
info_log_path = /var/log/dovecot-lda.log
log_path = /var/log/dovecot-lda-errors.log
mail_plugins = fts fts_solr autocreate sieve quota
postmaster_address = postmaster at user.com
}
And this shows that indexes are correct:
doveadm -Dv index -u jeff at user.com testing
doveadm(jeff at user.com): Info: testing: Cache is already up to date
using solr-schema.xml from dovecot-2.2.13 sources in
/etc/solr/conf/schema.xml
dovecot-sql.conf
# http://wiki.dovecot.org/AuthDatabase/SQL
#
# CREATE TABLE users (
# userid VARCHAR(128) NOT NULL,
# password VARCHAR(64) NOT NULL,
# home VARCHAR(255) NOT NULL,
# uid INTEGER NOT NULL,
# gid INTEGER NOT NULL,
# active CHAR(1) DEFAULT 'Y' NOT NULL
# );
driver = mysql
connect = host=localhost dbname=dbispconfig user=ispconfig
password=xxxxxxxxxxxxxxxxxxxxx
default_pass_scheme = CRYPT
password_query = SELECT password FROM mail_user WHERE (login = '%u' OR
email = '%u') AND `disable%Ls` = 'n' AND server_id = '1'
user_query = SELECT email as user, maildir as home, CONCAT('maildir:',
maildir, '/Maildir') as mail, uid, gid, CONCAT('*:storage=', quota, 'B')
AS quota_rule, CONCAT(maildir, '/.sieve') as sieve FROM mail_user WHERE
(login = '%u' OR email = '%u') AND `disable%Ls` = 'n' AND server_id =
'1'
# The iterate_query is required for the doveadm command only and works
only on dovecot 2 servers.
# Do not enable it on Dovecot 1.x servers
iterate_query = SELECT email as user FROM mail_user
More information about the dovecot
mailing list