On 2016-06-23 11:24, aki.tuomi@dovecot.fi wrote:

...

On June 23, 2016 at 9:06 PM Jeff Gamsby <jeffgamsby@merlock.com> wrote:

I am running ISPConfig 3 on Debian and have managed to install the dovecot-solr and dovecot-fts plugins.

I have solr running undet tomcat at http://localhost:8880 but the indexing is not working.

I am using Dovecot 2.17

I do not understand namespaces and why fts_solr needs them, I just want to index the entire users Maildir.

I am trying to index a users mailbox but am getting the following error:

(changed user name) doveadm fts rescan -u user@user.com inbox doveadm(user@user.com): Error: Namespace prefix not found: inbox

running that in debig mode gives:

doveadm(root): Debug: Loading modules from directory: /usr/lib/dovecot/modules doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib20_fts_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm doveadm(root): Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_zlib_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_zlib_plugin.so: undefined symbol: i_stream_create_deflate (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so doveadm(user@user.com): Debug: Added userdb setting: mail=maildir:/var/vmail/user.com/user/Maildir doveadm(user@user.com): Debug: Added userdb setting: plugin/quota_rule=*:storage=0B doveadm(user@user.com): Debug: Added userdb setting: plugin/sieve=/var/vmail/user.com/user/.sieve doveadm(user@user.com): Debug: Effective uid=5000, gid=5000, home=/var/vmail/user.com/user doveadm(user@user.com): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/var/vmail/user.com/user/Maildir doveadm(user@user.com): Debug: maildir++: root=/var/vmail/user.com/user/Maildir, index=, control=, inbox=/var/vmail/user.com/user/Maildir, alt= doveadm(user@user.com): Debug: fts: No fts setting - plugin disabled doveadm(user@user.com): Error: Namespace prefix not found: inbox

in conf.d/10-mail.conf I have this namespace defined:

namespace inbox { # Namespace type: private, shared or public #type = private

# Hierarchy separator to use. You should use the same separator for all # namespaces or some clients get confused. '/' is usually a good one. # The default however depends on the underlying mail storage format. #separator =

# Prefix required to access this namespace. This needs to be different for # all namespaces. For example "Public/". #prefix =

# Physical location of the mailbox. This is in same format as # mail_location, which is also the default for it. #location =

# There can be only one INBOX, and this setting defines which namespace # has it. inbox = yes

# If namespace is hidden, it's not advertised to clients via NAMESPACE # extension. You'll most likely also want to set list=no. This is mostly # useful when converting from another server with different namespaces which # you want to deprecate but still keep working. For example you can create # hidden namespaces with prefixes "~/mail/", "~%u/mail/" and "mail/". #hidden = no protocol imap { plugin { fts = solr fts_solr = break-imap-search url=http://localhost:8880/solr/ } } protocol pop3 { plugin { fts = solr fts_solr = break-imap-search url=http://localhost:8880/solr/ }

# Show the mailboxes under this namespace with LIST command. This makes the # namespace visible for clients that don't support NAMESPACE extension. # "children" value lists child mailboxes, but hides the namespace prefix. #list = yes

# Namespace handles its own subscriptions. If set to "no", the parent # namespace handles them (empty prefix should always have this as "yes") #subscriptions = yes }

I had to put this in dovecot.conf in order for the plugin to be enabled:

mail_plugins = fts fts_solr

solr is reachable at localhost:8880/solr and appears to be working.

Please help, any suggestions are welcome

Thanks

Can you please send doveconf -n?

Aki Tuomi

I managed to get fts_slor working and now I can index mailboxes, but I am getting a solr error and cannot use the indexes that were created

I get:

Error: fts_solr: Lookup failed: Internal Server Error

dovecot -n

# 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.1 auth_mechanisms = plain login default_vsz_limit = 2 G disable_plaintext_auth = no lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = *,[::] log_timestamp = "%Y-%m-%d %H:%M:%S " mail_plugins = fts fts_solr mail_privileged_group = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { fts = solr fts_autoindex = yes fts_solr = break-imap-search url=http://localhost:8880/solr/ quota = dict:user::file:/var/vmail/%d/%n/.quotausage sieve = ~/.dovecot.sieve sieve_after = /etc/sieve/after sieve_before = /etc/sieve/before sieve_default = /var/vmail/sieve/default.sieve sieve_dir = ~/sieve } protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = vmail mode = 0660 user = vmail } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = root } service imap-login { client_limit = 1000 process_limit = 500 } service imap { vsz_limit = 2 G } ssl_cert = </etc/postfix/smtpd.cert ssl_key = </etc/postfix/smtpd.key userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol pop3 { pop3_uidl_format = %08Xu%08Xv } protocol lda { auth_socket_path = /var/run/dovecot/auth-master info_log_path = /var/log/dovecot-lda.log log_path = /var/log/dovecot-lda-errors.log mail_plugins = fts fts_solr autocreate sieve quota postmaster_address = postmaster@user.com }

And this shows that indexes are correct:

doveadm -Dv index -u jeff@user.com testing

doveadm(jeff@user.com): Info: testing: Cache is already up to date

using solr-schema.xml from dovecot-2.2.13 sources in /etc/solr/conf/schema.xml

dovecot-sql.conf

# http://wiki.dovecot.org/AuthDatabase/SQL # # CREATE TABLE users ( # userid VARCHAR(128) NOT NULL, # password VARCHAR(64) NOT NULL, # home VARCHAR(255) NOT NULL, # uid INTEGER NOT NULL, # gid INTEGER NOT NULL, # active CHAR(1) DEFAULT 'Y' NOT NULL # );

driver = mysql connect = host=localhost dbname=dbispconfig user=ispconfig password=xxxxxxxxxxxxxxxxxxxxx default_pass_scheme = CRYPT

password_query = SELECT password FROM mail_user WHERE (login = '%u' OR email = '%u') AND disable%Ls = 'n' AND server_id = '1' user_query = SELECT email as user, maildir as home, CONCAT('maildir:', maildir, '/Maildir') as mail, uid, gid, CONCAT('*:storage=', quota, 'B') AS quota_rule, CONCAT(maildir, '/.sieve') as sieve FROM mail_user WHERE (login = '%u' OR email = '%u') AND disable%Ls = 'n' AND server_id = '1'

# The iterate_query is required for the doveadm command only and works only on dovecot 2 servers. # Do not enable it on Dovecot 1.x servers iterate_query = SELECT email as user FROM mail_user