[Dovecot] Migration to Dovecot 2.2.12 - How to trigger full site indexing
Hi list,
We are in the process of migrating our old (dovecot 1.2.4 based) mail system to the new one:
- Centos 6 x86_64
- Dovecot 2.2.12
- Users in an openldap directory
- clucene FTS
I'm trying to trigger a full indexing for all the user and all their mailboxes (avoiding them to individually trigger it using a search within their MUA). I'm trying to use the "doveadm index" command.
First question, is there some sort of 'wildcard' option for the name of the mailbox to index, as I dont know all the mailboxes's names the users have created (beside the regular Sent Inbox Trash and Drafts ?
Second question I've tried: # doveadm -D index -A Sent which gave me: doveadm(root): Error: User listing returned failure doveadm: Error: Failed to iterate through some users
Can the '-A' option work with ldap backend ?
Thanks a lot and have a nice day, kfx
# 2.2.12: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-431.11.2.el6.x86_64 x86_64 CentOS release 6.5 (Final) ext4 auth_cache_size = 10 M auth_debug = yes auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes auth_verbose_passwords = sha1 base_dir = /var/run/dovecot/ debug_log_path = /var/log/dovecot-debug.log default_vsz_limit = 2 G hostname = mydomain.com info_log_path = /var/log/dovecot.log lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * log_path = /var/log/dovecot-error.log login_greeting = Welcome login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c mail_debug = yes mail_gid = vmail mail_location = maildir:/var/vmail/%Ln mail_plugins = fts fts_lucene quota stats mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = maildir:/var/vmail/%Ln mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = type = private } passdb { args = /usr/local/etc/dovecot/deny-users deny = yes driver = passwd-file } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { fts = lucene fts_autoindex = yes fts_autoindex_max_recent = 1000 fts_lucene = whitespace_chars=@ quota = maildir:User quota quota_rule = *:storage=20GB quota_rule2 = Trash:storage=+200M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /var/sieve-scripts/%Ln.sieve sieve_dir = /var/vmail/%Ln/home/sieve stats_command_min_time = 1 mins stats_domain_min_time = 12 hours stats_ip_min_time = 12 hours stats_memory_limit = 16 M stats_refresh = 30 secs stats_session_min_time = 15 mins stats_track_cmds = yes stats_user_min_time = 1 hours } postmaster_address = postmaster@mydomain.com protocols = imap pop3 sieve sieve service auth-worker { user = root } service auth { unix_listener /var/spool/postfix/private/auth { mode = 0777 user = vmail } } service imap-login { inet_listener imaps { port = 993 ssl = yes } process_min_avail = 5 } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3s { port = 995 ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh user = nobody } service stats { fifo_listener stats-mail { mode = 0600 user = vmail } } ssl_cert =
On Wednesday, April 9, 2014 12:57:13 PM CEST, kadafax@gmail.com wrote:
Hi list,
We are in the process of migrating our old (dovecot 1.2.4 based) mail system to the new one:
- Centos 6 x86_64
- Dovecot 2.2.12
- Users in an openldap directory
- clucene FTS
I'm trying to trigger a full indexing for all the user and all their mailboxes (avoiding them to individually trigger it using a search within their MUA). I'm trying to use the "doveadm index" command.
First question, is there some sort of 'wildcard' option for the name of the mailbox to index, as I dont know all the mailboxes's names the users have created (beside the regular Sent Inbox Trash and Drafts ?
I'm using a totally different set of things, but at least dovecot and clucene are agreeable.. ;-)
You should have a fts plugin to doveadm that should solve this issue:
doveadm fts rescan doveadm fts optimize
That's what I use to trigger rebuilds.
Second question I've tried: # doveadm -D index -A Sent which gave me: doveadm(root): Error: User listing returned failure doveadm: Error: Failed to iterate through some users
Can the '-A' option work with ldap backend ?
http://wiki2.dovecot.org/AuthDatabase/LDAP/Userdb
# For using doveadm -A: iterate_attrs = uid=user iterate_filter = (objectClass=posixAccount)
Are you defining iterate_* for your userdb?
Regards, Ben
Le 09/04/2014 13:04, Benjamin Podszun a écrit :
On Wednesday, April 9, 2014 12:57:13 PM CEST, kadafax@gmail.com wrote:
Hi list,
We are in the process of migrating our old (dovecot 1.2.4 based) mail system to the new one:
- Centos 6 x86_64
- Dovecot 2.2.12
- Users in an openldap directory
- clucene FTS
I'm trying to trigger a full indexing for all the user and all their mailboxes (avoiding them to individually trigger it using a search within their MUA). I'm trying to use the "doveadm index" command.
First question, is there some sort of 'wildcard' option for the name of the mailbox to index, as I dont know all the mailboxes's names the users have created (beside the regular Sent Inbox Trash and Drafts ?
I'm using a totally different set of things, but at least dovecot and clucene are agreeable.. ;-)
You should have a fts plugin to doveadm that should solve this issue:
doveadm fts rescan doveadm fts optimize
That's what I use to trigger rebuilds. Yes it works but not (at least for me) on the initial lucene index creation (when the user don't even have a 'lucene-indexes' folder). For now I use in some bash script: doveadm -v index -u $user Trash doveadm -v index -u $user Drafts doveadm -v index -u $user Sent doveadm -v index -u $user Inbox
but that will not index other mailboxes the user has created. Is there a way to make doveadm to iterate through all user's mailboxes ?
Second question I've tried: # doveadm -D index -A Sent which gave me: doveadm(root): Error: User listing returned failure doveadm: Error: Failed to iterate through some users
Can the '-A' option work with ldap backend ?
http://wiki2.dovecot.org/AuthDatabase/LDAP/Userdb
# For using doveadm -A: iterate_attrs = uid=user iterate_filter = (objectClass=posixAccount)
Are you defining iterate_* for your userdb?
Yes I am and it's not working (see debug output below [1]) :( Maybe it's because I'm using a static userdb ?
###### dovecot-ldap.conf.ext AND dovecot-ldap-userdb.conf.ext (symlinked) ###### uris = ldaps://ldap.mydomain.com ldaps://ldap2.mydomain.com tls = no tls_ca_cert_file = /etc/pki/tls/certs/ca.crt tls_require_cert = never debug_level = 0 auth_bind = yes auth_bind_userdn = uid=%u,ou=people,dc=mydomain,dc=com ldap_version = 3 base = ou=people,dc=mydomain,dc=com pass_filter = (&(accountStatus=active)(uid=%n)) iterate_attrs = uid=user iterate_filter = (accountStatus=active)
###### auth-ldap.conf.ext ###### passdb { driver = ldap args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext } userdb { driver = static args = uid=11111 gid=11111 home=/var/vmail/%Ln/home allow_all_users=yes }
Subsidiary question: dovecot offers to index attachments using decode2text.sh which in turn make use of xmlunzip. No rpm for Centos/RHEL propose this tool. Is this is the one I should use: https://raw.githubusercontent.com/GNOME/gnumeric/master/tools/xmlunzip
[1]: # doveadm -D index -A INBOX doveadm(root): Debug: Loading modules from directory: /usr/local/lib/dovecot doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib10_quota_plugin.so doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib20_fts_plugin.so doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib21_fts_lucene_plugin.so doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib90_stats_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm doveadm(root): Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_lookup (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib10_doveadm_sieve_plugin.so doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_lucene_plugin.so doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so doveadm(root): Error: User listing returned failure
doveadm: Error: Failed to iterate through some users
participants (2)
-
Benjamin Podszun
-
kadafax＠gmail.com