November 2013 - dovecot

[Dovecot] maildir_very_dirty_syncs and GFS
by Andrea gabellini - SC 11 Nov '13

11 Nov '13

Hello, I have two dovecot 2.2.7 and postfix with LDA backend that shares a GFS partition as email storage. In front of them there is a proxy with Director. In both backend servers only Dovecot access the cur/ directory. Can I use maildir_very_dirty_syncs=yes? Thanks, Andrea -- ---------------------------------------------------------------- That is what learning is. You suddenly understand something you've understood all your life, but in a new way. ---------------------------------------------------------------- Ing. *Andrea Gabellini* Email: andrea.gabellini(a)telecomitalia.sm <mailto:Andrea%20Gabellini%20%3Candrea.gabellini@telecomitalia.sm%3E> Skype: andreagabellini Tel: (+378) 0549 886111 Fax: (+378) 0549 886188 Telecom Italia San Marino S.p.A. Strada degli Angariari, 3 47891 Rovereta Republic of San Marino http://www.telecomitalia.sm

1 0

Re: [Dovecot] Dovecot MTA
by Peter Mogensen 11 Nov '13

11 Nov '13

Timo Sirainen wrote: > And Dovecot roadmap is slowly shrinking .. there aren’t all that many > big features left anymore. Soon it’s mainly going to be improvements > to reliability and performance. So I need to find some new things to > do in any case. :) True ... If I try to make a wish list for features many of them requires fixing the IMAP protocol it self. (Like not having the folder display name being the unique identifier) Which reminds me that the IMAP5 process (if there ever was one) seems to be slowed to a halt. Now, there's a task for a developer looking for something to do ;-) /Peter

1 0

[Dovecot] Getting debug output from antispam plugin with dspam
by Igor Zinovik 11 Nov '13

11 Nov '13

Hello. I'm trying to get some output from antispam plugin when it invokes dspam binary for retraining. Currently I see no messages related to dspam execution. Dovecot IMAP server works as 'vmail' user: # ps aux|grep 'dovecot/imap' vmail 3062 0.7 0.0 29824 3008 ? S 16:22 0:00 dovecot/imap User 'vmail' is listed as trusted in /etc/dspam/dspam.conf # fgrep vmail /etc/dspam/dspam.conf Trust vmail Dovecot IMAP plugins and antispam configuration: # doveconf -n | egrep -e '(^#|antispam|protocol imap)' # 2.2.7: /etc/dovecot/dovecot.conf # OS: Linux 3.7.10-1.16-default x86_64 openSUSE 12.3 (x86_64) antispam_allow_append_to_spam = YES antispam_backend = dspam antispam_dspam_args = --client;--user;%Lu;--deliver=spam,innocent;--source=error;--signature=%%s antispam_dspam_binary = /usr/sbin/dspam antispam_dspam_notspam = --class=innocent antispam_dspam_result_header = X-DSPAM-Result antispam_dspam_spam = --class=spam antispam_signature = X-DSPAM-Signature antispam_signature_missing = move antispam_spam = Spam antispam_trash = trash;Trash;Deleted;deleted;Deleted Items protocol imap { mail_plugins = expire mail_log notify quota trash imap_quota antispam Antispam plugin is successfully loaded: # grep antispam_plugin /var/log/dovecot/dovecot.log Nov 11 16:22:32 imap: Debug: Module loaded: /usr/lib64/dovecot/modules/lib90_antispam_plugin.so As i understand plugin would not work with message that do not contains dspam signature, so I sent GTUBE message to myself and when I move it to Spam folder I see only Dovecot messages: Nov 11 16:28:06 imap(zinovik(a)example.org): Info: copy from INBOX: box=Spam, uid=3422, msgid=<20131111144014.731f035b871deb0e082bc22e(a)example.org>, size=1130 Nov 11 16:28:06 imap(zinovik(a)example.org): Info: delete: box=INBOX, uid=26792, msgid=<20131111144014.731f035b871deb0e082bc22e(a)example.org>, size=1130 Nov 11 16:28:06 imap(zinovik(a)example.org): Info: expunge: box=INBOX, uid=26792, msgid=<20131111144014.731f035b871deb0e082bc22e(a)example.org>, size=1130 Dspam logs nothing, though debugging is turned on # ps aux | egrep -e '^dspam' dspam 2939 0.0 0.0 116408 2420 ? S 15:38 0:00 /usr/sbin/dspam --daemon --debug # egrep '^Debug ' /etc/dspam/dspam.conf Debug * Even if I set dspam binary to file that do not exist i do not get any debugging messages (according to code function i_debug("execution %s failed...") should be triggered). Maybe there is some kind of verbose output knob for plugins in Dovecot? I turned on everything I found (also using mail_log for imap protocol): # doveconf -n|egrep -e '(verbose|debug)' auth_debug = yes auth_verbose = yes mail_debug = yes verbose_ssl = yes Is there anybody out there who use Dovecot 2.2.7 with antispam plugin? Antispam plugin revision 5ebc6aae4d7c<http://hg.dovecot.org/dovecot-antispam-plugin/rev/5ebc6aae4d7c>

1 0

[Dovecot] Dovecot replication not redirecting if server is down
by Patrick Westenberg 11 Nov '13

11 Nov '13

Hi everyone, I'm running a test environment with a proxy in front of working replication between two backends but redirecting in case of a backend failure is not working. Nov 09 21:03:59 imap-login: Error: proxy(mail(a)example.net): connect(10.5.29.211, 143) failed: Connection refused (after 0 secs, local=10.5.29.201:38333) I appreciate any advice. Regards Patrick Proxy: # 2.2.7: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.2 auth_debug = yes auth_mechanisms = plain login auth_verbose = yes default_process_limit = 150 director_mail_servers = 10.5.29.211 10.5.29.212 director_servers = 10.5.29.201 director_user_expire = 5 mins disable_plaintext_auth = no lmtp_proxy = yes log_path = /var/log/dovecot.log mail_plugins = notify replication managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = proxy=y nopassword=y driver = static } protocols = imap pop3 lmtp sieve service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service auth { unix_listener auth-userdb { user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { address = 10.5.29.201 port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director } service lmtp { inet_listener lmtp { address = 10.5.29.201 port = 24 } } service managesieve-login { executable = managesieve-login director inet_listener sieve { port = 4190 } } service pop3-login { executable = pop3-login director } service replicator { unix_listener replicator-doveadm { mode = 0600 } } ssl = no protocol lmtp { auth_socket_path = director-userdb } Backend 1: # 2.2.7: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.2 auth_debug = yes auth_mechanisms = plain login auth_verbose = yes disable_plaintext_auth = no dotlock_use_excl = no doveadm_password = secret doveadm_port = 12345 dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u hostname = mb01.example.net listen = 10.5.29.211 log_path = /var/log/dovecot.log mail_debug = yes mail_fsync = always mail_gid = vmail mail_home = /var/mail/%d/%n mail_location = maildir:~/Maildir mail_plugins = quota notify replication mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Spamverdacht { auto = subscribe } mailbox Trash { auto = subscribe special_use = \Trash } prefix = INBOX. separator = . type = private } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { mail_replica = tcp:10.5.29.212 quota = dict:User quota::file:%h/Maildir/dovecot-quota quota_rule2 = INBOX.Trash:ignore quota_warning = storage=90%% quota-warning 90 %u quota_warning2 = storage=75%% quota-warning 75 %u sieve = ~/.dovecot.sieve sieve_after = /usr/local/etc/dovecot/sieve/sieve_after.sieve sieve_default = /usr/local/etc/dovecot/sieve/default.sieve sieve_dir = ~/sieve } postmaster_address = postmaster(a)example.net protocols = imap pop3 lmtp sieve service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service auth { unix_listener auth-userdb { mode = 0666 user = vmail } } service doveadm { inet_listener { port = 12345 } } service lmtp { inet_listener lmtp { address = 10.5.29.211 port = 24 } } service managesieve-login { inet_listener sieve { port = 4190 } } service quota-warning { executable = script /usr/local/etc/dovecot/quota_warning.sh unix_listener quota-warning { user = vmail } user = root } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 } } ssl = no submission_host = mf01.example.net userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = quota notify replication sieve } protocol imap { mail_max_userip_connections = 30 mail_plugins = quota notify replication imap_quota } Backend 2: # 2.2.7: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.2 auth_debug = yes auth_mechanisms = plain login auth_verbose = yes disable_plaintext_auth = no dotlock_use_excl = no doveadm_password = secret doveadm_port = 12345 dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u hostname = mb02.example.net listen = 10.5.29.212 log_path = /var/log/dovecot.log mail_debug = yes mail_fsync = always mail_gid = vmail mail_home = /var/mail/%d/%n mail_location = maildir:~/Maildir mail_plugins = quota notify replication mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Spamverdacht { auto = subscribe } mailbox Trash { auto = subscribe special_use = \Trash } prefix = INBOX. separator = . type = private } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { mail_replica = tcp:10.5.29.211 quota = dict:User quota::file:%h/Maildir/dovecot-quota quota_rule2 = INBOX.Trash:ignore quota_warning = storage=90%% quota-warning 90 %u quota_warning2 = storage=75%% quota-warning 75 %u sieve = ~/.dovecot.sieve sieve_after = /usr/local/etc/dovecot/sieve/sieve_after.sieve sieve_default = /usr/local/etc/dovecot/sieve/default.sieve sieve_dir = ~/sieve } postmaster_address = postmaster(a)example.net protocols = imap pop3 lmtp sieve service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service auth { unix_listener auth-userdb { mode = 0666 user = vmail } } service doveadm { inet_listener { port = 12345 } } service lmtp { inet_listener lmtp { address = 10.5.29.212 port = 24 } } service managesieve-login { inet_listener sieve { port = 4190 } } service quota-warning { executable = script /usr/local/etc/dovecot/quota_warning.sh unix_listener quota-warning { user = vmail } user = root } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 user = vmail } } ssl = no submission_host = mf01.example.net userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = quota notify replication sieve } protocol imap { mail_max_userip_connections = 30 mail_plugins = quota notify replication imap_quota }

6 10

[Dovecot] sieve + notify + $subject$ + empty subject = segv
by Christian Ferbar 11 Nov '13

11 Nov '13

Hello, we can reproduce a bug where lmtp crashes: LMTP telnet session: Trying 127.0.0.1... Connected to localhost.localdomain (127.0.0.1). Escape character is '^]'. 220 lisa.m-otion.at Dovecot ready. lhlo xyz 250-xyz.m-otion.at 250-8BITMIME 250-ENHANCEDSTATUSCODES 250 PIPELINING mail from:<xyz(a)m-otion.at> 250 2.1.0 OK rcpt to:<my_mailbox> 250 2.1.5 OK data 354 OK From:<xyz(a)m-otion.at> test . Connection closed by foreign host. /var/log/messages tells me: Nov 11 00:07:00 mail kernel: lmtp[15868]: segfault at 0000000000000000 rip 00002b1960d01d70 rsp 00007fff3fa95908 error 4 pigeonhole 0.4.2, dovecot 2.2.6 and this is what I found out: If you use a sieve script containing the following line notify :method "mailto" :options "xyz" :message "$from$*#+$subject$"; and send a mail without a Subject: header line lmtp crashes. The bug is related to the sieve module in /src/lib-sieve/plugins/notify/ext-notify-common.c:266. The line contains only a check if mail_get_headers_utf8 returns an error. So the workaround for this line would be to add a check if header[0] is not NULL. if ( (mail_get_headers_utf8(msgdata->mail, "subject", &header) >= 0) && header[0] ) str_append(out_msg, header[0]); IMHO this patch should be applied to the "$from$" replacement as well. The comments in the dovecot's header file says for mail_get_headers: Returns -1 if error, 0 otherwise (with or without headers found). hope it helps, Regards Christian Ferbar

2 1

[Dovecot] Replication on v2.2.6 - I'm stuck (again)
by IT geek 31 10 Nov '13

10 Nov '13

Hi, I'm trying to get Dovecot replication working between two servers. I didn't have much luck on v2.1.3, so after receiving advice from the list I have upgraded to v2.2.6. I now get the error: Oct 30 11:50:16 server1 dovecot: doveadm(mark): Error: user mark: Auth PASS lookup failed Oct 30 11:50:16 server2 dovecot: doveadm(mark): Error: sync: /var/run/dovecot/auth-userdb: passdb lookup failed (to see if user is proxied, because doveadm_port is set) Here is my config: # 2.2.6: /usr/pkg/etc/dovecot/dovecot.conf # OS: NetBSD 5.2 cobalt auth_mechanisms = plain login doveadm_password = mylittlesecret doveadm_port = 12345 dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u listen = 192.168.1.1 login_greeting = Go on then, let's have it... mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_plugins = " notify replication" passdb { driver = passwd } plugin { mail_replica = remote:vmail@server2.mydomain.com replication_full_sync_interval = 1 hours } protocols = imap service aggregator { fifo_listener replication-notify-fifo { mode = 0600 user = vmail } unix_listener replication-notify { mode = 0600 user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = root } service doveadm { inet_listener { port = 12345 } } service imap-login { inet_listener imap { port = 0 } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 } } ssl_cert = </etc/openssl/certs/mail.mydomain.com.crt ssl_key = </etc/openssl/private/mail.mydomain.com.key userdb { driver = passwd } protocol lda { postmaster_address = postmaster(a)mydomain.com Can anyone give me some guidance on where I've gone wrong with my config? Any help would be appreciated, -Mark

2 7

[Dovecot] Question about folder sharing
by Achim Gottinger 09 Nov '13

09 Nov '13

Hi, I run dovecot (2.1.7) on debian wheezy in conjuniction with postfix, samba4 (as ldap backend) and sogo. I configured folder sharing but have an few issues. With my current config users can share the inbox and other folders. If the acl allows creatings subfolders this does work for all folders beside inbox. What i want to archiev is the following: If an user shares his inbox, others should be able to create subfolders and those should inherit the inboxe's acl. All subfolders of inbox should appear as folders at root level and not as subfolders of the inbox. I thought this can be done by setting the prefix of namespace inbox to INBOX/. I did this and changed the IMAP Server Folder setting in thunderbird to INBOX (like it was earlier when i used courier). Now subfolders created at rootlevel or as subfolders of the inbox appear on rootlevel in thunderbird but they do not inherit the acl's from inbox. Is there an way to achive this? doveconf -n # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.2 ext4 auth_debug = yes auth_gssapi_hostname = $ALL auth_krb5_keytab = /etc/dovecot/dovecot.keytab auth_master_user_separator = % auth_mechanisms = plain login gssapi auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@% auth_username_format = %n auth_verbose = yes debug_log_path = syslog disable_plaintext_auth = no first_valid_gid = 998 first_valid_uid = 998 info_log_path = syslog mail_debug = yes mail_gid = 998 mail_home = /home/vmail/%u mail_location = maildir:/home/vmail/%u/mail mail_plugins = acl mail_shared_explicit_inbox = no mail_uid = 998 maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { list = children location = maildir:/home/vmail/%%u/mail:INDEX=/home/vmail/%u/mail/shared/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = maildir:/home/vmail/%u/mail prefix = separator = / type = private } passdb { args = /etc/dovecot/dovecot-ldap-passdb.conf.ext driver = ldap } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes } plugin { acl = vfile acl_anyone = allow acl_shared_dict = file:/home/vmail/.shared-mailboxes sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = " imap lmtp sieve" service auth-worker { user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = vmail mode = 0777 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service imap-postlogin { executable = script-login /etc/dovecot/acl-groups.sh user = $default_internal_user } service imap { executable = imap imap-postlogin } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0666 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } ssl_cert = </etc/easy-rsa/keys/dovecot.crt ssl_key = </etc/easy-rsa/keys/dovecot.key userdb { args = /etc/dovecot/dovecot-ldap-userdb.conf.ext driver = ldap } userdb { args = /etc/dovecot/dovecot-ldap-userdb-groups.conf.ext driver = ldap } protocol imap { imap_client_workarounds = delay-newmail mail_plugins = acl imap_acl } protocol lmtp { mail_plugins = acl sieve }

1 4

[Dovecot] Problem with master user
by Jakub Krzyżewski 08 Nov '13

08 Nov '13

Hello. I have problem as below: Nov 8 10:41:52 store1 dovecot: auth: Debug: auth(master@example.com,::1,master,</qEuMafqyAAAAAAAAAAAAAAAAAAAAAAB>): Master user lookup for login: jkrzyz(a)example.com Nov 8 10:41:52 store1 dovecot: auth: Debug: passwd-file(master@example.com,::1,master,</qEuMafqyAAAAAAAAAAAAAAAAAAAAAAB>): lookup: user=master(a)example.com file=/etc/dovecot/master-use rs Nov 8 10:41:52 store1 dovecot: auth: Debug: password(master@example.com,::1,master,</qEuMafqyAAAAAAAAAAAAAAAAAAAAAAB>): Generating DIGEST-MD5 from user 'master', password 'test' Nov 8 10:41:52 store1 dovecot: auth: passdb(master@example.com,::1,master,</qEuMafqyAAAAAAAAAAAAAAAAAAAAAAB>): Master user logging in as jkrzyz(a)example.com Nov 8 10:41:52 store1 dovecot: auth: Debug: ldap(jkrzyz@example.com,::1,</qEuMafqyAAAAAAAAAAAAAAAAAAAAAAB>): pass search: base=dc=example,dc=com scope=subtree filter=(&(locMailActive=TRUE)(| (uid=jkrzyz@example.com)(uid=jkrzyz)(mailRoutingAddress=jkrzyz@example.com))) fields=mailRoutingAddress,userPassword Nov 8 10:41:52 store1 dovecot: auth: Debug: ldap(jkrzyz@example.com,::1,</qEuMafqyAAAAAAAAAAAAAAAAAAAAAAB>): result: mailRoutingAddress=jkrzyz(a)example.com userPassword=test2 Nov 8 10:41:52 store1 dovecot: auth: Debug: password(jkrzyz@example.com,::1,</qEuMafqyAAAAAAAAAAAAAAAAAAAAAAB>): Generating DIGEST-MD5 from user 'master', password 'test2' Nov 8 10:41:52 store1 dovecot: auth: Debug: password(jkrzyz@example.com,::1,</qEuMafqyAAAAAAAAAAAAAAAAAAAAAAB>): Credentials: d64221d543d7c9a809c7d6e424d87be8 Nov 8 10:41:52 store1 dovecot: auth: digest-md5(jkrzyz@example.com,::1,</qEuMafqyAAAAAAAAAAAAAAAAAAAAAAB>): password mismatch As you can see, password is check against user passdb and not passwd-file, where master's password is stored. Test is password of master user, test2 is password of jkrzyz Setting pass=yes or no makes no difference. What is wrong with my config? dovecot --version 2.1.7 dovecot.conf snippet: passdb { args = scheme=PLAIN /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } /etc/dovecot/master-users: master:{PLAIN}test master@example.com:{PLAIN}test

2 1

[Dovecot] how to calculate mail storage/traffic used
by Götz Reinicke - IT Koordinator 08 Nov '13

08 Nov '13

Hi, may be som mailserver admins on the list can give me som hint. Thats not dovecot related, but I know the knowledg here is enormous. We have to setup a server which gets a copy of all messages send and received by our mailserver as a 1:1 copy. Mails send to multiple recipients should be calculated and saved per user. (great if you usually have lost of mails send to groups of people.) So no dedublication should be used here. (e.g. save the message and refer the different recipients to it.) How can I calculate the current traffic in the best way to extrapolate the amount of space to be planed for the new server? We think to have a couples of days to be saved. Thanks for any hint or comment. Regards Götz -- Götz Reinicke IT-Koordinator Tel. +49 7141 969 82 420 Fax +49 7141 969 55 420 E-Mail goetz.reinicke(a)filmakademie.de Filmakademie Baden-Württemberg GmbH Akademiehof 10 71638 Ludwigsburg www.filmakademie.de Eintragung Amtsgericht Stuttgart HRB 205016 Vorsitzender des Aufsichtsrats: Jürgen Walter MdL Staatssekretär im Ministerium für Wissenschaft, Forschung und Kunst Baden-Württemberg Geschäftsführer: Prof. Thomas Schadt

2 3

[Dovecot] Solr clusters
by Timo Sirainen 07 Nov '13

07 Nov '13

Hi all, Has anyone implemented larger Dovecot+Solr clusters and would be willing to give some details about how it works for you? My understanding about it so far is: - SolrCloud isn’t usable with Dovecot. Replication isn’t useful, because nobody wants to pay for double the disk space for indexes that could be regenerated anyway. The autosharding isn’t very useful also, because: I think the shard keys could be created in two possible ways: a) Mails would be entirely randomly distributed across the cluster. This would make updates efficient, because the writes would be fully distributed across all servers. But I think it would also make reads somewhat inefficient, since all the servers would have to be searched and the results combined. Also if a server is lost, there’s no easy way to reindex back the missing data, because it would contain a piece of pretty much all the users’ data. b) Shard keys could be created so that the same user would typically go only to 1-2 servers. It would be possible (at least in theory) to find a broken server’s list of users and reindex only their data, but I’m not sure if this method is any easier than the non-SolrCloud setup. - Without SolrCloud you’d then need to shard the data manually. This would be easy enough to do by just assigning different users to different shards. But at some point the index is going to become too large and you need to add more shards and move some existing users to them. To keep the search performance good during the move, I guess this could be done with a script that does: 1) reindex user to new shard, 2) update userdb to point to new shard, 3) delete user from old shard, 4) doveadm fts rescan the user to remove any mails already deleted during the reindexing. - It seems that Solr index shouldn’t grow above 200 GB or the performance will be getting too bad? I’ve seen this in a few web pages. So each server should likely be running multiple separate Solr instances (shards). - http://www.hathitrust.org/blogs/large-scale-search/scaling-large-scale-sear… recommends NFS (or I guess any kind of a shared filesystem), which does seem to make sense. Since Dovecot wants to get instantly updated search results after indexing, I think it’s probably better not to separate the indexing and searching servers. - Would be interesting to know what kind of hardware your Solr servers currently have, how well they’re performing and what are the bottlenecks? From the above URL it appears that disk I/O is first, but if there’s enough of that available then CPU usage is second. I’m not quite sure where most of the memory goes - caching? - I’m guessing users are doing relatively few searches compared to how many new emails are being indexed/deleted all the time?

1 1