[bug] Auth cache, proxy and ITERINDEX
Hi,
today we have exceprienced really major difficulties with our proxy
backend servers. Everything started after enabling auth cache:
auth_cache_size = 100M
auth_cache_verify_password_with_worker = yes
auth_cache_ttl = 1 hour
auth_cache_negative_ttl = 1 hour
Soon after we start receiving lots of calls from our customers about
'missing all e-mail messages'. This was in errors logs on master server:
Apr 16 14:37:10 server00 dovecot: imap(
Error: autoexpunge: Couldn't create dovecot.autoexpunge.lock lock:
file_create_locked(/var/mail/vhosts/
directory
Looks like that all users, who are placed on proxy backends, were
logged correctly BUT not proxied to the right server so they saw empty
mailboxes (Dovecot also created some directories on master server).
What is worse, their email software deleted all local emails so they
are now downloading everything from scratch (100Gs of mails so it will
take some time).
After auth cache was disabled, things started to go to normal EXCEPT
users were reporting they don't see all folders - which was true, only
few on them were visible (INBOX, trash, sent etc.). We tried
everything to make them visible again, for example deleting indexes,
but nothing helped. Finally, removing option ITERINDEX from
mail_location helped.
Can anyone explain what happened? Thank you
azurIt
Hi,
please post your doveconf -n
Sami
On 16 Apr 2018, at 20.17, azurit@pobox.sk wrote:
Hi,
today we have exceprienced really major difficulties with our proxy backend servers. Everything started after enabling auth cache: auth_cache_size = 100M auth_cache_verify_password_with_worker = yes auth_cache_ttl = 1 hour auth_cache_negative_ttl = 1 hour
Soon after we start receiving lots of calls from our customers about 'missing all e-mail messages'. This was in errors logs on master server:
Apr 16 14:37:10 server00 dovecot: imap(
): Error: autoexpunge: Couldn't create dovecot.autoexpunge.lock lock: file_create_locked(/var/mail/vhosts/ / /home/dovecot.autoexpunge.lock) failed: safe_mkstemp(/var/mail/vhosts/ / /home/dovecot.autoexpunge.lock) failed: No such file or directory Looks like that all users, who are placed on proxy backends, were logged correctly BUT not proxied to the right server so they saw empty mailboxes (Dovecot also created some directories on master server). What is worse, their email software deleted all local emails so they are now downloading everything from scratch (100Gs of mails so it will take some time).
After auth cache was disabled, things started to go to normal EXCEPT users were reporting they don't see all folders - which was true, only few on them were visible (INBOX, trash, sent etc.). We tried everything to make them visible again, for example deleting indexes, but nothing helped. Finally, removing option ITERINDEX from mail_location helped.
Can anyone explain what happened? Thank you
azurIt
Here it is:
# 2.2.34 (874deae): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.22 (22940fb7) # OS: Linux 4.4.127 x86_64 Debian 9.4 ext3 # Hostname: server00 auth_master_user_separator = * auth_mechanisms = plain login default_client_limit = 30000 default_process_limit = 6000 default_vsz_limit = 512 M disable_plaintext_auth = no first_valid_uid = 100 lda_original_recipient_header = X-Original-To lmtp_proxy = yes login_greeting = mail_fsync = never mail_gid = mail mail_location = maildir:/var/mail/vhosts/%d/%n:INDEX=/dovecot_indexes/%d/%n mail_plugins = acl mailbox_alias fts fts_solr stats mail_log notify mail_uid = postfix mailbox_list_index = yes mailbox_list_index_very_dirty_syncs = yes namespace { list = children location = maildir:/var/mail/vhosts/%%d/%%n:INDEX=/dovecot_indexes/%%d/%%n prefix = INBOX/Shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes list = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { autoexpunge = 30 days special_use = \Junk } mailbox Karantena { autoexpunge = 30 days } mailbox Kos { autoexpunge = 180 days special_use = \Trash } mailbox Odoslane { special_use = \Sent } mailbox Rozpisane { special_use = \Drafts } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { autoexpunge = 30 days special_use = \Junk } mailbox Trash { autoexpunge = 180 days special_use = \Trash } prefix = INBOX/ separator = / type = private } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile acl_shared_dict = file:/var/lib/dovecot/shared/shared-mailboxes fts = solr fts_autoindex = yes fts_autoindex_max_recent_msgs = 1000 fts_solr = url=http://127.0.0.1:8080/solr/ mail_log_events = delete expunge mail_log_fields = from subject sieve = ~/.dovecot.sieve sieve_after = /etc/dovecot/sieve-after sieve_before = /etc/dovecot/sieve-before sieve_dir = ~/sieve sieve_extensions = +vacation-seconds sieve_vacation_min_period = 0 stats_command_min_time = 1 mins stats_domain_min_time = 12 hours stats_ip_min_time = 12 hours stats_memory_limit = 16 M stats_refresh = 5s stats_session_min_time = 15 mins stats_track_cmds = yes stats_user_min_time = 1 hours } protocols = " imap lmtp pop3" service auth-worker { user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service dict { unix_listener dict { user = postfix } } service imap-login { process_min_avail = 20 } service imap { executable = imap postlogin process_limit = 3072 vsz_limit = 400 M } service indexer-worker { process_limit = 3 } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } process_min_avail = 15 unix_listener /var/spool/postfix-remote/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service pop3 { executable = pop3 postlogin process_limit = 2048 } service postlogin { executable = script-login /etc/dovecot/scripts/post-login-password.sh user = $default_internal_user } service stats { fifo_listener stats-mail { mode = 0666 } } ssl_cert = imap_acl imap_stats } protocol pop3 { mail_max_userip_connections = 20 mail_plugins = acl mailbox_alias fts fts_solr stats mail_log notify pop3_fast_size_lookups = yes }
Citát Sami Ketola sami.ketola@dovecot.fi:
Hi,
please post your doveconf -n
Sami
On 16 Apr 2018, at 20.17, azurit@pobox.sk wrote:
Hi,
today we have exceprienced really major difficulties with our proxy
backend servers. Everything started after enabling auth cache: auth_cache_size = 100M auth_cache_verify_password_with_worker = yes auth_cache_ttl = 1 hour auth_cache_negative_ttl = 1 hourSoon after we start receiving lots of calls from our customers
about 'missing all e-mail messages'. This was in errors logs on
master server:Apr 16 14:37:10 server00 dovecot: imap(
):
Error: autoexpunge: Couldn't create dovecot.autoexpunge.lock lock:
file_create_locked(/var/mail/vhosts// /home/dovecot.autoexpunge.lock) failed: safe_mkstemp(/var/mail/vhosts/ / /home/dovecot.autoexpunge.lock) failed: No such file or
directoryLooks like that all users, who are placed on proxy backends, were
logged correctly BUT not proxied to the right server so they saw
empty mailboxes (Dovecot also created some directories on master
server). What is worse, their email software deleted all local
emails so they are now downloading everything from scratch (100Gs
of mails so it will take some time).After auth cache was disabled, things started to go to normal
EXCEPT users were reporting they don't see all folders - which was
true, only few on them were visible (INBOX, trash, sent etc.). We
tried everything to make them visible again, for example deleting
indexes, but nothing helped. Finally, removing option ITERINDEX
from mail_location helped.Can anyone explain what happened? Thank you
azurIt
participants (2)
-
azurit@pobox.sk
-
Sami Ketola