Dovecot 2.2.16: disappearing messages, mismatched summaries, duplicated messages, excessive full re-downloads

[David Gessel]

-------- Original Message --------
Subject: Re: Dovecot 2.2.16: disappearing messages, mismatched summaries, duplicated messages, excessive full re-downloads
From: Benny Pedersen <me at junc.eu>
To: dovecot at dovecot.org
Date: Mon Jun 01 2015 16:47:48 GMT+0300 (Arabic Standard Time)

> Alessio Cecchi skrev den 2015-06-01 15:29:
>> Il 20/05/2015 10:44, David Gessel ha scritto:
>>> Is there a way to completely disable caching on dovecot?
>>
>> Yes: mail_location = maildir:~/Maildir:INDEX=MEMORY
> 
> dont know if that really solve it, imho it just move the problem from disk cache to memory cache, it does not imho disable it


It seemed to do the trick.  Behavior finally seems normal.  The best evidence:  At 10:00:00-ish I made the change in the config file and at 10:02:29 I restarted the server.

It is now after 17:14 in Oakland and the logs are quiet.

 09:00:06 host newsyslog[55292]: logfile turned over due to size>1000K
 09:00:08 imap(user1): Warning: Maildir /mail/dom1.com/user1/: Expunged message reappeared, giving a new UID (old uid=3558585, file=1434383291.M430136P55004.host,S=2380,W=2423:2,)
 09:00:08 imap(user1): Warning: Maildir /mail/dom1.com/user1/: Expunged message reappeared, giving a new UID (old uid=3558586, file=1434383253.M121801P54997.host,S=27419,W=28395:2,)
 09:01:28 indexer-worker(user1): Warning: Maildir /mail/dom1.com/user1/: Expunged message reappeared, giving a new UID (old uid=3558587, file=1434383997.M731644P55286.host,S=7334,W=7513:2,)
 09:07:49 indexer-worker(user1): Warning: Maildir /mail/dom1.com/user1/: Expunged message reappeared, giving a new UID (old uid=3561318, file=1434383997.M731644P55286.host,S=7334,W=7513:2,)
 09:08:49 imap(user2): Warning: Maildir /mail/dom2/dmag/: Expunged message reappeared, giving a new UID (old uid=1846654, file=1434381278.M451318P54372.host,S=5013,W=5116:2,)
 09:09:48 imap(user1): Warning: Maildir /mail/dom1.com/user1/: Expunged message reappeared, giving a new UID (old uid=3561319, file=1405470941.M117568P19536.host.dom1.com,S=8266,W=8382:2,)
 09:21:02 imap(user1): Warning: Maildir /mail/dom1.com/user1/: Expunged message reappeared, giving a new UID (old uid=3564050, file=1434384469.M876726P55434.host,S=7297,W=7418:2,)
 09:24:49 imap(user2): Warning: Maildir /mail/dom2/dmag/: Expunged message reappeared, giving a new UID (old uid=1847030, file=1434383414.M615990P55039.host,S=9890,W=10269:2,)
 09:24:49 imap(user2): Warning: Maildir /mail/dom2/dmag/: Expunged message reappeared, giving a new UID (old uid=1847031, file=1434384445.M934994P55428.host,S=3405,W=3487:2,)
 09:24:49 imap(user2): Warning: Maildir /mail/dom2/dmag/: Expunged message reappeared, giving a new UID (old uid=1847032, file=1434381278.M451318P54372.host,S=5013,W=5116:2,)
 09:27:18 indexer-worker(user2): Warning: Maildir /mail/dom2/dmag/: Expunged message reappeared, giving a new UID (old uid=1847033, file=1434384562.M535709P55456.host,S=3403,W=3485:2,)
 09:42:41 indexer-worker(user1): Warning: Maildir /mail/dom1.com/user1/: Expunged message reappeared, giving a new UID (old uid=3565416, file=1434385257.M437928P55657.host,S=176241,W=178722:2,)
 09:47:18 indexer-worker(user1): Warning: Maildir /mail/dom1.com/user1/: Expunged message reappeared, giving a new UID (old uid=3568150, file=1434386561.M197796P56113.host,S=905854,W=917647:2,)
 09:47:18 indexer-worker(user1): Warning: Maildir /mail/dom1.com/user1/: Expunged message reappeared, giving a new UID (old uid=3568151, file=1434385257.M437928P55657.host,S=176241,W=178722:2,)
 09:47:23 imap(user1): Warning: Maildir /mail/dom1.com/user1/: Expunged message reappeared, giving a new UID (old uid=3568152, file=1434386561.M197796P56113.host,S=905854,W=917647:2,)
 09:53:34 imap(user2): Warning: Maildir /mail/dom2/dmag/: Expunged message reappeared, giving a new UID (old uid=1847228, file=1434384562.M535709P55456.host,S=3403,W=3485:2,)
 09:59:22 master: Warning: Killed with signal 15 (by pid=56486 uid=0 code=kill)
 10:00:57 lda(user3): Warning: Maildir /mail/dom1.com/user3/: UIDVALIDITY changed (1404858902 -> 1433446125)
 10:00:57 indexer-worker(user3): Warning: fts-lucene: Settings have changed, rebuilding index for mailbox
 10:00:57 indexer-worker(user1): Warning: Maildir /mail/dom1.com/user1/: Expunged message reappeared, giving a new UID (old uid=3569520, file=1434386838.M74634P56231.host,S=905827,W=917620:2,)
 10:02:29 master: Warning: Killed with signal 15 (by pid=66696 uid=0 code=kill)
 10:05:13 indexer-worker(user1): Warning: fts-lucene: Settings have changed, rebuilding index for mailbox
 10:05:13 indexer-worker(user1): Error: stat(/mail/dom1.com/user1//.dovecot.sieve/tmp) failed: Not a directory
 10:08:09 indexer-worker(user3): Warning: fts-lucene: Settings have changed, rebuilding index for mailbox
 10:17:00 indexer-worker(user2): Warning: fts-lucene: Settings have changed, rebuilding index for mailbox
 15:24:16 imap(user1): Error: stat(/mail/dom1.com/user1//.dovecot.sieve/tmp) failed: Not a directory

The last log file was turned over when it got greater than 1000K.  It covered 1312 minutes and 3726 "expunged Message" warnings or one every 21 seconds.  There hasn't been a single one in the 7+ hours since I moved the index to MEMORY.

Whatever is confusing the index state on disk, it is not happening in RAM.