How to Analyze Missing UIDs (and Files) in Maildirs?
Hello everyone,
I would like pointers on how to analyze the following situation, please:
I'm running one test and one production dovecot IMAPS server for one of our platforms. The clients are essentially appliances we distribute, auth by client cert, virtual users only, mailboxes in maildir format:
auth_ssl_require_client_cert = yes auth_ssl_username_from_cert = yes auth_mechanisms = plain
ssl = required ssl_cert =
mail_location = maildir:~ userdb { driver = static args = uid=... gid=... home=/home/.../%Ld_realm/%Ln } passdb { driver = static args = password=... }
The client certs have CNs unique to the appliance and no client besides that appliance is supposed to access the mailbox. Appliances take note of the UIDs they've seen (and not yet deleted) and expect to be able to re-access e-mails by the UIDs they know. (Needless to say, I keep my hands away from the cache files so as not to cause the UIDs to be reassigned.)
The dovecot version used is updated along with the CentOS 6 source RPMs sources to improve closure of inactive SSL connections and one from me
- which contain a 2.0.9 plus whatever patches CentOS backported - and receives two additional patches from me, one from current dovecot
that turns the IMAP CREATE command into a NOP; the appliance code doesn't expect mailboxes to have folders besides INBOX.
QA is looking into an upcoming new feature that requires appliances to reread old e-mails, and got errors that UIDs the test appliances expected to still be on the (test) server weren't. I verified, the UIDs are indeed absent from the dovecot-uidlist files. The mails in question arrived *sometime* over the course of the last 12 months and must've vanished *sometime* since then ... ugh. No trace of what may have happened to them in any logs, of course.
On the test server, I re-enabled logging of DELETE, UNDELETE and EXPUNGE commands, in case the appliances delete mails themselves and forget to update their own cache. Are there other IMAP commands that could remove mails, and thus should be added to the list?
Would it be possible to have dovecot configured so that there's a log entry whenever a client tries to retrieve a UID that dovecot doesn't have in the maildir anymore?
While looking into the above issue, I noticed that a large number of mailboxes (test as well as production) have UIDs that *are* listed in dovecot-uidlist, yet, no corresponding file is present anymore. Example:
# grep -c : *_realm/$EXAMPLE_USER/dovecot-uidlist 58 # grep : *_realm/$EXAMPLE_USER/dovecot-uidlist | sed -e '2,57d' -e 's/ .*//' 1 152 # for PAIR in
grep : *_realm/$EXAMPLE_USER/dovecot-uidlist | sed -e 's/ .*:/:/'
; doIMAP_UID=
echo $PAIR | sed -e 's/:.*//'
; BASE=echo $PAIR | sed -e 's/.*://'
CNT=find *_realm/$EXAMPLE_USER -type f -name "$BASE"'*' -ls | wc -l
if [ $CNT -ne 1 ]; then echo "Found $CNT files for UID $IMAP_UID"; fi; done Found 0 files for UID 135 Found 0 files for UID 136 Found 0 files for UID 139 Found 0 files for UID 142 Found 0 files for UID 143 Found 0 files for UID 144 Found 0 files for UID 145 Found 0 files for UID 150 Found 0 files for UID 151
Is that normal behaviour? If not, how would I try to find out what happens there?
Kind regards,
Jochen Bern Systemingenieur
Fon: +49 6151 9067-231 Fax: +49 6151 9067-290 E-Mail: jochen.bern@binect.de
www.binect.de www.facebook.de/binect
Binect GmbH
Robert-Koch-Straße 9, 64331 Weiterstadt, DE
Geschäftsführung: Christian Ladner, Dr. Frank Wermeyer, Nils Manegold Unternehmenssitz: Weiterstadt Register: Amtsgericht Darmstadt, HRB 94685 Umsatzsteuer-ID: DE 221 302 264
MAX 21-Unternehmensgruppe
participants (1)
-
Jochen Bern