[Dovecot] IMAP labeling sometimes not taken into account

[Olivier Cailloux]

Dear Dovecot list,

I recently switched from uw-imapd to dovecot to manage my huge mbox 
files (about 1 GB). I now observe a very strange behavior, difficult to 
reproduce reliably, which might be related to some misunderstanding on 
my part on how either dovecot works or imap labeling works.

What I call "labeling" is the fact of marking some e-mail, in my e-mail 
client (currently Thunderbird), with a label, which can be chosen among 
five labels (plus the "no-label" case). Evolution also supports labeling.

As far as I understood until a few days ago, when I put a label, say, 
number five, on an e-mail, a keyword "$Label5" is added on the 
"X-Keywords:" line of the corresponding e-mail in my mbox file. Removing 
the label simply removes the keyword on the same line. That is, at 
least, what happened with uw-imapd (not that I have seen that documented 
somewhere but this is what I observed). The label is thus stored on the 
e-mail server, not on the client hard disk, which is important for me as 
I extensively use that feature to sort e-mails, and need to access them 
from several different places (and find the same labels at the same 
places, naturally, whatever client I use).

Since the change for dovecot, however, things are going strange. 
Sometimes the labels disappear for some reasons. Sometimes they stay 
there. And I can't reproduce the behavior, it just seems random. This is 
very annoying as it totally breaks my e-mail management.

Currently there is something strange I can reproduce however, and if 
someone could explain me what is happening I guess it would be a good 
step towards a solution. I see a specific e-mail as being labeled in 
thunderbird, having label number five. I also tried with evolution, 
accessing the same imap folder, evolution also sees that e-mail as 
having a label number five. I tried running these clients from my imap 
server, and I tried thunderbird on a Windows box at home, and every 
clients agree: this e-mail has a "5" label. BUT when I open the mbox 
file directly on the imap server (using "less"), and scroll to that 
given e-mail, I see NO "X-Keywords: " line. How then can the e-mail 
clients know that this e-mail is labeled?

Please note that this behavior does not happen with only one specific 
e-mail, but I take it as an example of what is strange. In fact I can 
see, especially at the end of the mbox file (thus the latest e-mails), 
that plenty of e-mails have either no "X-Keywords: " line or a line with 
"X-Keywords: " and nothing else, whereas they are seen as being labeled 
(and indeed I remember having labeled most of those through 
thunderbird). The former e-mails I checked in the m-box (for example, 
one month ago) all correctly have an X-Keywords line indicating 
"$Label5" or other appropriate labels.


It seems like dovecot is keeping some track of labeled e-mails somewhere 
which is NOT in the mbox file, probably related to its indexing 
capabilities. But I don't know how that works and where these index 
files are stored, how to purge them and rebuild the index, and so on. 
Apparently these index files have remembered some e-mails as being 
labeled (which is good, because I indeed labeled them). But I'd like to 
know how that works to be able to better troubleshoot why sometimes the 
label disappear. Note that restarting dovecot does not change the label 
status of the e-mail, so the status must be stored somewhere on disk and 
not only in memory.

Or is all these labeling stuff some unsupported, non-standard extension 
to IMAP that only uw-imapd supports? (I hope not because I find it very 
practical and I'd like to use it with dovecot as well!)

Thank you for any help. (Technical informations follow.)
Olivier

------
2-12h38-~#dovecot --version
1.0.15
------
2-12h38-~#dovecot -n
# 1.0.15: /etc/dovecot/dovecot.conf
log_timestamp: %Y-%m-%d %H:%M:%S
ssl_cert_file: /etc/ssl/certs/imapd.pem
ssl_key_file: /etc/ssl/private/imapd.pem
login_dir: /var/run/dovecot/login
login_executable: /usr/lib/dovecot/imap-login
mail_privileged_group: mail
auth default:
  passdb:
    driver: pam
  userdb:
    driver: passwd
------