[Dovecot] IMAP labeling sometimes not taken into account
Dear Dovecot list,
I recently switched from uw-imapd to dovecot to manage my huge mbox files (about 1 GB). I now observe a very strange behavior, difficult to reproduce reliably, which might be related to some misunderstanding on my part on how either dovecot works or imap labeling works.
What I call "labeling" is the fact of marking some e-mail, in my e-mail client (currently Thunderbird), with a label, which can be chosen among five labels (plus the "no-label" case). Evolution also supports labeling.
As far as I understood until a few days ago, when I put a label, say, number five, on an e-mail, a keyword "$Label5" is added on the "X-Keywords:" line of the corresponding e-mail in my mbox file. Removing the label simply removes the keyword on the same line. That is, at least, what happened with uw-imapd (not that I have seen that documented somewhere but this is what I observed). The label is thus stored on the e-mail server, not on the client hard disk, which is important for me as I extensively use that feature to sort e-mails, and need to access them from several different places (and find the same labels at the same places, naturally, whatever client I use).
Since the change for dovecot, however, things are going strange. Sometimes the labels disappear for some reasons. Sometimes they stay there. And I can't reproduce the behavior, it just seems random. This is very annoying as it totally breaks my e-mail management.
Currently there is something strange I can reproduce however, and if someone could explain me what is happening I guess it would be a good step towards a solution. I see a specific e-mail as being labeled in thunderbird, having label number five. I also tried with evolution, accessing the same imap folder, evolution also sees that e-mail as having a label number five. I tried running these clients from my imap server, and I tried thunderbird on a Windows box at home, and every clients agree: this e-mail has a "5" label. BUT when I open the mbox file directly on the imap server (using "less"), and scroll to that given e-mail, I see NO "X-Keywords: " line. How then can the e-mail clients know that this e-mail is labeled?
Please note that this behavior does not happen with only one specific e-mail, but I take it as an example of what is strange. In fact I can see, especially at the end of the mbox file (thus the latest e-mails), that plenty of e-mails have either no "X-Keywords: " line or a line with "X-Keywords: " and nothing else, whereas they are seen as being labeled (and indeed I remember having labeled most of those through thunderbird). The former e-mails I checked in the m-box (for example, one month ago) all correctly have an X-Keywords line indicating "$Label5" or other appropriate labels.
It seems like dovecot is keeping some track of labeled e-mails somewhere which is NOT in the mbox file, probably related to its indexing capabilities. But I don't know how that works and where these index files are stored, how to purge them and rebuild the index, and so on. Apparently these index files have remembered some e-mails as being labeled (which is good, because I indeed labeled them). But I'd like to know how that works to be able to better troubleshoot why sometimes the label disappear. Note that restarting dovecot does not change the label status of the e-mail, so the status must be stored somewhere on disk and not only in memory.
Or is all these labeling stuff some unsupported, non-standard extension to IMAP that only uw-imapd supports? (I hope not because I find it very practical and I'd like to use it with dovecot as well!)
Thank you for any help. (Technical informations follow.) Olivier
2-12h38-~#dovecot --version 1.0.15
2-12h38-~#dovecot -n # 1.0.15: /etc/dovecot/dovecot.conf log_timestamp: %Y-%m-%d %H:%M:%S ssl_cert_file: /etc/ssl/certs/imapd.pem ssl_key_file: /etc/ssl/private/imapd.pem login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login mail_privileged_group: mail auth default: passdb: driver: pam userdb: driver: passwd
On Mon, 2009-06-01 at 12:43 +0200, Olivier Cailloux wrote:
Since the change for dovecot, however, things are going strange. Sometimes the labels disappear for some reasons. Sometimes they stay there. And I can't reproduce the behavior, it just seems random. This is very annoying as it totally breaks my e-mail management.
Yes, there have been some bugs related to handling them.
clients agree: this e-mail has a "5" label. BUT when I open the mbox file directly on the imap server (using "less"), and scroll to that given e-mail, I see NO "X-Keywords: " line. How then can the e-mail clients know that this e-mail is labeled?
The keywords are also stored in Dovecot's index files. Although they should still be written to mbox files too.
2-12h38-~#dovecot --version 1.0.15
I'd first try a recent v1.1 release. It might have already fixed this bug. Also you could try if setting mbox_lazy_writes=no helps.
Timo Sirainen a écrit :
On Mon, 2009-06-01 at 12:43 +0200, Olivier Cailloux wrote:
Since the change for dovecot, however, things are going strange. Sometimes the labels disappear for some reasons. Sometimes they stay there. And I can't reproduce the behavior, it just seems random. This is very annoying as it totally breaks my e-mail management.
Yes, there have been some bugs related to handling them.
clients agree: this e-mail has a "5" label. BUT when I open the mbox file directly on the imap server (using "less"), and scroll to that given e-mail, I see NO "X-Keywords: " line. How then can the e-mail clients know that this e-mail is labeled?
The keywords are also stored in Dovecot's index files. Although they should still be written to mbox files too.
2-12h38-~#dovecot --version 1.0.15
I'd first try a recent v1.1 release. It might have already fixed this bug. Also you could try if setting mbox_lazy_writes=no helps.
Done that. Upgraded to 1.1.15, and set mbox_dirty_syncs: no and mbox_lazy_writes: no and restarted. Still the same behavior: some e-mails are still seen as been labeled by thunderbird while the mbox file does not contain any label for these e-mails. How can I tell dovecot to re-generate its index files? I guess they have been wrong at some point and now it keeps using that wrong information.
Or if anybody has any other idea... Olivier
participants (2)
-
Olivier Cailloux
-
Timo Sirainen