IMAP flags and dovecot-keywords not working as expected
Mark Foley
mfoley at ohprs.org
Mon Aug 1 04:20:29 UTC 2016
I think I've partially sorted out my issues. First off, as the wiki:
http://wiki2.dovecot.org/MailboxFormat/Maildir points out: "The file [dovecot-keywords] must
not be directly modified ...", which is what I tried. So the dovecot-keywords file I manually
created (entries 0-6):
0 Board_and_Committee
1 Completed
2 Health_Care_meetings
3 Notifications
4 OSHP-DAS
5 personal_or_To_Do
6 Retirement_exits
7 $label5
8 Junk
9 $Forwarded
10 $MDNSent
11 $label2
Was not used by Thunderbird. So, I removed that file, went into Thunderbird, and manually set a
random message to tag 1 which I had renamed as "Board and Committee". And behold! a new
dovecot-keyword file was created with the single entry:
0 $label1
As you (Peter) point out, Thunderbird retrieves the text for these from prefs.js. The reason,
therefore, I have e.g. "11 $label2" in my original hand-edited file is that Thunderbird caused
what I had designated as "Health Care Meeting" to be added to #11 since the previous slots were
already used by other things which it did not recognize. Hence, they ended up with a IMAP flag
of 'l'.
I repeated this for a total of 4 more tags, resulting in dovecot-keyword entries:
0 $label1
1 $label2
2 $label3
3 $label4
4 $label5
Now, setting the tag in Tbird results in the correct text and color being shown for the message
and the resulting IMAP tag being set correspondingly to 'a' thru 'e'. Flush with victory, I
then foolish violated the "don't modify directly principle" by adding:
5 $label6
6 $label7
Which I thought would correspond to my remaining TB tags. Of course, wrong. When I then set a
message to the 6th tag (keyword entry 5, $label6), it did not do it. Instead, for my last 2
tags it created 2 new entries:
7 personal_or_to_do
8 retirement_exits
So, IMAP flags 'f' and 'g' are skipped and "Personal or To Do" tags get an 'h' flag and
"Retirement Exits" get an 'i' flag. What was I thinking! Not worth starting over as this user
had no Outlook categories set to these values, so so-what if that user's files get flagged with
'h', 'i'.
Also interesting to note that the actual text of these last 2 tags is stored in the
dovecot-keywords file. I supposed this means that Thunderbird has a limit of 5 (or 7?) $label's
and any added beyond that get stored differently.
But, I'm not finished. Of course, Thunderbird expects to set tags/IMAPflas on virgin messages
in the user's mail folders. However, my need is to import Outlook categories and have
Thunderbird interpret the IMAP flags correctly having never set them directly.
So, I repeated the same procedure in other folders with messages having Outlook categories.
The problem here is that setting a message to "Board and Committee" (Tbird tag 1) did not
necessarily resulting in an IMAP flag of 'a'. In some folders it was 'b', some 'c' and some
'd'. I think my problem here was that I had manually set the IMAP flags on these message
beforehand. So (I think) when Thunderbird, coordinating with Dovecot, went to set Tag 1, it
saw that flag 'a' was already used and it picked the next free flag. Some folders had IMAP
files with 2 and 3 flags set, hence Tag 1 getting set to 'c' or 'd' in those folders. I
believe if I had not initially pre-set the IMAP flags on these files I would have seen the
correct correspondance: Tag 1 = flag 'a', Tag 2 = flag 'b', etc.
What I then did in these cases was first, find out what IMAP flag a Tbird tagged file would be
set to (e.g. 'c'), then renamed all my *a files to e.g. *c. That worked. Suddenly, folders
with 100 flags thusly set showed up with correct tags and colors in Thunderbird.
To summarize, and this what I will do next time:
1. DO NOT pre-add IMAP flags to mail files.
2. For each folder in which there are Outlook categorized message, set an arbitrary message in
that folder to each of the Thunderbird flags corresponding to Outlook categories to determine
the correct flag letter -- which also causes Thunderbird and dovecot to save their respective
settings.
3. Then, rename each mail file to have the correct IMAP flag(s). That can be quicky done using a
script and the list of message having Outlook categories.
I believe this procedure will work correctly and I will confirm that when I process my 2nd
Outlook user (who is, btw, the organization Director and who is a super user of Outlook
categories! Failure is not an option!).
For anyone needing to do the same thing (migrate Outlook categories), I've included below a VB
script to run in Outlook which will output a list of all messages having categories. The IMAP
mail file's Message ID can be used to locate the categorized message file in the IMAP folder
hierarchy and the category name (following the "~") indicates which Thunderbird tag to map it
to. I'll not include the bash script to mass-append IMAP flags to these files as that script
will need some revising based on my recent experimentation, but should be a rather simple bash
exercise in any case.
Note that the Outlook messages are also the same MAPI files, only the client used (Outlook
versus Thunderbird) are different. Outlook does not set IMAP flags to designate categories.
Categories are apparently stored in the user's .pst file.
------------CUT----------
Public Sub ListOutlookFolders()
Dim olApp As Outlook.Application
Dim olNamespace As Outlook.Namespace
Dim olFolder As Outlook.MAPIFolder
Set olApp = New Outlook.Application
Set olNamespace = olApp.GetNamespace("MAPI")
For Each olFolder In olNamespace.Folders
Debug.Print olFolder.Name; ":", olFolder.Description
ListFolders olFolder, 1
Next
Set olFolder = Nothing
Set olNamespace = Nothing
Set olApp = Nothing
End Sub
Sub ListFolders(myFolder As Outlook.MAPIFolder, Level As Integer)
Dim olFolder As Outlook.MAPIFolder
' go through each email
scanFolder myFolder
' Now we'll check for subfolders
For Each olFolder In myFolder.Folders
' Debug.Print ":"; String(Level * 2, "-"); olFolder.Name
' go through each email
scanFolder olFolder
If olFolder.Folders.Count > 0 Then
ListFolders olFolder, Level + 1
End If
Next
End Sub
Sub scanFolder(sFolder As Outlook.MAPIFolder)
Dim src As Folder
Dim oItem As Object
Dim propertyAccessor As Outlook.propertyAccessor
Set src = sFolder
Dim strHeader As String
For Each oItem In src.Items
If TypeOf oItem Is Outlook.MailItem And oItem.Categories <> "" Then
' Debug.Print "Cat: " + oItem.Categories
Set propertyAccessor = oItem.propertyAccessor
header =
propertyAccessor.GetProperty("http://schemas.microsoft.com/mapi/proptag/0x007D001E")
Dim headerLines() As String
headerLines() = Split(header, vbCrLf)
Dim thisHeader As Variant
For Each thisHeader In headerLines
If InStr(thisHeader, "Message-ID:") > 0 Then
Debug.Print thisHeader + "~" + oItem.Categories
Exit For
End If
Next
End If
Next
End Sub
----------CUT---------
Example of resulting output:
Message-ID: <201109011105.p81B5666028910 at webserver.ohprs.org>~Red Category
Hopefully someone finds this useful.
THX --Mark
-----Original Message-----
> Subject: Re: IMAP flags and dovecot-keywords not working as expected
> To: dovecot at dovecot.org
> From: Peter Chiochetti <pch at myzel.net>
> Date: Sat, 30 Jul 2016 11:26:09 +0200
>
> Am 2016-07-30 um 08:00 schrieb Mark Foley:
> ?
> >
> > However, none of the tags show up correctly in Thunderbird. If I manually set a message to
> > have a tag of 0, the corresponding IMAP file gets a flag of 'm', not 'a' and the following is
> > added to the dovecot-keywords files:
> >
> > 12 $label1
> >
> > How can I fix this? Where is "$label1" text defined? Why did Thunderbird not snag the text for
> > '0' from the dovecot-keywords file and give the IMAP file a tag of 'a'?
>
> Thunderbird flags are stored in the users prefs.js, eg:
> - user_pref("mailnews.tags.$label1.tag", "Important");
> - user_pref("mailnews.tags.$label1.color", "#FF0000");
>
> A kind of key->value assignment. The "$label[1-9]" keys are special,
> where the number magically corresponds to the keyboard shorcut to tag
> messages, 0 meaning clear all tags.
>
> There can be more than nine tags, but they wont have a shortcut then:
> - user_pref("mailnews.tags.ten.tag", "ten");
>
> 1) The server will only ever see the key. The user will only ever see
> the value.
>
> 2) If you rename a label in TB, then only the value will change and the
> server will still see the same key as before.
>
> 3) If you rename a key in dovecot, TB will not create a label for it and
> the affected messages will no longer appear tagged, if TB does not know
> about the key.
>
> 4) Dovecot adds to the keywords as it receives requests from clients:
> Very likely there is a limit of 26 (letters of the alphabet) per
> account; a-d=0-3 are already taken for internal use, so 22 remain.
>
>
> > My current theory is that the "Default" Thunderbird Tags corresponding to IMAP flags are not
> > changeable and if new tags are create in Tbird, they get new flag letters. That would, of
> > course, mean that if a user changes Thunderbird tag name, they would lose all tag settings on
> > existing message. That doesn't seem right and I hope my theory is wrong.
>
> I think you are mostly wrong: as long as you only use TB to work and as
> long as you do not exceed the limit you should be save.
>
> Notice that tags are a scarce resource: any key you ever created counts
> toward the limit - reusing old tags requires you to text-edit both
> dovecot-keywords and TB prefs.js.
>
> --
> peter
>
More information about the dovecot
mailing list