I think I've partially sorted out my issues. First off, as the wiki: http://wiki2.dovecot.org/MailboxFormat/Maildir points out: "The file [dovecot-keywords] must not be directly modified ...", which is what I tried. So the dovecot-keywords file I manually created (entries 0-6):

0 Board_and_Committee 1 Completed 2 Health_Care_meetings 3 Notifications 4 OSHP-DAS 5 personal_or_To_Do 6 Retirement_exits 7 $label5 8 Junk 9 $Forwarded 10 $MDNSent 11 $label2

Was not used by Thunderbird. So, I removed that file, went into Thunderbird, and manually set a random message to tag 1 which I had renamed as "Board and Committee". And behold! a new dovecot-keyword file was created with the single entry:

0 $label1

As you (Peter) point out, Thunderbird retrieves the text for these from prefs.js. The reason, therefore, I have e.g. "11 $label2" in my original hand-edited file is that Thunderbird caused what I had designated as "Health Care Meeting" to be added to #11 since the previous slots were already used by other things which it did not recognize. Hence, they ended up with a IMAP flag of 'l'.

I repeated this for a total of 4 more tags, resulting in dovecot-keyword entries:

0 $label1 1 $label2 2 $label3 3 $label4 4 $label5

Now, setting the tag in Tbird results in the correct text and color being shown for the message and the resulting IMAP tag being set correspondingly to 'a' thru 'e'. Flush with victory, I then foolish violated the "don't modify directly principle" by adding:

5 $label6 6 $label7

Which I thought would correspond to my remaining TB tags. Of course, wrong. When I then set a message to the 6th tag (keyword entry 5, $label6), it did not do it. Instead, for my last 2 tags it created 2 new entries:

7 personal_or_to_do 8 retirement_exits

So, IMAP flags 'f' and 'g' are skipped and "Personal or To Do" tags get an 'h' flag and "Retirement Exits" get an 'i' flag. What was I thinking! Not worth starting over as this user had no Outlook categories set to these values, so so-what if that user's files get flagged with 'h', 'i'.

Also interesting to note that the actual text of these last 2 tags is stored in the dovecot-keywords file. I supposed this means that Thunderbird has a limit of 5 (or 7?) $label's and any added beyond that get stored differently.

But, I'm not finished. Of course, Thunderbird expects to set tags/IMAPflas on virgin messages in the user's mail folders. However, my need is to import Outlook categories and have Thunderbird interpret the IMAP flags correctly having never set them directly.

So, I repeated the same procedure in other folders with messages having Outlook categories. The problem here is that setting a message to "Board and Committee" (Tbird tag 1) did not necessarily resulting in an IMAP flag of 'a'. In some folders it was 'b', some 'c' and some 'd'. I think my problem here was that I had manually set the IMAP flags on these message beforehand. So (I think) when Thunderbird, coordinating with Dovecot, went to set Tag 1, it saw that flag 'a' was already used and it picked the next free flag. Some folders had IMAP files with 2 and 3 flags set, hence Tag 1 getting set to 'c' or 'd' in those folders. I believe if I had not initially pre-set the IMAP flags on these files I would have seen the correct correspondance: Tag 1 = flag 'a', Tag 2 = flag 'b', etc.

What I then did in these cases was first, find out what IMAP flag a Tbird tagged file would be set to (e.g. 'c'), then renamed all my *a files to e.g. *c. That worked. Suddenly, folders with 100 flags thusly set showed up with correct tags and colors in Thunderbird.

To summarize, and this what I will do next time:

1. DO NOT pre-add IMAP flags to mail files.

2. For each folder in which there are Outlook categorized message, set an arbitrary message in that folder to each of the Thunderbird flags corresponding to Outlook categories to determine the correct flag letter -- which also causes Thunderbird and dovecot to save their respective settings.

3. Then, rename each mail file to have the correct IMAP flag(s). That can be quicky done using a script and the list of message having Outlook categories.

I believe this procedure will work correctly and I will confirm that when I process my 2nd Outlook user (who is, btw, the organization Director and who is a super user of Outlook categories! Failure is not an option!).

For anyone needing to do the same thing (migrate Outlook categories), I've included below a VB script to run in Outlook which will output a list of all messages having categories. The IMAP mail file's Message ID can be used to locate the categorized message file in the IMAP folder hierarchy and the category name (following the "~") indicates which Thunderbird tag to map it to. I'll not include the bash script to mass-append IMAP flags to these files as that script will need some revising based on my recent experimentation, but should be a rather simple bash exercise in any case.

Note that the Outlook messages are also the same MAPI files, only the client used (Outlook versus Thunderbird) are different. Outlook does not set IMAP flags to designate categories. Categories are apparently stored in the user's .pst file.

------------CUT---------- Public Sub ListOutlookFolders()

Dim olApp As Outlook.Application
Dim olNamespace As Outlook.Namespace
Dim olFolder As Outlook.MAPIFolder

Set olApp = New Outlook.Application
Set olNamespace = olApp.GetNamespace("MAPI")

For Each olFolder In olNamespace.Folders
    Debug.Print olFolder.Name; ":", olFolder.Description
    ListFolders olFolder, 1
Next

Set olFolder = Nothing
Set olNamespace = Nothing
Set olApp = Nothing

End Sub

Sub ListFolders(myFolder As Outlook.MAPIFolder, Level As Integer) Dim olFolder As Outlook.MAPIFolder ' go through each email scanFolder myFolder

' Now we'll check for subfolders For Each olFolder In myFolder.Folders ' Debug.Print ":"; String(Level * 2, "-"); olFolder.Name

' go through each email scanFolder olFolder

    If olFolder.Folders.Count > 0 Then
        ListFolders olFolder, Level + 1
    End If
Next

End Sub

Sub scanFolder(sFolder As Outlook.MAPIFolder) Dim src As Folder Dim oItem As Object Dim propertyAccessor As Outlook.propertyAccessor Set src = sFolder

Dim strHeader As String

For Each oItem In src.Items If TypeOf oItem Is Outlook.MailItem And oItem.Categories <> "" Then ' Debug.Print "Cat: " + oItem.Categories Set propertyAccessor = oItem.propertyAccessor header = propertyAccessor.GetProperty("http://schemas.microsoft.com/mapi/proptag/0x007D001E") Dim headerLines() As String headerLines() = Split(header, vbCrLf)

    Dim thisHeader As Variant

    For Each thisHeader In headerLines
        If InStr(thisHeader, "Message-ID:") > 0 Then
           Debug.Print thisHeader + "~" + oItem.Categories
           Exit For
       End If
    Next
End If

Next End Sub ----------CUT---------

Example of resulting output:

Message-ID: 201109011105.p81B5666028910@webserver.ohprs.org~Red Category

Hopefully someone finds this useful.

THX --Mark

-----Original Message-----

Subject: Re: IMAP flags and dovecot-keywords not working as expected To: dovecot@dovecot.org From: Peter Chiochetti pch@myzel.net Date: Sat, 30 Jul 2016 11:26:09 +0200

Am 2016-07-30 um 08:00 schrieb Mark Foley: ?

...

However, none of the tags show up correctly in Thunderbird. If I manually set a message to have a tag of 0, the corresponding IMAP file gets a flag of 'm', not 'a' and the following is added to the dovecot-keywords files:

12 $label1

How can I fix this? Where is "$label1" text defined? Why did Thunderbird not snag the text for '0' from the dovecot-keywords file and give the IMAP file a tag of 'a'?

Thunderbird flags are stored in the users prefs.js, eg:

• user_pref("mailnews.tags.$label1.tag", "Important");
• user_pref("mailnews.tags.$label1.color", "#FF0000");

A kind of key->value assignment. The "$label[1-9]" keys are special, where the number magically corresponds to the keyboard shorcut to tag messages, 0 meaning clear all tags.

There can be more than nine tags, but they wont have a shortcut then:

• user_pref("mailnews.tags.ten.tag", "ten");

1. The server will only ever see the key. The user will only ever see the value.

2. If you rename a label in TB, then only the value will change and the server will still see the same key as before.

3. If you rename a key in dovecot, TB will not create a label for it and the affected messages will no longer appear tagged, if TB does not know about the key.

4. Dovecot adds to the keywords as it receives requests from clients: Very likely there is a limit of 26 (letters of the alphabet) per account; a-d=0-3 are already taken for internal use, so 22 remain.

...

My current theory is that the "Default" Thunderbird Tags corresponding to IMAP flags are not changeable and if new tags are create in Tbird, they get new flag letters. That would, of course, mean that if a user changes Thunderbird tag name, they would lose all tag settings on existing message. That doesn't seem right and I hope my theory is wrong.

I think you are mostly wrong: as long as you only use TB to work and as long as you do not exceed the limit you should be save.

Notice that tags are a scarce resource: any key you ever created counts toward the limit - reusing old tags requires you to text-edit both dovecot-keywords and TB prefs.js.

-- peter