pre-define or alter list of IMAP keywords
I am using Dovecot 2.3.4.1 with sdbox and mostly Thunderbird on Linux and FairEmail on Android as IMAP clients.
I use a defined set of IMAP keywords to allow different users to mark and filter their mails based on keywords. That works pretty well but I stumble across an issue from time to time:
In TB I can pre-define a list of keywords which the user can choose one from.
FairEmail shows all those keywords Dovecot presents - there is nothing like the list in TB.
Issue 1: If there are no mails flagged in a folder, Dovecot doesn't present any keywords and therefore FairEmail shows none - the user has to remember and type in the correct keyword.
Issue 2: After some time FairEmail shows a huge list of keywords for the INBOX, even keywords not in use for quite some time. To get rid of these keywords I deleted the index files on Dovecot (which with sdbox are not meant to be deleted?). It seems, Dovecot caches the keywords in the index files.
Question:
Is there a way on Dovecot to define a list of keywords Dovecot presents to the IMAP clients?
If not, is there a nicer way to clean the keyword lists from old entries not assigned to any mail in that folder?
-- Cheers spi
Am 08.08.22 um 11:57 schrieb spi:
I am using Dovecot 2.3.4.1 with sdbox and mostly Thunderbird on Linux and FairEmail on Android as IMAP clients.
I use a defined set of IMAP keywords to allow different users to mark and filter their mails based on keywords. That works pretty well but I stumble across an issue from time to time:
In TB I can pre-define a list of keywords which the user can choose one from.
FairEmail shows all those keywords Dovecot presents - there is nothing like the list in TB.
Issue 1: If there are no mails flagged in a folder, Dovecot doesn't present any keywords and therefore FairEmail shows none - the user has to remember and type in the correct keyword.
Issue 2: After some time FairEmail shows a huge list of keywords for the INBOX, even keywords not in use for quite some time. To get rid of these keywords I deleted the index files on Dovecot (which with sdbox are not meant to be deleted?). It seems, Dovecot caches the keywords in the index files.
Question:
Is there a way on Dovecot to define a list of keywords Dovecot presents to the IMAP clients?
If not, is there a nicer way to clean the keyword lists from old entries not assigned to any mail in that folder?
I couldn't find any way to pre-define the list of keywords in dovecot so far. The issue here seems also, that dovecot sends a list of keywords after a 'select mailbox' based on what it finds in the index file. And in my case sometimes the keyword list in the index file differs from the keywords set in the respective mailbox' mails.
If there is no way to alter this list of keywords - is there any API or script interface (like lua) with which one could alter the IMAP responses dovecot is sending? For a selcet inbox the server answers with FLAGS and PERMANENTFLAGS - might there be a way to alter the PREMANENTFLAGS response?
Example:
b select inbox
- FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
- OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted.
-- Cheers spi
On 11/08/2022 10:23 EEST spi spi@nurfuerspam.de wrote:
Am 08.08.22 um 11:57 schrieb spi:
I am using Dovecot 2.3.4.1 with sdbox and mostly Thunderbird on Linux and FairEmail on Android as IMAP clients.
I use a defined set of IMAP keywords to allow different users to mark and filter their mails based on keywords. That works pretty well but I stumble across an issue from time to time:
In TB I can pre-define a list of keywords which the user can choose one from.
FairEmail shows all those keywords Dovecot presents - there is nothing like the list in TB.
Issue 1: If there are no mails flagged in a folder, Dovecot doesn't present any keywords and therefore FairEmail shows none - the user has to remember and type in the correct keyword.
Issue 2: After some time FairEmail shows a huge list of keywords for the INBOX, even keywords not in use for quite some time. To get rid of these keywords I deleted the index files on Dovecot (which with sdbox are not meant to be deleted?). It seems, Dovecot caches the keywords in the index files.
Question:
Is there a way on Dovecot to define a list of keywords Dovecot presents to the IMAP clients?
If not, is there a nicer way to clean the keyword lists from old entries not assigned to any mail in that folder?
I couldn't find any way to pre-define the list of keywords in dovecot so far. The issue here seems also, that dovecot sends a list of keywords after a 'select mailbox' based on what it finds in the index file. And in my case sometimes the keyword list in the index file differs from the keywords set in the respective mailbox' mails.
If there is no way to alter this list of keywords - is there any API or script interface (like lua) with which one could alter the IMAP responses dovecot is sending? For a selcet inbox the server answers with FLAGS and PERMANENTFLAGS - might there be a way to alter the PREMANENTFLAGS response?
Example:
b select inbox
- FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
- OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted.
-- Cheers spi
Maybe you should consider fixing this issue in FairMail instead? There is no supported way to manage keywords at this moment. Modifying the actual IMAP responses is very hard.
Aki
Am 11.08.22 um 09:29 schrieb Aki Tuomi:
Maybe you should consider fixing this issue in FairMail instead? There is no supported way to manage keywords at this moment. Modifying the actual IMAP responses is very hard.
Aki
Am trying... But still there is the issue with the difference of keywords actually used in the mails and the ones in the index file. I had to delete the index files to reset that list - and I don't like deleting the index files on sdbox.
Is there a way to fix or circumvent that?
-- Cheers spi
On 11/08/2022 11:02 EEST spi spi@nurfuerspam.de wrote:
Am 11.08.22 um 09:29 schrieb Aki Tuomi:
Maybe you should consider fixing this issue in FairMail instead? There is no supported way to manage keywords at this moment. Modifying the actual IMAP responses is very hard.
Aki
Am trying... But still there is the issue with the difference of keywords actually used in the mails and the ones in the index file. I had to delete the index files to reset that list - and I don't like deleting the index files on sdbox. Is there a way to fix or circumvent that?
-- Cheers spi
Can you show examples of this? All clients should only use the ones provided over IMAP connection, not what the index files contain.
Aki
Am 11.08.22 um 10:03 schrieb Aki Tuomi:
Maybe you should consider fixing this issue in FairMail instead? There is no supported way to manage keywords at this moment. Modifying the actual IMAP responses is very hard.
Aki
Am trying... But still there is the issue with the difference of keywords actually used in the mails and the ones in the index file. I had to delete the index files to reset that list - and I don't like deleting the index files on sdbox. Is there a way to fix or circumvent that?
Can you show examples of this? All clients should only use the ones provided over IMAP connection, not what the index files contain.
Aki
Currently there are some mails in my inbox.
'doveadm -f flow -v fetch -u spi 'flags uid' mailbox Inbox' shows
flags=\Seen uid=3 flags=\Seen $Forwarded uid=4 flags=\Seen NonJunk 3_warten uid=5 flags=\Seen NonJunk uid=6 flags=\Seen NonJunk uid=7 flags=\Seen NonJunk uid=8 flags=\Seen uid=9 flags=\Seen NonJunk uid=10 flags=\Seen NonJunk uid=11 flags=\Seen NonJunk uid=12 flags=\Seen NonJunk uid=13 flags=\Seen NonJunk uid=14 flags=\Seen NonJunk uid=15 flags=\Answered \Seen uid=18 flags=\Seen uid=28 flags=\Seen uid=29 flags=\Seen NonJunk 3_warten uid=102
My client receives from dovecot the following list:
b select inbox
- FLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk $Forwarded 3_warten 1_aktuell 2_projekt 4_irgendwann)
- OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk $Forwarded 3_warten 1_aktuell 2_projekt 4_irgendwann \*)] Flags permitted.
Please seaa for the [1-4]_.* keywords - only 3_warten is used in the inbox, but dovecot sends more. I had even a longer list, that's why I deleted the index files.
To deal with that there are two options - a fixed list on the client with the disadvantage of missing new keywords.
A defined list on the server - with the downside of the user being unable to add new keywords.
So I thing both are somehow needed - a current list sent by dovecot and pre-defined list omm the client.
-- Cheers spi
On 11/08/2022 11:26 EEST spi spi@nurfuerspam.de wrote:
Am 11.08.22 um 10:03 schrieb Aki Tuomi:
Maybe you should consider fixing this issue in FairMail instead? There is no supported way to manage keywords at this moment. Modifying the actual IMAP responses is very hard.
Aki
Am trying... But still there is the issue with the difference of keywords actually used in the mails and the ones in the index file. I had to delete the index files to reset that list - and I don't like deleting the index files on sdbox. Is there a way to fix or circumvent that?
Can you show examples of this? All clients should only use the ones provided over IMAP connection, not what the index files contain.
Aki
Currently there are some mails in my inbox. 'doveadm -f flow -v fetch -u spi 'flags uid' mailbox Inbox' shows flags=\Seen uid=3 flags=\Seen $Forwarded uid=4 flags=\Seen NonJunk 3_warten uid=5 flags=\Seen NonJunk uid=6 flags=\Seen NonJunk uid=7 flags=\Seen NonJunk uid=8 flags=\Seen uid=9 flags=\Seen NonJunk uid=10 flags=\Seen NonJunk uid=11 flags=\Seen NonJunk uid=12 flags=\Seen NonJunk uid=13 flags=\Seen NonJunk uid=14 flags=\Seen NonJunk uid=15 flags=\Answered \Seen uid=18 flags=\Seen uid=28 flags=\Seen uid=29 flags=\Seen NonJunk 3_warten uid=102
My client receives from dovecot the following list: b select inbox
- FLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk $Forwarded 3_warten 1_aktuell 2_projekt 4_irgendwann)
- OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk $Forwarded 3_warten 1_aktuell 2_projekt 4_irgendwann \*)] Flags permitted.
Please seaa for the [1-4]_.* keywords - only 3_warten is used in the inbox, but dovecot sends more. I had even a longer list, that's why I deleted the index files. To deal with that there are two options - a fixed list on the client with the disadvantage of missing new keywords. A defined list on the server - with the downside of the user being unable to add new keywords. So I thing both are somehow needed - a current list sent by dovecot and pre-defined list omm the client.
-- Cheers spi
The reason for this is that keywords cannot be currently deleted, and they do not become deleted when nothing uses them.
If you ever use a keyword for a specific mailbox it stays there until you delete the mailbox (or the indexes).
Aki
Am 11.08.22 um 10:55 schrieb Aki Tuomi:
Currently there are some mails in my inbox. 'doveadm -f flow -v fetch -u spi 'flags uid' mailbox Inbox' shows flags=\Seen uid=3 flags=\Seen $Forwarded uid=4 flags=\Seen NonJunk 3_warten uid=5 flags=\Seen NonJunk uid=6 flags=\Seen NonJunk uid=7 flags=\Seen NonJunk uid=8 flags=\Seen uid=9 flags=\Seen NonJunk uid=10 flags=\Seen NonJunk uid=11 flags=\Seen NonJunk uid=12 flags=\Seen NonJunk uid=13 flags=\Seen NonJunk uid=14 flags=\Seen NonJunk uid=15 flags=\Answered \Seen uid=18 flags=\Seen uid=28 flags=\Seen uid=29 flags=\Seen NonJunk 3_warten uid=102
My client receives from dovecot the following list: b select inbox
- FLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk $Forwarded 3_warten 1_aktuell 2_projekt 4_irgendwann)
- OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk $Forwarded 3_warten 1_aktuell 2_projekt 4_irgendwann \*)] Flags permitted.
Please seaa for the [1-4]_.* keywords - only 3_warten is used in the inbox, but dovecot sends more. I had even a longer list, that's why I deleted the index files. To deal with that there are two options - a fixed list on the client with the disadvantage of missing new keywords. A defined list on the server - with the downside of the user being unable to add new keywords. So I thing both are somehow needed - a current list sent by dovecot and pre-defined list omm the client.
-- Cheers spi The reason for this is that keywords cannot be currently deleted, and they do not become deleted when nothing uses them.
If you ever use a keyword for a specific mailbox it stays there until you delete the mailbox (or the indexes).
Aki
Ok, that explains what I figured out. Is this behavior intended? I could imagine there is a limit of keywords the IMAP server may respond with after a select mailbox.
What would happen if the list is full and the user needs to change a keyword?
Is it safe then to delete the index files on sdbox from time to time?
Is there any way you might think of to get the response overwritten? so no matter what is in the index file the server responds with "a b c"?
-- Cheers spi
On 11/08/2022 12:04 EEST spi spi@nurfuerspam.de wrote:
Am 11.08.22 um 10:55 schrieb Aki Tuomi:
Currently there are some mails in my inbox. 'doveadm -f flow -v fetch -u spi 'flags uid' mailbox Inbox' shows flags=\Seen uid=3 flags=\Seen $Forwarded uid=4 flags=\Seen NonJunk 3_warten uid=5 flags=\Seen NonJunk uid=6 flags=\Seen NonJunk uid=7 flags=\Seen NonJunk uid=8 flags=\Seen uid=9 flags=\Seen NonJunk uid=10 flags=\Seen NonJunk uid=11 flags=\Seen NonJunk uid=12 flags=\Seen NonJunk uid=13 flags=\Seen NonJunk uid=14 flags=\Seen NonJunk uid=15 flags=\Answered \Seen uid=18 flags=\Seen uid=28 flags=\Seen uid=29 flags=\Seen NonJunk 3_warten uid=102
My client receives from dovecot the following list: b select inbox
- FLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk $Forwarded 3_warten 1_aktuell 2_projekt 4_irgendwann)
- OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk $Forwarded 3_warten 1_aktuell 2_projekt 4_irgendwann \*)] Flags permitted.
Please seaa for the [1-4]_.* keywords - only 3_warten is used in the inbox, but dovecot sends more. I had even a longer list, that's why I deleted the index files. To deal with that there are two options - a fixed list on the client with the disadvantage of missing new keywords. A defined list on the server - with the downside of the user being unable to add new keywords. So I thing both are somehow needed - a current list sent by dovecot and pre-defined list omm the client.
-- Cheers spi The reason for this is that keywords cannot be currently deleted, and they do not become deleted when nothing uses them.
If you ever use a keyword for a specific mailbox it stays there until you delete the mailbox (or the indexes).
Aki
Ok, that explains what I figured out. Is this behavior intended? I could imagine there is a limit of keywords the IMAP server may respond with after a select mailbox.
What would happen if the list is full and the user needs to change a keyword?
Is it safe then to delete the index files on sdbox from time to time?
Is there any way you might think of to get the response overwritten? so no matter what is in the index file the server responds with "a b c"?
-- Cheers spi
The maximum limit is pretty large, so it's unlikely you end up with that many keywords.
It's not safe to delete index files from sdbox, it will reset e.g. unseen flags making the mailbox seem like no emails have been read yet.
Aki
Am 11.08.22 um 11:15 schrieb Aki Tuomi:
The reason for this is that keywords cannot be currently deleted, and they do not become deleted when nothing uses them.
If you ever use a keyword for a specific mailbox it stays there until you delete the mailbox (or the indexes).
Aki
Ok, that explains what I figured out. Is this behavior intended? I could imagine there is a limit of keywords the IMAP server may respond with after a select mailbox.
What would happen if the list is full and the user needs to change a keyword?
Is it safe then to delete the index files on sdbox from time to time?
Is there any way you might think of to get the response overwritten? so no matter what is in the index file the server responds with "a b c"?
-- Cheers spi The maximum limit is pretty large, so it's unlikely you end up with that many keywords.
It's not safe to delete index files from sdbox, it will reset e.g. unseen flags making the mailbox seem like no emails have been read yet.
Aki
Hm, that scares me as after some time users get hit by dozens of keywords, most of them not in use anymore. That's what happened to me and that's why I started this discussion.
I understand that the mail client needs to support the definition of (user) keywords, but it seems to me that there needs to be some 'garbage collection' process on dovecot to clean up keywords. BTW - how about other mailbox formats - do they have same issues about collecting keywords?
I am stuck now with that issue. Even tried to alter the IMAP server response on my IMAP reverse proxy.
Is there anything else you could think of and recommend to solve that issue? Like rebuilding still existent index files or like I already asked rewrite the IMAP response?
-- Cheers spi
On 11.08.22 12:22, spi wrote:
What would happen if the list is full and the user needs to change a keyword?
The maximum limit is pretty large, so it's unlikely you end up with that many keywords.
Hm, that scares me as after some time users get hit by dozens of keywords, most of them not in use anymore. That's what happened to me and that's why I started this discussion.
I understand that the mail client needs to support the definition of (user) keywords, but it seems to me that there needs to be some 'garbage collection' process on dovecot to clean up keywords.
(My first thought was "how many users may there be who flag incoming e-mails with the due date for the request contained in them, rather than using a separate calendar or to-do-list application?" ...)
Regards
Jochen Bern Systemingenieur
Binect GmbH
On 11.08.22 12:22, spi wrote:
What would happen if the list is full and the user needs to change a keyword?
The maximum limit is pretty large, so it's unlikely you end up with that many keywords.
Hm, that scares me as after some time users get hit by dozens of keywords, most of them not in use anymore. That's what happened to me and that's why I started this discussion.
I understand that the mail client needs to support the definition of (user) keywords, but it seems to me that there needs to be some 'garbage collection' process on dovecot to clean up keywords.
(My first thought was "how many users may there be who flag incoming e-mails with the due date for the request contained in them, rather than using a separate calendar or to-do-list application?" ...)
Regards
Jochen Bern Systemingenieur
Binect GmbH It has nothing to do with due dates. Currently the users use 4 user keywords and the RFC defined ones (like seen or flagged). Although this
Am 11.08.22 um 12:42 schrieb Jochen Bern: list is pretty short recently I got a list of around 30 keywords presented - just because dovecot collected them over time and reported them back to the mail client.
-- Cheers spi
participants (3)
-
Aki Tuomi
-
Jochen Bern
-
spi