Shared mailboxes, users with dots and a bug in subscriptions
Hi,
i'm running Dovecot 2.3.14.alpha0 with shared namespaces and stumbled across some errors messages logged, when the list of subscribed mailboxes is queried by a client. For every distinct account of in the list of subscriptions two corresponding lines are logged:
Jan 28 11:42:34 mx1 dovecot: auth: missing passwd file: /etc/dovecot/private/example/users
Jan 28 11:42:34 mx1 dovecot: auth: missing passwd file: /etc/dovecot/private/example/users
Jan 28 11:42:34 mx1 dovecot: auth: missing passwd file: /etc/dovecot/private/example/users
Jan 28 11:42:36 mx1 dovecot: imap(example_user@example.com)<3638>
Similar messages are logged, when invalid entries are listed in '/var/lib/dovecot/db/shared-mailboxes', which i already pruned and haven't received them anymore since.
I think these errors are caused by an unintended behaviour when writing "~/Maildir/subscriptions", which looks (shortened) like this.
V 2
INBOX/INBOX shared/noc@example com/INBOX shared/info@example com/INBOX shared/root@example com/test shared/root@example com/test test_sub
The subscription-file.c explodes the name on every hierarchy separator ('.','\0') and inserts a TAB character. Unfortunately it also explodes on the DNS label delimiter „.‟. This should probably be fixed by passing a structure containing the required information to the formatter to distinguish mailboxes from domain-names.
Subscription in combination with multiple domains and shared mailboxes seems broken to me. Actually i can't even explain to me, why it is working in face of the errors. :-)
Unfortunately in Maildir++ the separator dot is hard-coded.
There is a very old thread on this mailing list, that suggests using „auth_username_translation‟ to replace dots with a different character, but this idea is getting worse the longer i think about it.
I absolutely dislike the idea to set LAYOUT=fs, namespace/separator = § to change the separators to split on, because this would mean to restructure the physical layout of all mailboxes (hierarchically) and mess around with lots of files.
I attached a dovecot-sysreport to reproduce the behaviour.
/etc/dovecot/private/example.com/users looks like this: ### user:password:uid:gid:(gecos):home:(shell):extra_fields noc:{SSHA512}_hash_:::::: info:{SSHA512}_hash_:::::: root:{SSHA512}_hash_::::::
Please correct me if i'm wrong or point me to a workaround, but i think the layout code needs some love. :-)
Best regards Tobias
On 28/01/2021 15:15 Tobias Stein tobias_stein@rockstable.it wrote:
Hi,
i'm running Dovecot 2.3.14.alpha0 with shared namespaces and stumbled across some errors messages logged, when the list of subscribed mailboxes is queried by a client. For every distinct account of in the list of subscriptions two corresponding lines are logged:
Jan 28 11:42:34 mx1 dovecot: auth: missing passwd file: /etc/dovecot/private/example/users Jan 28 11:42:34 mx1 dovecot: auth: missing passwd file: /etc/dovecot/private/example/users Jan 28 11:42:34 mx1 dovecot: auth: missing passwd file: /etc/dovecot/private/example/users Jan 28 11:42:36 mx1 dovecot: imap(example_user@example.com)<3638>
: Error: mkdir(/var/run/dovecot/user-not-found/noc@example) failed: Permission denied (euid=109(vmail) egid=118(vmail) missing +w perm: /var/run/dovecot, dir owned by 0:0 mode=0755) Jan 28 11:42:36 mx1 dovecot: imap(example_user@example.com)<3638> : Error: mkdir(/var/run/dovecot/user-not-found/info@example) failed: Permission denied (euid=109(vmail) egid=118(vmail) missing +w perm: /var/run/dovecot, dir owned by 0:0 mode=0755) Jan 28 11:42:36 mx1 dovecot: imap(example_user@example.com)<3638> : Error: mkdir(/var/run/dovecot/user-not-found/root@example) failed: Permission denied (euid=109(vmail) egid=118(vmail) missing +w perm: /var/run/dovecot, dir owned by 0:0 mode=0755) Similar messages are logged, when invalid entries are listed in '/var/lib/dovecot/db/shared-mailboxes', which i already pruned and haven't received them anymore since.
I think these errors are caused by an unintended behaviour when writing "~/Maildir/subscriptions", which looks (shortened) like this.
V 2
INBOX/INBOX shared/noc@example com/INBOX shared/info@example com/INBOX shared/root@example com/test shared/root@example com/test test_sub
The subscription-file.c explodes the name on every hierarchy separator ('.','\0') and inserts a TAB character. Unfortunately it also explodes on the DNS label delimiter „.‟. This should probably be fixed by passing a structure containing the required information to the formatter to distinguish mailboxes from domain-names.
Subscription in combination with multiple domains and shared mailboxes seems broken to me. Actually i can't even explain to me, why it is working in face of the errors. :-)
Unfortunately in Maildir++ the separator dot is hard-coded.
There is a very old thread on this mailing list, that suggests using „auth_username_translation‟ to replace dots with a different character, but this idea is getting worse the longer i think about it.
I absolutely dislike the idea to set LAYOUT=fs, namespace/separator = § to change the separators to split on, because this would mean to restructure the physical layout of all mailboxes (hierarchically) and mess around with lots of files.
I attached a dovecot-sysreport to reproduce the behaviour.
/etc/dovecot/private/example.com/users looks like this: ### user:password:uid:gid:(gecos):home:(shell):extra_fields noc:{SSHA512}_hash_:::::: info:{SSHA512}_hash_:::::: root:{SSHA512}_hash_::::::
Please correct me if i'm wrong or point me to a workaround, but i think the layout code needs some love. :-)
Best regards Tobias
You should probably add :LAYOUT=FS on your mail locations. This will change the folder naming into foo/bar/baz instead of .foo.bar.baz.
Aki
Hi Aki,
Thanks for your prompt reply! :-) And because i classically forgot to attach the dovecot-sysreport, i'll deliver it now. :-)
Yes, you're right. Setting :LAYOUT=fs would be a workaround. I'd also have to migrate every single mailbox to the new hierarchical layout. The hierarchical separator list->sep would indeed change to „/‟ and the subscriptions would be split differently.
Please correct me when i'm wrong, but the namespace/separator would have to be changed too, to prevent splitting on another "wrong" position. The current shared/root@example com/test subtest would become to shared root@example.com test subtest. Which is also wrong because there is no user shared. So the namespace separator could be set to again something different (from „auth_username_chars‟ + "/+") like „^°!§%&=?;:#¹²³‟ which all would be ugly. And with namespace/sep set to „°‟ leading to the form shared°root@example.com°test subtest.
But this would not resolve the actual bug, that subscriptions are not split and persisted correctly. In the end i would just be forced to use :LAYOUT=fs to mitigate the bug, even if i like the flat layout. :-)
I think there should be a default, which is valid for a common deployment with all features working. Maildir++ for sure is a great choice for this, but the implementation has a flaw: a hard-coded „separator‟, which collides with the DNS label delimiter, when storing subscriptions.
Best Regards Tobias
On 28/01/2021 16:55 Tobias Stein tobias_stein@rockstable.it wrote:
Hi Aki,
Thanks for your prompt reply! :-) And because i classically forgot to attach the dovecot-sysreport, i'll deliver it now. :-)
Yes, you're right. Setting :LAYOUT=fs would be a workaround. I'd also have to migrate every single mailbox to the new hierarchical layout. The hierarchical separator list->sep would indeed change to „/‟ and the subscriptions would be split differently.
Please correct me when i'm wrong, but the namespace/separator would have to be changed too, to prevent splitting on another "wrong" position. The current shared/root@example com/test subtest would become to shared root@example.com test subtest. Which is also wrong because there is no user shared. So the namespace separator could be set to again something different (from „auth_username_chars‟ + "/+") like „^°!§%&=?;:#¹²³‟ which all would be ugly. And with namespace/sep set to „°‟ leading to the form shared°root@example.com°test subtest.
But this would not resolve the actual bug, that subscriptions are not split and persisted correctly. In the end i would just be forced to use :LAYOUT=fs to mitigate the bug, even if i like the flat layout. :-)
I think there should be a default, which is valid for a common deployment with all features working. Maildir++ for sure is a great choice for this, but the implementation has a flaw: a hard-coded „separator‟, which collides with the DNS label delimiter, when storing subscriptions.
Best Regards Tobias
You can also just change the namespace hierarchy separator to fix this:
namespace { separator = / }
This will cause clients to redownload mails but requires no other changes.
Aki
Hi,
Thanks for your support.
So a workaround would involve the migration from the flat to an hierarchical LAYOUT=fs, change the hierarchy separator to „/‟ and the namespace separator to a rarely used symbol like „§‟. Okay, that actually sounds like a nightmare to me.
But this would not resolve the actual bug, that subscriptions are not split and persisted correctly. In the end i would just be forced to use :LAYOUT=fs to mitigate the bug, even if i like the flat layout. :-)
What do you think about accepting the miss-behaviour and fixing the splitting function in "subscription-file.c" ? :-D
Best regards Tobias
Rockstable IT UG (haftungsbeschränkt) Löhrstr. 19 04105 Leipzig
Sitz der Gesellschaft: Leipzig Registergericht: Amtsgericht Leipzig, HRB 36289 Geschäftsführer: Tobias Stein USt-IdNr.: DE324059204 https://www.rockstable.it/
Free Software Foundation Europe - Was ist Freie Software? https://fsfe.org/freesoftware/freesoftware.de.html Public Money? Public Code! https://publiccode.eu/de/
Am Donnerstag, 28. Januar 2021, 16:06:46 CEST schrieb Aki Tuomi:
On 28/01/2021 16:55 Tobias Stein tobias_stein@rockstable.it wrote:
Hi Aki,
Thanks for your prompt reply! :-) And because i classically forgot to attach the dovecot-sysreport, i'll deliver it now. :-)
Yes, you're right. Setting :LAYOUT=fs would be a workaround. I'd also have to migrate every single mailbox to the new hierarchical layout. The hierarchical separator list->sep would indeed change to „/‟ and the subscriptions would be split differently.
Please correct me when i'm wrong, but the namespace/separator would have to be changed too, to prevent splitting on another "wrong" position. The current shared/root@example com/test subtest would become to shared root@example.com test subtest. Which is also wrong because there is no user shared. So the namespace separator could be set to again something different (from „auth_username_chars‟ + "/+") like „^°!§%&=?;:#¹²³‟ which all would be ugly. And with namespace/sep set to „°‟ leading to the form shared°root@example.com°test subtest.
But this would not resolve the actual bug, that subscriptions are not split and persisted correctly. In the end i would just be forced to use :LAYOUT=fs to mitigate the bug, even if i like the flat layout. :-)
I think there should be a default, which is valid for a common deployment with all features working. Maildir++ for sure is a great choice for this, but the implementation has a flaw: a hard-coded „separator‟, which collides with the DNS label delimiter, when storing subscriptions.
participants (2)
-
Aki Tuomi
-
Tobias Stein