[Dovecot] dot-containing foldernames \HasNoChildren bug ?
Hello,
we experience problems with dovecot configuration using the listescape plugin to allow dots in foldernames. In some mail clients, dot-containing folders won't show any subfolders. This seems to be related to the fact that the \HasNoChildren flag is set on such folders even if subfolders do exits.
Here is an example from telnet session:
- LIST (\HasNoChildren) "/" "my.test"
- LIST (\HasNoChildren) "/" "my.test/aaa"
- LIST (\HasNoChildren) "/" "my.test/folder.first"
- LIST (\HasNoChildren) "/" "my.test/folder.first/sub.sub.folder"
- LIST (\HasNoChildren) "/" "my.test/folder.second"
when we rename my.test to my_test everything works as expected:
LIST (\HasChildren) "/" "my_test"
LIST (\HasNoChildren) "/" "my_test/aaa"
LIST (\HasNoChildren) "/" "my_test/folder.first"
LIST (\HasNoChildren) "/" "my_test/folder.first/sub.sub.folder"
LIST (\HasNoChildren) "/" "my_test/folder.second"
Could somebody look into this issue or give us some more debugging hints ?
We use dovecot 1.2.13 + maildir + acl+quota+listescape, the config is:
# 1.2.13: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.31.13-18-xen x86_64 openSUSE 11.2 (x86_64) xfs
base_dir: /var/run/dovecot/
log_path: /var/log/dovecot
protocols: imap imaps managesieve
ssl: no
disable_plaintext_auth: no
login_dir: /var/run/dovecot/login
login_executable(default): /usr/lib/dovecot/imap-login
login_executable(imap): /usr/lib/dovecot/imap-login
login_executable(managesieve): /usr/lib/dovecot/managesieve-login
login_processes_count: 5
mail_privileged_group: vmail
mail_uid: vmail
mail_gid: vmail
mail_location: maildir:/data/vmail/%u/Maildir
mail_debug: yes
mail_executable(default): /etc/dovecot/gmember.sh
mail_executable(imap): /etc/dovecot/gmember.sh
mail_executable(managesieve): /usr/lib/dovecot/managesieve
mail_plugins(default): fts fts_squat acl imap_acl quota imap_quota
listescape
mail_plugins(imap): fts fts_squat acl imap_acl quota imap_quota
listescape
mail_plugins(managesieve):
mail_plugin_dir(default):
/usr/lib64/dovecot/modules/imap
mail_plugin_dir(imap):
/usr/lib64/dovecot/modules/imap
mail_plugin_dir(managesieve):
/usr/lib64/dovecot/modules/managesieve
namespace:
type:
private
separator:
/
inbox:
yes
list:
yes
subscriptions:
yes
namespace:
type:
shared
separator:
/
prefix:
Shared/%%u/
location:
maildir:/data/vmail/%%u/Maildir:INDEX=/data/vmail/%u/index/shared/%%u
list:
yes
namespace:
type:
public
separator:
/
prefix:
Public/
location:
maildir:/data/vmail/Public:CONTROL=/data/vmail/%u/index/public:INDEX=/data/vmail/%u/index/public
list:
yes
lda:
postmaster_address: postmaster@xxx.co.at hostname: imap
mail_plugins: acl sieve quota
listescape
mail_plugin_dir:
/usr/lib64/dovecot/modules/lda
log_path: /var/log/dovecot-deliver-errors.log
info_log_path: /var/log/dovecot-deliver.log
auth_socket_path: /var/run/dovecot/auth-master auth default: mechanisms: plain login user: vmail master_user_separator: # passdb: driver: passwd-file args: /etc/dovecot/dovecot.deny deny: yes passdb: driver: passwd-file args: /etc/dovecot/passwd.masters master: yes passdb: driver: ldap args: /etc/dovecot/dovecot-ldap_pass.conf userdb: driver: ldap args: /etc/dovecot/dovecot-ldap_user.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail plugin: quota: maildir quota_rule: *:storage=16G quota_rule2: Trash:storage=100M quota_rule3: Junk:ignore quota_warning: storage=85%% /etc/dovecot/quota_warning.sh 85 %u quota_warning2: storage=90%% /etc/dovecot/quota_warning.sh 90 %u quota_warning3: storage=95%% /etc/dovecot/quota_warning.sh 95 %u acl: vfile:/etc/dovecot/acls:cache_secs=300 acl_shared_dict: file:/var/lib/dovecot/shared-mailboxes.db sieve: /data/vmail/%u/.dovecot.sieve sieve_dir: /data/vmail/%u/sieve fts: squat fts_squat: partial=4 full=7
On Thu, 2010-08-05 at 10:06 +0200, Samuel Kvasnica wrote:
we experience problems with dovecot configuration using the listescape plugin to allow dots in foldernames. In some mail clients, dot-containing folders won't show any subfolders. This seems to be related to the fact that the \HasNoChildren flag is set on such folders even if subfolders do exits. .. We use dovecot 1.2.13 + maildir + acl+quota+listescape, the config is:
I think this problem shows up only with acl+listescape combination. I fixed it for v2.0 now with http://hg.dovecot.org/dovecot-2.0/rev/e77310eedfbb. Some of the other commits before that were also required to get it working correctly. I'm not sure if porting that patch to v1.2 would fix it or not, probably too much trouble anyway..
Hi Timo,
On 08/05/2010 06:26 PM, Timo Sirainen wrote:
I think this problem shows up only with acl+listescape combination. I fixed it for v2.0 now with http://hg.dovecot.org/dovecot-2.0/rev/e77310eedfbb. Some of the other commits before that were also required to get it working correctly. I'm not sure if porting that patch to v1.2 would fix it or not, probably too much trouble anyway..
Well, my trouble is, i need shared folders, so i have to use acls. And we have dots in existing mail folders which have to be migrated to dovecot very soon.
Is it really not feasible to port the patch to 1.2 ? (could you give a hint, what other patches might be needed?)
Is 2.0 stable enough to be deployed in production environment right now ?
regards,
Sam
On 5.8.2010, at 21.13, Samuel Kvasnica wrote:
Hi Timo,
On 08/05/2010 06:26 PM, Timo Sirainen wrote:
I think this problem shows up only with acl+listescape combination. I fixed it for v2.0 now with http://hg.dovecot.org/dovecot-2.0/rev/e77310eedfbb. Some of the other commits before that were also required to get it working correctly. I'm not sure if porting that patch to v1.2 would fix it or not, probably too much trouble anyway..
Is it really not feasible to port the patch to 1.2 ? (could you give a hint, what other patches might be needed?)
If that one patch alone fixes it (but I'm 70% sure it doesn't), it wouldn't be too difficult to backport it. Who knows, the patch might even apply without any conflicts (you can try). But if that isn't enough, then the changes would be too big.
Is 2.0 stable enough to be deployed in production environment right now ?
I'll probably release 2.0.rc5 tomorrow and 2.0.0 next week. The current v2.0 rcs are also in production in several small/medium sized installations.
On 08/06/2010 12:49 AM, Timo Sirainen wrote:
On 5.8.2010, at 21.13, Samuel Kvasnica wrote:
Hi Timo,
On 08/05/2010 06:26 PM, Timo Sirainen wrote:
I think this problem shows up only with acl+listescape combination. I fixed it for v2.0 now with http://hg.dovecot.org/dovecot-2.0/rev/e77310eedfbb. Some of the other commits before that were also required to get it working correctly. I'm not sure if porting that patch to v1.2 would fix it or not, probably too much trouble anyway.. Is it really not feasible to port the patch to 1.2 ? (could you give a hint, what other patches might be needed?) If that one patch alone fixes it (but I'm 70% sure it doesn't), it wouldn't be too difficult to backport it. Who knows, the patch might even apply without any conflicts (you can try). But if that isn't enough, then the changes would be too big.
I tried already but it does not even compile, some basic functionality in array.h is missing, seems really like big changes...
Is 2.0 stable enough to be deployed in production environment right now ?
I'll probably release 2.0.rc5 tomorrow and 2.0.0 next week. The current v2.0 rcs are also in production in several small/medium sized installations. Hmm, might be really the best way, I would give it a try. But I'm having quite a hard time to convert configs to 2.0. The docs on wiki is not complete and the
doveconf -n -c dovecot-1.conf > dovecot-2.conf
command seems to remove rather big part of of original config... still fighting with sieve and some other plugins.
On 08/06/2010 01:24 PM, Timo Sirainen wrote:
On 6.8.2010, at 8.38, Samuel Kvasnica wrote:
doveconf -n -c dovecot-1.conf > dovecot-2.conf
command seems to remove rather big part of of original config... What parts? Seems like something went wrong (might be my fault), there was simply some stuff like sieve, fts completely missing. But now it seems ok. Anyway I've rewritten my configs by hand to use the new nice conf.d layout.
But as I written recently, listescape is still badly broken, cannot use any subfolders now if I use activate plugin. But the \hasnochildren bug is away !
I also have troubles to get managesieve work again, client getting errors and seen some signal 11 on imap service in log, but I have to investigate some more hour to get usable infos.
On Fri, 2010-08-06 at 14:48 +0200, Samuel Kvasnica wrote:
On 08/06/2010 01:24 PM, Timo Sirainen wrote:
On 6.8.2010, at 8.38, Samuel Kvasnica wrote:
doveconf -n -c dovecot-1.conf > dovecot-2.conf
command seems to remove rather big part of of original config... What parts? Seems like something went wrong (might be my fault), there was simply some stuff like sieve, fts completely missing. But now it seems ok.
It was supposed to do a full conversion, so if there are bugs I would like to know about it.
I also have troubles to get managesieve work again, client getting errors and seen some signal 11 on imap service in log, but I have to investigate some more hour to get usable infos.
Signal 11 means crash, and getting a gdb backtrace helps locate and fix it: http://dovecot.org/bugreport.html
Hi Timo,
I got 2.0 from trunk kind of working now, but the listescape is still broken ! The haschildren flag is correct now, but I cannot use (list or create) any subfolders in general when listescape enabled in config, only folders directly in inbox root work normal.
When clicking e.g. at folder 'zzz', getting these error messages on client: "The current operation on 'zzz' did not suceed. The mail server for account xxx@aaa.com responded: Invalid mailbox name: xxx/zzz"
I'll probably release 2.0.rc5 tomorrow and 2.0.0 next week. The current v2.0 rcs are also in production in several small/medium sized installations.
On Fri, 2010-08-06 at 11:31 +0200, Samuel Kvasnica wrote:
I got 2.0 from trunk kind of working now, but the listescape is still broken ! The haschildren flag is correct now, but I cannot use (list or create) any subfolders in general when listescape enabled in config, only folders directly in inbox root work normal.
When clicking e.g. at folder 'zzz', getting these error messages on client: "The current operation on 'zzz' did not suceed. The mail server for account xxx@aaa.com responded: Invalid mailbox name: xxx/zzz"
And this is the latest hg? I thought http://hg.dovecot.org/dovecot-2.0/rev/827ebadc5995 fixed this bug. Also I can't reproduce it:
x create a.b/x x OK Create completed. x create x/y x OK Create completed.
What's your current doveconf -n output?
Hi Timo,
On 08/06/2010 03:54 PM, Timo Sirainen wrote:
On Fri, 2010-08-06 at 11:31 +0200, Samuel Kvasnica wrote:
I got 2.0 from trunk kind of working now, but the listescape is still broken ! The haschildren flag is correct now, but I cannot use (list or create) any subfolders in general when listescape enabled in config, only folders directly in inbox root work normal.
When clicking e.g. at folder 'zzz', getting these error messages on client: "The current operation on 'zzz' did not suceed. The mail server for account xxx@aaa.com responded: Invalid mailbox name: xxx/zzz" And this is the latest hg? I thought http://hg.dovecot.org/dovecot-2.0/rev/827ebadc5995 fixed this bug. Also Well, I work only with subversion, no experience with mercurial. But I downloaded the bz2 for tip from dovecot-2.0, I suppose tip is the "trunk" which should include all latest changes. Currently it is dovecot-2-0-4ef2280ee490 and it is still broken. If listescape is included and I click on any subfolder in thunderbird, get those error messages.
I can't reproduce it:
x create a.b/x x OK Create completed. x create x/y x OK Create completed.
for me it is:
. create my.test/eee . NO Invalid mailbox name: my.test/eee
or even without dot
. create xxx/aaa . NO Invalid mailbox name: xxx/aaa
while root level works: . create eee . OK Create completed.
What's your current doveconf -n output?
# 2.0.rc4: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.31.13-18-xen x86_64 openSUSE 11.2 (x86_64) xfs
auth_master_user_separator = #
auth_mechanisms = plain login
auth_verbose = yes
disable_plaintext_auth = no
hostname = imap
log_path = /var/log/dovecot
mail_access_groups = vmail
mail_debug = yes
mail_gid = vmail
mail_location = maildir:/data/vmail/%u/Maildir
mail_plugins = fts fts_squat acl imap_acl quota imap_quota
mail_privileged_group = vmail
mail_uid = vmail
managesieve_notify_capability = mailto
managesieve_sieve_capability = comparator-i;octet
comparator-i;ascii-casemap fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex
imap4flags copy include variables body enotify environment mailbox date
spamtest spamtestplus
virustest
namespace {
hidden = no
inbox = yes
list = yes
location
prefix
separator = /
subscriptions = yes
type = private
}
namespace {
list = children
location = maildir:/data/vmail/%%u/Maildir:INDEX=/data/vmail/%u/index/shared/%%u
prefix = Shared/%%u/
separator = /
subscriptions = no
type = shared
}
namespace {
location = maildir:/data/vmail/Public:CONTROL=/data/vmail/%u/index/public:INDEX=/data/vmail/%u/index/public
prefix = Public/
separator = /
subscriptions = no
type = public
}
passdb {
args = /etc/dovecot/dovecot.deny
deny = yes
driver = passwd-file
}
passdb {
args = /etc/dovecot/passwd.masters
driver = passwd-file
master = yes
}
passdb {
args = /etc/dovecot/dovecot-ldap_pass.conf
driver = ldap
}
plugin {
acl = vfile:/etc/dovecot/global-acls:cache_secs=300
acl_shared_dict = file:/var/lib/dovecot/shared-mailboxes.db
fts = squat
fts_squat = partial=4 full=7
listescape_char =
quota = maildir:User quota
quota_rule = *:storage=16G
quota_rule2 = Trash:storage=+100M
quota_rule3 = Junk:ignore
quota_warning = storage=85%% quota_warning 85 %u
quota_warning2 = storage=95%% quota_warning 95 %u
sieve = /data/vmail/%u/.dovecot.sieve
sieve_dir = /data/vmail/%u/sieve
}
postmaster_address = postmaster@ims.co.at
protocols = imap sieve
service imap {
process_limit = 2048
}
service quota-warning {
executable = script /etc/dovecot/quota-warning.sh
unix_listener quota-warning {
user = vmail
}
user = dovecot
}
ssl = no
userdb {
args = /etc/dovecot/dovecot-ldap_user.conf
driver = ldap
}
verbose_proctitle = yes
protocol lda {
mail_plugins = $mail_plugins acl sieve quota listescape
}
protocol imap {
mail_plugins = $mail_plugins fts fts_squat acl imap_acl quota
imap_quota listescape
}
On Fri, 2010-08-06 at 16:51 +0200, Samuel Kvasnica wrote:
. create my.test/eee . NO Invalid mailbox name: my.test/eee
Looks like yet another problem with handling hooks. Fixed: http://hg.dovecot.org/dovecot-2.0/rev/e54df5af98ac
I hope this doesn't now break another plugin (combination).. I guess I'll have to delay v2.0.0 release to make sure.
Hi Timo,
yes this works now for thunderbird. But still getting the segfault when using open-xchange weblient right on login. I'm attaching the stacktrace of imap daemon below. Seems to appear in combination with a shared folder of a user with loginID "firstname.lastname". Strangely, it wont crash when using thunderbird.
...yes it might take fes more day until declaring it stable. On the other hand, in 1.2.x listescape is badly broken anyway.
On 08/06/2010 06:06 PM, Timo Sirainen wrote:
On Fri, 2010-08-06 at 16:51 +0200, Samuel Kvasnica wrote:
. create my.test/eee . NO Invalid mailbox name: my.test/eee
Looks like yet another problem with handling hooks. Fixed: http://hg.dovecot.org/dovecot-2.0/rev/e54df5af98ac
I hope this doesn't now break another plugin (combination).. I guess I'll have to delay v2.0.0 release to make sure.
On Fri, 2010-08-06 at 19:51 +0200, SK wrote:
yes this works now for thunderbird. But still getting the segfault when using open-xchange weblient right on login. ..
#0 list_unescape_str (dest=<value optimized out>, str=0x6c8597 "sharing.user", ns=<value optimized out>) at listescape-plugin.c:91
mlist = 0x0
num = <value optimized out>
#1 listescape_mailbox_list_iter_next (dest=<value optimized out>, str=0x6c8597 "sharing.user", ns=<value optimized out>)
at listescape-plugin.c:211
On 08/06/2010 07:59 PM, Timo Sirainen wrote:
num = <value optimized out>
#1 listescape_mailbox_list_iter_next (dest=<value optimized out>, str=0x6c8597 "sharing.user", ns=<value optimized out>)
at listescape-plugin.c:211
Wow that was fast !
Ok, now it does not crash anymore, But...: I cannot see shared folders for users containing a dot (e.g. the "shared.user" above) on open-exchange webclient. If I disable the listscape plugin, the folder is visible. Something more is still broken. Any idea where to look ?
On Fri, 2010-08-06 at 20:41 +0200, SK wrote:
Ok, now it does not crash anymore, But...: I cannot see shared folders for users containing a dot (e.g. the "shared.user" above) on open-exchange webclient. If I disable the listscape plugin, the folder is visible. Something more is still broken. Any idea where to look ?
I'm not sure if this is fixable. I'll look into it later..
On 08/06/2010 09:04 PM, Timo Sirainen wrote:
On Fri, 2010-08-06 at 20:41 +0200, SK wrote:
Ok, now it does not crash anymore, But...: I cannot see shared folders for users containing a dot (e.g. the "shared.user" above) on open-exchange webclient. If I disable the listscape plugin, the folder is visible. Something more is still broken. Any idea where to look ?
I'm not sure if this is fixable. I'll look into it later..
Well, that would be an essential showstopper for the migration from scalix to dovecot in our setup. But I don't believe it is unfixable. I did some tcpdump logging in the meantime, The open-xchange client seems to rely heavily on the LSUB command which is different from e.g. thunderbird which seems to use mostly LIST.
It appears that the dovecot replies for LSUB commands with '%' wildcard are different when listescape is enabled. Please have a look at the attached comparison. At least for the public folders it is pretty clear (even when not using dots there) and subfolders in public area are broken as well. I cannot find the clear difference for the shared folder case though, but more info comes later on... the public folder bug might be the key...
Hi Timo,
in addition to my last message regarding public folders I did some more testing on the shared folder issue.
It seems like there are 2 separate issues, both related to misbehaving LSUB command but different.
For a shared folder a command like e.g.:
, lsub "" "Shared/dot.user/dot_share"
will return nothing, even if this folder is subscribed because list_escape (out of some strange reason) gets an empty prefix and wrongly escapes the user name dot.user to dot\2euser.
For the LIST command on other hand it works, because it first fails with zero prefix but then all namespaces are probed which will succeed for prefix "Shared/dot.user". The difference seems to be the condition in cmd-list.c, list_namespace_init() testing for some subscription flags.
Could you give some hint, whether a quick workaround is possible ? I have a feeling the listescape plugin is implemented at wrong place, somewhere directly above the storage stuff or as an virtual storage interface would be cleaner ?
regards,
Sam
On 08/06/2010 10:40 PM, SK wrote:
On 08/06/2010 09:04 PM, Timo Sirainen wrote:
On Fri, 2010-08-06 at 20:41 +0200, SK wrote:
Ok, now it does not crash anymore, But...: I cannot see shared folders for users containing a dot (e.g. the "shared.user" above) on open-exchange webclient. If I disable the listscape plugin, the folder is visible. Something more is still broken. Any idea where to look ?
I'm not sure if this is fixable. I'll look into it later..
Well, that would be an essential showstopper for the migration from scalix to dovecot in our setup. But I don't believe it is unfixable. I did some tcpdump logging in the meantime, The open-xchange client seems to rely heavily on the LSUB command which is different from e.g. thunderbird which seems to use mostly LIST.
It appears that the dovecot replies for LSUB commands with '%' wildcard are different when listescape is enabled. Please have a look at the attached comparison. At least for the public folders it is pretty clear (even when not using dots there) and subfolders in public area are broken as well. I cannot find the clear difference for the shared folder case though, but more info comes later on... the public folder bug might be the key...
Hi Timo,
since I could not just let it be, i found a workaround, could you review the patch in attachment ?
I'm going to look into the second subfolder bug...
On 08/07/2010 05:36 PM, Samuel Kvasnica wrote:
Hi Timo,
in addition to my last message regarding public folders I did some more testing on the shared folder issue.
It seems like there are 2 separate issues, both related to misbehaving LSUB command but different.
For a shared folder a command like e.g.:
, lsub "" "Shared/dot.user/dot_share"
will return nothing, even if this folder is subscribed because list_escape (out of some strange reason) gets an empty prefix and wrongly escapes the user name dot.user to dot\2euser.
For the LIST command on other hand it works, because it first fails with zero prefix but then all namespaces are probed which will succeed for prefix "Shared/dot.user". The difference seems to be the condition in cmd-list.c, list_namespace_init() testing for some subscription flags.
Could you give some hint, whether a quick workaround is possible ? I have a feeling the listescape plugin is implemented at wrong place, somewhere directly above the storage stuff or as an virtual storage interface would be cleaner ?
regards,
Sam
On Sat, 2010-08-07 at 19:29 +0200, Samuel Kvasnica wrote:
} else if (*str == ns->list->hierarchy_sep ||*str == mlist->escape_char || *str == '/')
} else {if (*str == ns->list->hierarchy_sep || ( mlist && // SK mlist was zero check added !*str == mlist->escape_char) || *str == '/'){ str_printfa(esc, "%c%02x", mlist->escape_char, *str);
else
}else{ str_append_c(esc, *str);}}
Was there a reason for this change?
Anyway, this should also fix it: http://hg.dovecot.org/dovecot-2.0/rev/18d95a2d0aba
There's also a bug with listing shared mailboxes that contain '.' characters. Lets see if I can fix that too..
On Sat, 2010-08-07 at 19:29 +0200, Samuel Kvasnica wrote:
} else if (*str == ns->list->hierarchy_sep ||*str == mlist->escape_char || *str == '/')
} else {if (*str == ns->list->hierarchy_sep || ( mlist && // SK mlist was zero check added !*str == mlist->escape_char) || *str == '/'){ str_printfa(esc, "%c%02x", mlist->escape_char, *str);
else
}else{ str_append_c(esc, *str);}}
On 08/09/2010 06:01 PM, Timo Sirainen wrote: patch without understanding what going on... i'm just terribly under pressure to get the setup working
Anyway, this should also fix it: http://hg.dovecot.org/dovecot-2.0/rev/18d95a2d0aba
There's also a bug with listing shared mailboxes that contain '.' characters. Lets see if I can fix that too..
actually, the LSUB command is totaly broken with listescape for subfolders when using wildcards, and I cannot find the reason why. Not only in shared area, also in INBOX root. Really got lost in the pointer jungle, not sure how to debug further.
no listescape example:
. lsub "" "Public/public/%" listescape active:
- LSUB () "/" "Public/public/ccc" . OK Lsub completed.
. lsub "" "Public/public/%" . OK Lsub completed.
Hope to get this working soon so that I can shut down our buggy scalix server asap.
On Mon, 2010-08-09 at 18:26 +0200, Samuel Kvasnica wrote:
if (*str == ns->list->hierarchy_sep || ( mlist && // SK mlist was zero check added !*str == mlist->escape_char) || *str == '/'){ str_printfa(esc, "%c%02x", mlist->escape_char, *str);
else
}else{ str_append_c(esc, *str);}}
http://hg.dovecot.org/dovecot-2.0/rev/ed61aee681b8 should fix that.
i'm just terribly under pressure to get the setup working
Have you considered on getting rid of listescape plugin and simply using :LAYOUT=fs? :) That should be more reliable..
actually, the LSUB command is totaly broken with listescape for subfolders when using wildcards, and I cannot find the reason why. Not only in shared area, also in INBOX root. Really got lost in the pointer jungle, not sure how to debug further.
no listescape example:
. lsub "" "Public/public/%"
- LSUB () "/" "Public/public/ccc" . OK Lsub completed.
This looks like a public namespace, not private namespace? Anyway I see that the problem with shared/public mailboxes has to do with ACL plugin and listescape plugin interaction.
Problem 1) Rebuilding dovecot-acl-list:
ACL plugin sees "foo.bar" mailbox name and then tries to look for its dovecot-acl file without going through listescape plugin, because it never calls mailbox_alloc().
In acllist_append() you could try replacing mail_namespace_get_storage_name() call with mailbox_alloc() and then using mailbox_get_name(). Then it should write the escaped name to dovecot-acl-list file.
Problem 2) Reading dovecot-acl-list file:
In acl_mailbox_try_list_fast() it reads the dovecot-acl-list with escaped names, but acl_mailbox_list_iter_next_info() compares them with unescaped names.
Maybe in 1) case it should just keep the mailbox names in escaped form in dovecot-acl-list file so 2) wouldn't need to be changed.. In 1) case then you'd still need mailbox_alloc() and pass the escaped name to acl_object_init_from_name() so that it can read the right dovecot-acl file.
I don't really want to try to fix this for v2.0. Seems like a pretty ugly mess that would be fixed much more easily by some API changes.
Hi Timo,
hope this mails gets proper References in header, just bypassed the upstream scalix server by postfix
Was there a reason for this change?
yes, I've got segfaults, mlist was zero ! But thats just an "intuitive" patch without understanding what going on... http://hg.dovecot.org/dovecot-2.0/rev/ed61aee681b8 should fix that.
i'm just terribly under pressure to get the setup working Have you considered on getting rid of listescape plugin and simply using :LAYOUT=fs? :) That should be more reliable.. Yes, I actually started my dovecot1.2 setup using fs layout because I
On 08/09/2010 06:48 PM, Timo Sirainen wrote: thanks, sounds like never-ending escape story liked the idea. Unfortunately I've badly hit the brickwall during the old server migration as it became during imapsync clear that 'new' or 'tmp' are pretty common subfolder names for many users on site. And I really did not want to restrict allowed folder names that much. Any idea how to make 'new' or 'tmp' work with fs layout ?
actually, the LSUB command is totaly broken with listescape for subfolders when using wildcards, and I cannot find the reason why. Not only in shared area, also in INBOX root. Really got lost in the pointer jungle, not sure how to debug further.
no listescape example:
. lsub "" "Public/public/%" that the problem with shared/public mailboxes has to do with ACL plugin
- LSUB () "/" "Public/public/ccc" . OK Lsub completed. This looks like a public namespace, not private namespace? Anyway I see
and listescape plugin interaction. You are right, it is only related to public or shared namespaces.
Problem 1) Rebuilding dovecot-acl-list:
ACL plugin sees "foo.bar" mailbox name and then tries to look for its dovecot-acl file without going through listescape plugin, because it never calls mailbox_alloc().
In acllist_append() you could try replacing mail_namespace_get_storage_name() call with mailbox_alloc() and then using mailbox_get_name(). Then it should write the escaped name to dovecot-acl-list file.
Problem 2) Reading dovecot-acl-list file:
In acl_mailbox_try_list_fast() it reads the dovecot-acl-list with escaped names, but acl_mailbox_list_iter_next_info() compares them with unescaped names.
Maybe in 1) case it should just keep the mailbox names in escaped form in dovecot-acl-list file so 2) wouldn't need to be changed.. In 1) case then you'd still need mailbox_alloc() and pass the escaped name to acl_object_init_from_name() so that it can read the right dovecot-acl file.
I don't really want to try to fix this for v2.0. Seems like a pretty ugly mess that would be fixed much more easily by some API changes.
Ugh, this sounds really, really ugly. alc plugin was the only part where I did not put printfs... ok, now I understand, there was just another hook inbetween. Well, if I don't get it working with fs layout, I will have to do some "dirty" patch. I just wish you had written the code in c++...
Thanks.
On 9.8.2010, at 21.15, Samuel Kvasnica wrote:
Have you considered on getting rid of listescape plugin and simply using :LAYOUT=fs? :) That should be more reliable.. Yes, I actually started my dovecot1.2 setup using fs layout because I liked the idea. Unfortunately I've badly hit the brickwall during the old server migration as it became during imapsync clear that 'new' or 'tmp' are pretty common subfolder names for many users on site. And I really did not want to restrict allowed folder names that much. Any idea how to make 'new' or 'tmp' work with fs layout ?
I think you can add :DIRNAME=mAildirMaiLs or something. Then e.g. cur/new mailbox exists in directory Maildir/cur/new/mAildirMaiLs/
INBOX should be then in Maildir/mAildirMaiLs/ (probably clearer if you also added :INBOX=~/Maildir/INBOX/ to make it less special case).
On 08/10/2010 12:33 AM, Timo Sirainen wrote:
On 9.8.2010, at 21.15, Samuel Kvasnica wrote:
Have you considered on getting rid of listescape plugin and simply using :LAYOUT=fs? :) That should be more reliable.. Yes, I actually started my dovecot1.2 setup using fs layout because I liked the idea. Unfortunately I've badly hit the brickwall during the old server migration as it became during imapsync clear that 'new' or 'tmp' are pretty common subfolder names for many users on site. And I really did not want to restrict allowed folder names that much. Any idea how to make 'new' or 'tmp' work with fs layout ? I think you can add :DIRNAME=mAildirMaiLs or something. Then e.g. cur/new mailbox exists in directory Maildir/cur/new/mAildirMaiLs/
INBOX should be then in Maildir/mAildirMaiLs/ (probably clearer if you also added :INBOX=~/Maildir/INBOX/ to make it less special case). Wow ! Is this a new undocumented feature ? Exactly what I was looking for. Discarding listescape, going for LAYOUT=fs, in hope it won't break anything else depending on maildir layout...
Thanks a lot !
On 10.8.2010, at 23.07, Samuel Kvasnica wrote:
I think you can add :DIRNAME=mAildirMaiLs or something. Then e.g. cur/new mailbox exists in directory Maildir/cur/new/mAildirMaiLs/
INBOX should be then in Maildir/mAildirMaiLs/ (probably clearer if you also added :INBOX=~/Maildir/INBOX/ to make it less special case). Wow ! Is this a new undocumented feature ?
It was already in v1.2, but yeah, it's undocumented because it's mainly used by dbox internally. Didn't think other people would find it very interesting. :)
On 08/09/2010 06:01 PM, Timo Sirainen wrote:
On Sat, 2010-08-07 at 19:29 +0200, Samuel Kvasnica wrote:
} else if (*str == ns->list->hierarchy_sep ||*str == mlist->escape_char || *str == '/')
} else {if (*str == ns->list->hierarchy_sep || ( mlist && // SK mlist was zero check added !*str == mlist->escape_char) || *str == '/'){ str_printfa(esc, "%c%02x", mlist->escape_char, *str);
else
}else{ str_append_c(esc, *str);}}
Core was generated by `dovecot/imap'.
Program terminated with signal 11, Segmentation
fault.
#0 list_escape (ns=0x6488b0, str=0x659eff "%", vname=true) at
listescape-plugin.c:75
75 *str == mlist->escape_char || *str ==
'/')
(gdb) bt
full
#0 list_escape (ns=0x6488b0, str=0x659eff "%", vname=true) at
listescape-plugin.c:75
mlist =
0x0
esc =0x621470
i =0
#1 0x00007fa3d07d5ef8 in iter_escape_patterns (flags=<value optimized out>, patterns=<value optimized out>, list=<value optimized out>) at listescape-plugin.c:151 orig_ns = 0x6214af
i =1
#2 listescape_mailbox_list_iter_init (flags=<value optimized out>,
patterns=<value optimized out>, list=<value optimized out>) at
listescape-plugin.c:169
mlist =
0x64f590
ctx = <value optimizedout>
#3 0x00007fa3d1001f8b in acl_mailbox_list_iter_init (list=0x64f1c8,
patterns=0x6213c8, flags=4866) at
acl-mailbox-list.c:194
ctx =
0x688ca0
i = <value optimizedout>
inboxcase = <value optimizedout>
#4 0x000000000040c746 in list_namespace_init (ctx=<value optimized out>) at cmd-list.c:773
pat =0x6213c8
ns =0x645d10
cur_ref = 0x659ef0""
pattern = 0x659ef8"Shared/%"
inbox_match = <value optimizedout>
#5 cmd_list_continue (ctx=<value optimized out>) at cmd-list.c:810
_data_stack_cur_id =4
ctx =0x6576a8
#6 0x000000000040cd02 in cmd_list_full (cmd=0x6575a8, lsub=true) at cmd-list.c:992
client =0x656e48
args =0x659e40
list_args =0x7fa3d1bdca18
arg_count = <value optimizedout>
ctx =0x6576a8
patterns = {arr = {buffer = 0x6576f8, element_size = 8}, v =0x6576f8, v_modifiable =
0x6576f8}
pattern = 0x659ef8
"Shared/%"
patterns_strarr =0x657730
#7 0x00000000004103ad in client_command_input (cmd=0x6575a8) at imap-client.c:660
client = 0x656e48
command = <value optimized out>
__FUNCTION__ = "client_command_input"#8 0x000000000041048d in client_command_input (cmd=0x6575a8) at imap-client.c:710 client = 0x656e48 command = 0x651898 __FUNCTION__ = "client_command_input" #9 0x00000000004106b5 in client_handle_next_command (remove_io_r=<value optimized out>, client=<value optimized out>) at imap-client.c:751 size = 25 #10 client_handle_input (remove_io_r=<value optimized out>, client=<value optimized out>) at imap-client.c:763 _data_stack_cur_id = 3 ---Type <return> to continue, or q <return> to quit--- ret = false remove_io = <value optimized out> handled_commands = false __FUNCTION__ = "client_handle_input" #11 0x0000000000410fdf in client_input (client=0x656e48) at imap-client.c:802 cmd = <value optimized out> output = 0x657428 bytes = <value optimized out> __FUNCTION__ = "client_input" #12 0x00007fa3d1bce5f5 in io_loop_handler_run (ioloop=<value optimized out>) at ioloop-epoll.c:212 ctx = 0x6297d0 event = 0x629840 list = 0x657500 io = 0x6574c0 tv = {tv_sec = 4, tv_usec = 481822} t_id = 2 msecs = <value optimized out> ret = 1 i = 0 call = <value optimized out> #13 0x00007fa3d1bcd778 in io_loop_run (ioloop=0x6295c0) at ioloop.c:350 No locals. #14 0x00007fa3d1bbd3c3 in master_service_run (service=0x629490, callback=0x659eff) at master-service.c:496 No locals. #15 0x0000000000418f71 in main (argc=1, argv=0x629370) at main.c:358 set_roots = {0x41c480, 0x0} service_flags = <value optimized out> storage_service_flags = MAIL_STORAGE_SERVICE_FLAG_DISALLOW_ROOT postlogin_socket_path = 0x0 username = 0x0 c = <value optimized out>
participants (3)
-
Samuel Kvasnica
-
SK
-
Timo Sirainen