[Dovecot] 1.0 RC1 status
Today I finally caught up with my mail. I've now read all the mails from the Dovecot list from this year and all my private mails from last few years (totaling around 500 unread mails).
If you think there's still something I should have replied to, please re-send it because I've either missed it or just skipped it for some reason. (Except there are still a couple of mails from this week I've yet to reply to.)
Besides reading mails I've also fixed a lot of bugs today. What's left from my original 1.0 roadmap is:
- Solaris sendfile is broken? Is it? Can someone try? You can test it with maildir by setting mail_save_crlf=yes, saving a somewhat large mail and FETCHing it (and truss that it really uses sendfile)
- LDA: empty mail gives an error.
- quota code should probably be always doing some quota_set_critical() instead of using mail_storage_set_critical(), so that quota_last_error() would work properly
- rfc2557 support for BODYSTRUCTURE, as specified by RFC3501
- keywords: add some limits to how many there can be
- don't return * in PERMANENTFLAGS when we're full
- mbox: how well does dirty sync + status work? it reads the last mail every time? not very good..
- mbox: dirty state should be stored to index (with mbox_very_dirty_syncs)
I think 1.0 could be released without any of those changes.
But what about the connection hangs that some people have had? Do they still occur? Is it only with SSL? I guess I should look at my previous ssl-proxy rewrite, figure out what went wrong with it and try again.
I'm thinking about releasing 1.0 RC1 within a few days. Would be nice if that release actually worked, so testing would be appreciated :) You can try it by getting the dovecot-latest from http://dovecot.org/nightly/
The current changes include:
* PAM: If user's password is expired, give "Password expired" error
message to the user. Now actually working thanks to Vaidas Pilkauskas
* Relicensed dovecot-auth, lib-sql and lib-ntlm to MIT license. See
COPYING file for more information.
* Abuse prevention: When creating a mailbox, limit the number of
hierarchies (up to 20) and the length of the mailbox name within
a hierarchy (up to 200 characters).
* mbox: If saved mail doesn't end with LF, add it ourself so that the
mails always have one empty line before the next From-line.
+ Added --with-statedir configure option which defaults to
$localstatedir/lib/dovecot. ssl-parameters.dat is permanently
stored in that directory and is copied to login_dirs from there.
+ IMAP: Support SASL-IR extension (SASL initial response)
+ Support initial SASL response with LOGIN mechanism. Patch by Anders
Karlsson
+ Added PLAIN-MD4 password scheme. Patch by Andrey Panin.
+ Added support for XFS disk quotas. Patch by Pawel Jarosz
+ If another process deletes the opened mailbox, try to handle it
without writing errors to log file. Handles the most common cases.
+ Added TLS support for LDAP if the library supports it.
- Dovecot corrupted mbox files which had CR+LF linefeeds in headers
- MySQL code could have crashed while escaping strings
- MD4 code with NTLM authentication was broken with 64bit systems.
Patch by Andrey Panin
- Plugin loading was broken in some OSes (eg. FreeBSD)
- Several fixes to handling empty values in configuration file
- Several fixes to dictionary quota backend and dict server.
Also changed how they're configured.
- deliver: Fixed plugin handling settings
- mbox_min_index_size handling was somewhat broken
- passdb passwd-file: extra_args field wasn't read unless the file
was also used as userdb.
Quoting Timo Sirainen:
I'm thinking about releasing 1.0 RC1 within a few days. Would be nice if that release actually worked, so testing would be appreciated :) You can try it by getting the dovecot-latest from http://dovecot.org/nightly/
epoll gets deactivated on my FC5 system. I looked around a little bit and found that for the test, linux/unistd.h should be included instead of asm/unistd.h (the former #includes the latter). Patch attached.
diff -u -r1.277.2.11 configure.in --- dovecot/configure.in 17 Jun 2006 19:32:13 -0000 1.277.2.11 +++ dovecot/configure.in 18 Jun 2006 00:11:35 -0000 @@ -377,7 +377,7 @@ #include <sys/epoll.h> #include <errno.h>
- #include <asm/unistd.h>
#include <linux/unistd.h>
_syscall1 (int, epoll_create, int, size)
Hi,
Timo Sirainen wrote:
Today I finally caught up with my mail. I've now read all the mails from the Dovecot list from this year and all my private mails from last few years (totaling around 500 unread mails).
If you think there's still something I should have replied to, please re-send it because I've either missed it or just skipped it for some reason. (Except there are still a couple of mails from this week I've yet to reply to.)
Besides reading mails I've also fixed a lot of bugs today. What's left from my original 1.0 roadmap is:
- Solaris sendfile is broken? Is it? Can someone try? You can test it with maildir by setting mail_save_crlf=yes, saving a somewhat large mail and FETCHing it (and truss that it really uses sendfile)
- LDA: empty mail gives an error.
- quota code should probably be always doing some quota_set_critical() instead of using mail_storage_set_critical(), so that quota_last_error() would work properly
- rfc2557 support for BODYSTRUCTURE, as specified by RFC3501
- keywords: add some limits to how many there can be
- don't return * in PERMANENTFLAGS when we're full
- mbox: how well does dirty sync + status work? it reads the last mail every time? not very good..
- mbox: dirty state should be stored to index (with mbox_very_dirty_syncs)
I think 1.0 could be released without any of those changes.
But what about the connection hangs that some people have had? Do they still occur? Is it only with SSL? I guess I should look at my previous ssl-proxy rewrite, figure out what went wrong with it and try again.
I'm thinking about releasing 1.0 RC1 within a few days. Would be nice if that release actually worked, so testing would be appreciated :) You can try it by getting the dovecot-latest from http://dovecot.org/nightly/
The current changes include:
- PAM: If user's password is expired, give "Password expired" error message to the user. Now actually working thanks to Vaidas Pilkauskas
- Relicensed dovecot-auth, lib-sql and lib-ntlm to MIT license. See COPYING file for more information.
- Abuse prevention: When creating a mailbox, limit the number of hierarchies (up to 20) and the length of the mailbox name within a hierarchy (up to 200 characters).
- mbox: If saved mail doesn't end with LF, add it ourself so that the mails always have one empty line before the next From-line.
- Added --with-statedir configure option which defaults to $localstatedir/lib/dovecot. ssl-parameters.dat is permanently stored in that directory and is copied to login_dirs from there.
- IMAP: Support SASL-IR extension (SASL initial response)
- Support initial SASL response with LOGIN mechanism. Patch by Anders Karlsson
- Added PLAIN-MD4 password scheme. Patch by Andrey Panin.
- Added support for XFS disk quotas. Patch by Pawel Jarosz
I compiled dovecot-20060618.tar.gz on Debian Sarge in order to test XFS disk quotas.
My config: protocols = imap imaps pop3 pop3s ssl_disable = yes disable_plaintext_auth = no log_timestamp = "%Y-%m-%d %H:%M:%S " mail_extra_groups = mail mail_debug = yes protocol imap { mail_plugins = quota imap_quota mail_plugin_dir = /usr/lib/dovecot/imap } protocol pop3 { pop3_uidl_format = %v.%u } auth_debug = yes auth default { mechanisms = plain passdb shadow { } userdb passwd { } user = root } plugin { quota = fs }
# lsmod|grep quota quota_v2 10240 0
# mount|grep sda9 /dev/sda9 on /home type xfs (rw,usrquota)
# cat /etc/mtab |grep sda9 /dev/sda9 /home xfs rw,usrquota 0 0
# quota -v dumitru
Disk quotas for user dumitru (uid 1048):
Filesystem blocks quota limit grace files quota
limit grace
/dev/sda9 557452 5242880 5451776 16427 0 0
There is no errors in logs regarding quota, but the problem is that server reports wrong quota info.
Request: A004 getquota "" Response:
- QUOTA "" (STORAGE 2286043136 20971520)
Getquota completed.
- If another process deletes the opened mailbox, try to handle it without writing errors to log file. Handles the most common cases.
- Added TLS support for LDAP if the library supports it.
- Dovecot corrupted mbox files which had CR+LF linefeeds in headers
- MySQL code could have crashed while escaping strings
- MD4 code with NTLM authentication was broken with 64bit systems. Patch by Andrey Panin
- Plugin loading was broken in some OSes (eg. FreeBSD)
- Several fixes to handling empty values in configuration file
- Several fixes to dictionary quota backend and dict server. Also changed how they're configured.
- deliver: Fixed plugin handling settings
- mbox_min_index_size handling was somewhat broken
- passdb passwd-file: extra_args field wasn't read unless the file was also used as userdb.
Regards, Dumitru
søn, 18,.06.2006 kl. 13.48 +0300, skrev Dumitru Negara:
# quota -v dumitru Disk quotas for user dumitru (uid 1048): Filesystem blocks quota limit grace files quota
limit grace /dev/sda9 557452 5242880 5451776 16427 0 0There is no errors in logs regarding quota, but the problem is that server reports wrong quota info.
Request: A004 getquota "" Response:
- QUOTA "" (STORAGE 2286043136 20971520)
Getquota completed.
I have always needed this patch for filesystem quota to show the right quota info. Also with the regular quota-code. But I have just tested the XFS-quota, and except for this old problem, it works perfect! Thanks! :)
Best regards, Stian
--- quota-fs.c.orig 2006-06-18 15:46:20.606456150 +0200 +++ quota-fs.c 2006-06-18 15:43:07.299881841 +0200 @@ -243,9 +243,9 @@ } #endif *value_r = (uint64_t)dqblk.dqb_curblocks *
(uint64_t)root->mount->blk_size / 1024;
(uint64_t)root->mount->blk_size / 1024 / 4096; *limit_r = (uint64_t)dqblk.dqb_bsoftlimit *
(uint64_t)root->mount->blk_size / 1024;
(uint64_t)root->mount->blk_size / 1024 / 4; return 1;
}
Hi,
Stian Jordet wrote:
søn, 18,.06.2006 kl. 13.48 +0300, skrev Dumitru Negara:
# quota -v dumitru Disk quotas for user dumitru (uid 1048): Filesystem blocks quota limit grace files quota limit grace /dev/sda9 557452 5242880 5451776 16427 0 0
There is no errors in logs regarding quota, but the problem is that server reports wrong quota info.
Request: A004 getquota "" Response:
- QUOTA "" (STORAGE 2286043136 20971520)
Getquota completed.
I have always needed this patch for filesystem quota to show the right quota info. Also with the regular quota-code. But I have just tested the XFS-quota, and except for this old problem, it works perfect! Thanks! :)
Best regards, Stian
--- quota-fs.c.orig 2006-06-18 15:46:20.606456150 +0200 +++ quota-fs.c 2006-06-18 15:43:07.299881841 +0200 @@ -243,9 +243,9 @@ } #endif *value_r = (uint64_t)dqblk.dqb_curblocks *
- (uint64_t)root->mount->blk_size / 1024;
- (uint64_t)root->mount->blk_size / 1024 / 4096;
*limit_r = (uint64_t)dqblk.dqb_bsoftlimit *
- (uint64_t)root->mount->blk_size / 1024;
- (uint64_t)root->mount->blk_size / 1024 / 4;
return 1; }
Thanks Stian,
Your patch did the trick. :)
# quota -v dumitru
Disk quotas for user dumitru (uid 1048):
Filesystem blocks quota limit grace files quota
limit grace
/dev/sda9 565448 5242880 5451776 16462 0 0
Request: A004 getquota "" Response:
- QUOTA "" (STORAGE 565448 5242880)
Getquota completed.
Timo, is it possible to add this patch to official code or it can break something in fs quota code?
I am very happy with working XFS fs quota in dovecot (thanks, Pawel), but just for the record:
- fs quota on ext3/reiserfs still doesn't work, same error message -
quotactl(Q_GETQUOTA, /dev/sdbx) failed: Invalid argument
Regards, Dumitru
On 169, 06 18, 2006 at 02:30:02AM +0300, Timo Sirainen wrote:
Today I finally caught up with my mail. I've now read all the mails from the Dovecot list from this year and all my private mails from last few years (totaling around 500 unread mails).
If you think there's still something I should have replied to, please re-send it because I've either missed it or just skipped it for some reason. (Except there are still a couple of mails from this week I've yet to reply to.)
Besides reading mails I've also fixed a lot of bugs today. What's left from my original 1.0 roadmap is:
- Solaris sendfile is broken? Is it? Can someone try? You can test it with maildir by setting mail_save_crlf=yes, saving a somewhat large mail and FETCHing it (and truss that it really uses sendfile)
- LDA: empty mail gives an error.
- quota code should probably be always doing some quota_set_critical() instead of using mail_storage_set_critical(), so that quota_last_error() would work properly
- rfc2557 support for BODYSTRUCTURE, as specified by RFC3501
- keywords: add some limits to how many there can be
- don't return * in PERMANENTFLAGS when we're full
- mbox: how well does dirty sync + status work? it reads the last mail every time? not very good..
- mbox: dirty state should be stored to index (with mbox_very_dirty_syncs)
I think 1.0 could be released without any of those changes.
But what about the connection hangs that some people have had? Do they still occur? Is it only with SSL? I guess I should look at my previous ssl-proxy rewrite, figure out what went wrong with it and try again.
I'm thinking about releasing 1.0 RC1 within a few days. Would be nice if that release actually worked, so testing would be appreciated :) You can try it by getting the dovecot-latest from http://dovecot.org/nightly/
Here is a big problem with transaction support in MySQL driver: it simpy doesn't work :( sql_transaction_commit_s() function fails and MySQL reports this error: You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near ';UPDATE ...
Looks like mysql_query() chokes if query contains semicolon.
IMHO we must just use MySQL BEGIN/COMMIT/ROLLBACK and let users care about transaction safety of their databases.
The current changes include:
- PAM: If user's password is expired, give "Password expired" error message to the user. Now actually working thanks to Vaidas Pilkauskas
- Relicensed dovecot-auth, lib-sql and lib-ntlm to MIT license. See COPYING file for more information.
- Abuse prevention: When creating a mailbox, limit the number of hierarchies (up to 20) and the length of the mailbox name within a hierarchy (up to 200 characters).
- mbox: If saved mail doesn't end with LF, add it ourself so that the mails always have one empty line before the next From-line.
- Added --with-statedir configure option which defaults to $localstatedir/lib/dovecot. ssl-parameters.dat is permanently stored in that directory and is copied to login_dirs from there.
- IMAP: Support SASL-IR extension (SASL initial response)
- Support initial SASL response with LOGIN mechanism. Patch by Anders Karlsson
- Added PLAIN-MD4 password scheme. Patch by Andrey Panin.
- Added support for XFS disk quotas. Patch by Pawel Jarosz
- If another process deletes the opened mailbox, try to handle it without writing errors to log file. Handles the most common cases.
- Added TLS support for LDAP if the library supports it.
- Dovecot corrupted mbox files which had CR+LF linefeeds in headers
- MySQL code could have crashed while escaping strings
- MD4 code with NTLM authentication was broken with 64bit systems. Patch by Andrey Panin
- Plugin loading was broken in some OSes (eg. FreeBSD)
- Several fixes to handling empty values in configuration file
- Several fixes to dictionary quota backend and dict server. Also changed how they're configured.
- deliver: Fixed plugin handling settings
- mbox_min_index_size handling was somewhat broken
- passdb passwd-file: extra_args field wasn't read unless the file was also used as userdb.
On Sun, 2006-06-18 at 19:59 +0400, Andrey Panin wrote:
Here is a big problem with transaction support in MySQL driver: it simpy doesn't work :( sql_transaction_commit_s() function fails and MySQL reports this error: You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near ';UPDATE ...
Looks like mysql_query() chokes if query contains semicolon.
Did you really test with the few days old code? Because I added this which fixed it for me:
#ifdef CLIENT_MULTI_STATEMENTS /* Updates require this because everything is committed in one large SQL statement. */ db->client_flags |= CLIENT_MULTI_STATEMENTS; #endif
IMHO we must just use MySQL BEGIN/COMMIT/ROLLBACK and let users care about transaction safety of their databases.
I don't think that's such a good idea, because if transactions aren't supported (and isn't that how MySQL is most commonly still used?) the rollback command does nothing and that can cause problems (like wrongly updating quota).
On 169, 06 18, 2006 at 07:31:05PM +0300, Timo Sirainen wrote:
On Sun, 2006-06-18 at 19:59 +0400, Andrey Panin wrote:
Here is a big problem with transaction support in MySQL driver: it simpy doesn't work :( sql_transaction_commit_s() function fails and MySQL reports this error: You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near ';UPDATE ...
Looks like mysql_query() chokes if query contains semicolon.
Did you really test with the few days old code? Because I added this which fixed it for me:
#ifdef CLIENT_MULTI_STATEMENTS /* Updates require this because everything is committed in one large SQL statement. */ db->client_flags |= CLIENT_MULTI_STATEMENTS; #endif
Yes, it works with MySQL >= 4.1. Seems like I need to upgrade MySQL on my home computer. But sql_update() still fails, because mysql_store_result() returns NULL if the query didn't return data. Of course UPDATE queries do not return data. Attached patch solved this problem for me, but I'm not sure how good it is for common use. Please take a look.
IMHO we must just use MySQL BEGIN/COMMIT/ROLLBACK and let users care about transaction safety of their databases.
I don't think that's such a good idea, because if transactions aren't supported (and isn't that how MySQL is most commonly still used?) the rollback command does nothing and that can cause problems (like wrongly updating quota).
Currently users with MySQL < 4.1 will get obscure SQL syntax error message and think "Where this fscking semicolon came from ?!". May be we should somehow notify user that update isn't actually supported...
On Thu, 2006-06-22 at 16:01 +0400, Andrey Panin wrote:
On 169, 06 18, 2006 at 07:31:05PM +0300, Timo Sirainen wrote:
On Sun, 2006-06-18 at 19:59 +0400, Andrey Panin wrote:
Here is a big problem with transaction support in MySQL driver: it simpy doesn't work :( sql_transaction_commit_s() function fails and MySQL reports this error: You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near ';UPDATE ...
Looks like mysql_query() chokes if query contains semicolon.
Did you really test with the few days old code? Because I added this which fixed it for me:
#ifdef CLIENT_MULTI_STATEMENTS /* Updates require this because everything is committed in one large SQL statement. */ db->client_flags |= CLIENT_MULTI_STATEMENTS; #endif
Yes, it works with MySQL >= 4.1. Seems like I need to upgrade MySQL on my home computer.
I thought earlier versions of MySQL allowed multi-statements by default.. Or is it maybe earlier versions of the server?
But sql_update() still fails, because mysql_store_result() returns NULL if the query didn't return data. Of course UPDATE queries do not return data. Attached patch solved this problem for me, but I'm not sure how good it is for common use. Please take a look.
Right. I fixed this a while ago also by checking mysql_errno() return values.
IMHO we must just use MySQL BEGIN/COMMIT/ROLLBACK and let users care about transaction safety of their databases.
I don't think that's such a good idea, because if transactions aren't supported (and isn't that how MySQL is most commonly still used?) the rollback command does nothing and that can cause problems (like wrongly updating quota).
Currently users with MySQL < 4.1 will get obscure SQL syntax error message and think "Where this fscking semicolon came from ?!". May be we should somehow notify user that update isn't actually supported...
The good thing is that currently the only place in Dovecot where SQL updates are done is the dict quota backend, which not many people are using. :) Would be nice to get proper error message about it, but I'm not really sure how to do it. Especially if the only broken combination is MySQL client < 4.1 + MySQL server > 4.1.
On 175, 06 24, 2006 at 09:35:22 +0300, Timo Sirainen wrote:
On Thu, 2006-06-22 at 16:01 +0400, Andrey Panin wrote:
On 169, 06 18, 2006 at 07:31:05PM +0300, Timo Sirainen wrote:
On Sun, 2006-06-18 at 19:59 +0400, Andrey Panin wrote:
Here is a big problem with transaction support in MySQL driver: it simpy doesn't work :( sql_transaction_commit_s() function fails and MySQL reports this error: You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near ';UPDATE ...
Looks like mysql_query() chokes if query contains semicolon.
Did you really test with the few days old code? Because I added this which fixed it for me:
#ifdef CLIENT_MULTI_STATEMENTS /* Updates require this because everything is committed in one large SQL statement. */ db->client_flags |= CLIENT_MULTI_STATEMENTS; #endif
Yes, it works with MySQL >= 4.1. Seems like I need to upgrade MySQL on my home computer.
I thought earlier versions of MySQL allowed multi-statements by default.. Or is it maybe earlier versions of the server?
Earlier versions of the server do not work. Tested with MySQL client 5.0 and MySQL server 4.0.
But sql_update() still fails, because mysql_store_result() returns NULL if the query didn't return data. Of course UPDATE queries do not return data. Attached patch solved this problem for me, but I'm not sure how good it is for common use. Please take a look.
Right. I fixed this a while ago also by checking mysql_errno() return values.
Unfortunately I don't think it's fixed :(
IMHO we must just use MySQL BEGIN/COMMIT/ROLLBACK and let users care about transaction safety of their databases.
I don't think that's such a good idea, because if transactions aren't supported (and isn't that how MySQL is most commonly still used?) the rollback command does nothing and that can cause problems (like wrongly updating quota).
Currently users with MySQL < 4.1 will get obscure SQL syntax error message and think "Where this fscking semicolon came from ?!". May be we should somehow notify user that update isn't actually supported...
The good thing is that currently the only place in Dovecot where SQL updates are done is the dict quota backend, which not many people are using. :)
I'm working on the second user :)
Would be nice to get proper error message about it, but I'm not really sure how to do it. Especially if the only broken combination is MySQL client < 4.1 + MySQL server > 4.1.
Timo Sirainen wrote:
If you think there's still something I should have replied to, please re-send it because I've either missed it or just skipped it for some reason. (Except there are still a couple of mails from this week I've yet to reply to.)
beta 9 still has a permissions problem when a user creates a new folder in a public namespace. The file dovecot-shared is used to assign permissions and ownership for new files, but not for new directories.
In the example below, the public directory .myTest was created by user mark. Other users cannot read it. All users are in the group Everyone. These are real system users.
[root@tesla Maildir]# ls -la /home/public/Maildir/ total 103k drwxrws--- 11 dovecot Everyone 4.1k Jun 20 11:17 ./ drwxrwxrwx 3 root root 4.1k Jun 7 09:15 ../ drwxrws--- 2 dovecot Everyone 13k May 24 14:43 cur/ -rw-rw---- 1 dovecot Everyone 0 May 12 13:46 dovecot-shared drwx--S--- 5 mark Everyone 4.1k Jun 20 11:17 .myTest/ drwxrws--- 2 dovecot Everyone 4.1k May 24 14:44 new/ drwxrws--- 2 dovecot Everyone 4.1k May 19 17:48 tmp/
Note that a message moved into the .myTest directory has correct permissions
[root@tesla Maildir]# ls -la /home/public/Maildir/.myTest/cur/ total 259k drwx--S--- 2 mark Everyone 4.1k Jun 20 11:19 ./ drwx--S--- 5 mark Everyone 4.1k Jun 20 11:17 ../ -rw-rw---- 1 mark Everyone 234k Jun 16 04:02 11508.P2Q3.my.example.com:2,S
Workaround is to set umask=0007 in dovecot.conf, but this gives loose permissions on private mailboxes.
Mark
On Tue, 2006-06-20 at 11:33 -0700, Mark Nienberg wrote:
Timo Sirainen wrote:
If you think there's still something I should have replied to, please re-send it because I've either missed it or just skipped it for some reason. (Except there are still a couple of mails from this week I've yet to reply to.)
beta 9 still has a permissions problem when a user creates a new folder in a public namespace. The file dovecot-shared is used to assign permissions and ownership for new files, but not for new directories.
In the example below, the public directory .myTest was created by user mark. Other users cannot read it. All users are in the group Everyone. These are real system users.
[root@tesla Maildir]# ls -la /home/public/Maildir/ total 103k drwxrws--- 11 dovecot Everyone 4.1k Jun 20 11:17 ./ drwxrwxrwx 3 root root 4.1k Jun 7 09:15 ../ drwxrws--- 2 dovecot Everyone 13k May 24 14:43 cur/ -rw-rw---- 1 dovecot Everyone 0 May 12 13:46 dovecot-shared drwx--S--- 5 mark Everyone 4.1k Jun 20 11:17 .myTest/ drwxrws--- 2 dovecot Everyone 4.1k May 24 14:44 new/ drwxrws--- 2 dovecot Everyone 4.1k May 19 17:48 tmp/
Each created folder is supposed to have their own dovecot-shared file. Having the dovecot-shared in here makes it used only for INBOX, nothing else.
Also if you create new folders the dovecot-shared file isn't copied or anything else. So you always have to create it manually for each shared folder..
I don't think I'm going to change this behavior before writing proper shared folder support, sometimes after v1.0.
Timo Sirainen wrote:
On Tue, 2006-06-20 at 11:33 -0700, Mark Nienberg wrote:
beta 9 still has a permissions problem when a user creates a new folder in a public namespace. The file dovecot-shared is used to assign permissions and ownership for new files, but not for new directories.
In the example below, the public directory .myTest was created by user mark. Other users cannot read it. All users are in the group Everyone. These are real system users.
[root@tesla Maildir]# ls -la /home/public/Maildir/ total 103k drwxrws--- 11 dovecot Everyone 4.1k Jun 20 11:17 ./ drwxrwxrwx 3 root root 4.1k Jun 7 09:15 ../ drwxrws--- 2 dovecot Everyone 13k May 24 14:43 cur/ -rw-rw---- 1 dovecot Everyone 0 May 12 13:46 dovecot-shared drwx--S--- 5 mark Everyone 4.1k Jun 20 11:17 .myTest/ drwxrws--- 2 dovecot Everyone 4.1k May 24 14:44 new/ drwxrws--- 2 dovecot Everyone 4.1k May 19 17:48 tmp/
Each created folder is supposed to have their own dovecot-shared file. Having the dovecot-shared in here makes it used only for INBOX, nothing else.
The permission problem is for the new folder itself (.myTest in this example). It should be created with group read/write privileges since it is in a public namespace. Alternatively, it should be created with group privileges of the dovecot-shared folder.
Also if you create new folders the dovecot-shared file isn't copied or anything else. So you always have to create it manually for each shared folder..
With respect, you are wrong about that. Dovecot does create a copy of dovecot-shared in the new folder. In the example above, inside the .myTest directory is another dovecot-shared file, and I didn't create it, dovecot did! The administrator need only create dovecot-shared once in the parent folder. All new subfolders created by users inherit a copy. It already works.
I don't think I'm going to change this behavior before writing proper shared folder support, sometimes after v1.0.
I'm not sure what you mean by proper support. All I really need is something equivalent to UW-IMAP public folders. It seems to me dovecot already does that and has the added improvement of allowing each user to maintain his own index file. The only problem I've seen is the minor permissions issue described above. I'm still testing, with plans to convert our production server next weekend.
Mark
On Jun 26, 2006, at 8:13 PM, Mark Nienberg wrote:
Also if you create new folders the dovecot-shared file isn't
copied or anything else. So you always have to create it manually for each
shared folder..With respect, you are wrong about that. Dovecot does create a copy
of dovecot-shared in the new folder. In the example above, inside
the .myTest directory is another dovecot-shared file, and I didn't
create it, dovecot did! The administrator need only create dovecot- shared once in the parent folder. All new subfolders created by
users inherit a copy. It already works.
Oh. I don't remember adding such feature there, but looks like it is
there.. Well, I guess it'll have to be fixed then. Fixed in CVS now.
On Sun, 2006-06-18 at 02:30 +0300, Timo Sirainen wrote:
Today I finally caught up with my mail. I've now read all the mails from the Dovecot list from this year and all my private mails from last few years (totaling around 500 unread mails).
If you think there's still something I should have replied to, please re-send it because I've either missed it or just skipped it for some reason. (Except there are still a couple of mails from this week I've yet to reply to.)
Not sure if I sent this before, but here it goes again:
We are having a lot of problems with Outlook 2003 (and, but I couldn't verify this) Outlook 2000 when using dovecot + IMAP. Outlook 2002 works like a charm.
The problem is best described as totally random disconnects. Hitting Send/Receive a couple of times usually makes it work again, sometimes not.
Does anyone experience the same issues, or is it just us again? :) For now, and I'm totally guessing here, I think that 2003 and 2000 expect some wacky NOOP behavior.
Cheers, Marc
On Wed, 2006-06-21 at 11:09 +0200, J.M. Maurer wrote:
We are having a lot of problems with Outlook 2003 (and, but I couldn't verify this) Outlook 2000 when using dovecot + IMAP. Outlook 2002 works like a charm.
The problem is best described as totally random disconnects. Hitting Send/Receive a couple of times usually makes it work again, sometimes not.
Do you use SSL? If yes, can you test if this happens without SSL?
If it's not because of SSL, then I've no idea :)
On Wed, 2006-06-21 at 12:21 +0300, Timo Sirainen wrote:
On Wed, 2006-06-21 at 11:09 +0200, J.M. Maurer wrote:
We are having a lot of problems with Outlook 2003 (and, but I couldn't verify this) Outlook 2000 when using dovecot + IMAP. Outlook 2002 works like a charm.
The problem is best described as totally random disconnects. Hitting Send/Receive a couple of times usually makes it work again, sometimes not.
Do you use SSL?
Yes I did
If yes, can you test if this happens without SSL?
I can reproduce it with SSL as well
If it's not because of SSL, then I've no idea :)
Any pointers how I could debug this? I can invest some time in it.
Marc
On Wed, 2006-06-21 at 11:39 +0200, J.M. Maurer wrote:
On Wed, 2006-06-21 at 12:21 +0300, Timo Sirainen wrote:
On Wed, 2006-06-21 at 11:09 +0200, J.M. Maurer wrote:
We are having a lot of problems with Outlook 2003 (and, but I couldn't verify this) Outlook 2000 when using dovecot + IMAP. Outlook 2002 works like a charm.
The problem is best described as totally random disconnects. Hitting Send/Receive a couple of times usually makes it work again, sometimes not.
Do you use SSL?
Yes I did
If yes, can you test if this happens without SSL?
I can reproduce it with SSL as well
s/with/without
Marc
On Wednesday 21 June 2006 02:40, J.M. Maurer wrote:
On Wed, 2006-06-21 at 11:39 +0200, J.M. Maurer wrote:
On Wed, 2006-06-21 at 12:21 +0300, Timo Sirainen wrote:
On Wed, 2006-06-21 at 11:09 +0200, J.M. Maurer wrote:
We are having a lot of problems with Outlook 2003 (and, but I couldn't verify this) Outlook 2000 when using dovecot + IMAP. Outlook 2002 works like a charm.
The problem is best described as totally random disconnects. Hitting Send/Receive a couple of times usually makes it work again, sometimes not.
Do you use SSL?
Yes I did
If yes, can you test if this happens without SSL?
I can reproduce it with SSL as well
s/with/without
Marc
Did you look at the Wiki?
http://wiki.dovecot.org/Clients
I don't know if your problems are related/known...
If yes, can you test if this happens without SSL?
I can reproduce it with SSL as well
s/with/without
Marc
Did you look at the Wiki?
http://wiki.dovecot.org/Clients
I don't know if your problems are related/known...
Yes I did. None of the issues described there looks like our problem (I tried the Outlook specific workarounds as well, just to be sure, but they only solve the problem that they are meant to solve :)
We've seen this issue from the beginnings of the 1.0 series, and iirc in the 0.99 series as well.
Marc
On Wed, 2006-06-21 at 11:39 +0200, J.M. Maurer wrote:
We are having a lot of problems with Outlook 2003 (and, but I couldn't verify this) Outlook 2000 when using dovecot + IMAP. Outlook 2002 works like a charm.
The problem is best described as totally random disconnects. Hitting Send/Receive a couple of times usually makes it work again, sometimes not.
If it's not because of SSL, then I've no idea :)
Any pointers how I could debug this? I can invest some time in it.
I guess the first thing would be to start using Dovecot's rawlog or some other means of getting traffic dumps. Then see what are the last things that the client sent and what it got as a return.
If it's not because of SSL, then I've no idea :)
Any pointers how I could debug this? I can invest some time in it.
I guess the first thing would be to start using Dovecot's rawlog or some other means of getting traffic dumps. Then see what are the last things that the client sent and what it got as a return.
Attached 2 different rawlogs that lead the the hang/disconnect.
- The short one was just 1) Open Outlook 2) Hit send/receive.
- The long one was 1) Open Outlook 2) Hit send/receive a couple times
I'm rather clueless about the imap protocol atm, so any insights will be appreciated.
Marc
On Mon, 2006-06-26 at 11:07 +0200, Marc Maurer wrote:
If it's not because of SSL, then I've no idea :)
Any pointers how I could debug this? I can invest some time in it.
I guess the first thing would be to start using Dovecot's rawlog or some other means of getting traffic dumps. Then see what are the last things that the client sent and what it got as a return.
Attached 2 different rawlogs that lead the the hang/disconnect.
- The short one was just 1) Open Outlook 2) Hit send/receive.
- The long one was 1) Open Outlook 2) Hit send/receive a couple times
I'm rather clueless about the imap protocol atm, so any insights will be appreciated.
Well, a few things:
You should remove .subscriptions and .customflags files. They're no longer used with 1.0betas and they're giving some errors because they're treated as maildirs. But I don't think that causes the hangs.
Second problem is:
- 315 FETCH (UID 6052 FLAGS (\Seen))
316 FETCH (UID 6094 FLAGS (\Seen))
- 317 FETCH (UID 6116 FLAGS (\Seen))
Is the '*' really missing from the file? I thought all such problems were fixed already. And the shorter rawlog didn't contain anything like that.
Anyway both connections are properly ended with LOGOUT command and they receive reply for it, so I don't think it hanged because of the IMAP connection itself..
The command tags look a bit weird though. Does Outlook really use those? Or do you have some kind of firewall or proxy in the middle?
- The short one was just 1) Open Outlook 2) Hit send/receive.
- The long one was 1) Open Outlook 2) Hit send/receive a couple times
I'm rather clueless about the imap protocol atm, so any insights will be appreciated.
Well, a few things:
You should remove .subscriptions and .customflags files. They're no longer used with 1.0betas and they're giving some errors because they're treated as maildirs. But I don't think that causes the hangs.
Will do
Second problem is:
- 315 FETCH (UID 6052 FLAGS (\Seen))
316 FETCH (UID 6094 FLAGS (\Seen))
- 317 FETCH (UID 6116 FLAGS (\Seen))
Is the '*' really missing from the file? I thought all such problems were fixed already. And the shorter rawlog didn't contain anything like that.
Erm, i think your mail client got confused, as I see it in the files just fine. [1]
Anyway both connections are properly ended with LOGOUT command and they receive reply for it, so I don't think it hanged because of the IMAP connection itself..
The command tags look a bit weird though. Does Outlook really use those? Or do you have some kind of firewall or proxy in the middle?
Yep, it really does send those command tags. No firewall, no proxy... a direct connection from the Win32 machine to the Linux machine.
Thanks, Marc
On Tue, 2006-06-27 at 13:16 +0200, J.M. Maurer wrote:
Second problem is:
- 315 FETCH (UID 6052 FLAGS (\Seen))
316 FETCH (UID 6094 FLAGS (\Seen))
- 317 FETCH (UID 6116 FLAGS (\Seen))
Is the '*' really missing from the file? I thought all such problems were fixed already. And the shorter rawlog didn't contain anything like that.
Erm, i think your mail client got confused, as I see it in the files just fine. [1]
No, even more stupid. I opened it with a text editor and accidentally deleted it myself.. :)
Anyway both connections are properly ended with LOGOUT command and they receive reply for it, so I don't think it hanged because of the IMAP connection itself..
The command tags look a bit weird though. Does Outlook really use those? Or do you have some kind of firewall or proxy in the middle?
Yep, it really does send those command tags. No firewall, no proxy... a direct connection from the Win32 machine to the Linux machine.
Well, I can't really think of anything then. Usually the hangs have been because the connection hangs for some reason, like with large FETCH replies. But there was nothing like that in your case. The hangs seem to happen after Outlook has closed the IMAP connection, so maybe it doesn't like some mail or something..
Erm, i think your mail client got confused, as I see it in the files just fine. [1]
No, even more stupid. I opened it with a text editor and accidentally deleted it myself.. :)
:-P
The command tags look a bit weird though. Does Outlook really use those? Or do you have some kind of firewall or proxy in the middle?
Yep, it really does send those command tags. No firewall, no proxy... a direct connection from the Win32 machine to the Linux machine.
Well, I can't really think of anything then. Usually the hangs have been because the connection hangs for some reason, like with large FETCH replies. But there was nothing like that in your case. The hangs seem to happen after Outlook has closed the IMAP connection, so maybe it doesn't like some mail or something..
Hmm, that would be strange, as it happens with all our Outlook 2003 clients.. then again, you never know how braindead Outlook is, and which mails it can't handle.
Btw, the disconnect only came after a long (ie. 1 min) wait while doing send/receive. When the disconnect came in the rawlog, Outlook still kept showing the progress bar, waiting forever.
Marc
On Tue, 2006-06-27 at 14:21 +0200, J.M. Maurer wrote:
Well, I can't really think of anything then. Usually the hangs have been because the connection hangs for some reason, like with large FETCH replies. But there was nothing like that in your case. The hangs seem to happen after Outlook has closed the IMAP connection, so maybe it doesn't like some mail or something..
Hmm, that would be strange, as it happens with all our Outlook 2003 clients.. then again, you never know how braindead Outlook is, and which mails it can't handle.
Btw, the disconnect only came after a long (ie. 1 min) wait while doing send/receive. When the disconnect came in the rawlog, Outlook still kept showing the progress bar, waiting forever.
Weird. Is it possible for you to look at the IMAP traffic from the client's computer? Or possibly some computer between the server and the client? Maybe it doesn't see the logout reply for some reason. I don't know what tools to use in Windows to do that though.
Windows 2000 server has a free network monitor tool...
On Wed, 2006-06-28 at 00:34 +0300, Timo Sirainen wrote:
On Tue, 2006-06-27 at 14:21 +0200, J.M. Maurer wrote:
Well, I can't really think of anything then. Usually the hangs have been because the connection hangs for some reason, like with large FETCH replies. But there was nothing like that in your case. The hangs seem to happen after Outlook has closed the IMAP connection, so maybe it doesn't like some mail or something..
Hmm, that would be strange, as it happens with all our Outlook 2003 clients.. then again, you never know how braindead Outlook is, and which mails it can't handle.
Btw, the disconnect only came after a long (ie. 1 min) wait while doing send/receive. When the disconnect came in the rawlog, Outlook still kept showing the progress bar, waiting forever.
Weird. Is it possible for you to look at the IMAP traffic from the client's computer? Or possibly some computer between the server and the client? Maybe it doesn't see the logout reply for some reason. I don't know what tools to use in Windows to do that though.
Sure, ethereal works just fine on Win32 it seems :)
Attached is a traffic log taken from the windows client (192.168.2.25), connecting to the dovecot server (192.168.2.2).
There are 2 'issues' here: the first is, again, a very long wait during a "send/receive" action done from Outlook 2003. After some time, this action finished or timed-out, not sure there. Secondly, it ends with the same issue, where Outlook 2003 never came through, and just kept hanging (showing the progress bar).
If you need any more info, please ask. We are very eager to track this issue down (as it affects a lot of our clients).
Thanks, Marc
Timo Sirainen wrote: [snip]
I'm thinking about releasing 1.0 RC1 within a few days. Would be nice if that release actually worked, so testing would be appreciated :) You can try it by getting the dovecot-latest from http://dovecot.org/nightly/
There was a change to how configure detects if kqueue is supported (made sometime after the 10th (I suspect the change on the 16th, but haven't looked too closely) which prevents it from being detected on NetBSD. Forcing kqueue to be used for file change and i/o loop by hacking up the configure script allows it to be used.
It also works correctly, save for the messages:
Jun 21 17:22:13 intrepid dovecot: auth(default): kevent(10) in io_loop_handle_remove failed: Bad file descriptor Jun 21 17:22:13 intrepid dovecot: auth(default): kevent(12) in io_loop_handle_remove failed: Bad file descriptor Jun 21 17:22:13 intrepid dovecot: auth(default): kevent(11) in io_loop_handle_remove failed: Bad file descriptor
This doesn't appear to cause any bad side effects though, as logging in and mail notification both work just fine.
ScottE
On Wed, 2006-06-21 at 17:23 -0700, Scott Ellis wrote:
Timo Sirainen wrote: [snip]
I'm thinking about releasing 1.0 RC1 within a few days. Would be nice if that release actually worked, so testing would be appreciated :) You can try it by getting the dovecot-latest from http://dovecot.org/nightly/
There was a change to how configure detects if kqueue is supported (made sometime after the 10th (I suspect the change on the 16th, but haven't looked too closely) which prevents it from being detected on NetBSD. Forcing kqueue to be used for file change and i/o loop by hacking up the configure script allows it to be used.
Any idea what exactly? I noticed one test had == instead of =. Was it just because of that? http://dovecot.org/list/dovecot-cvs/2006-June/005920.html
Jun 21 17:22:13 intrepid dovecot: auth(default): kevent(10) in io_loop_handle_remove failed: Bad file descriptor
Well, these aren't good anyway.. Would be nice to know where those came from. If you want you could change i_error calls to i_panic in src/lib/ioloop-kqueue.c and then get gdb backtraces when it crashes (http://dovecot.org/bugreport.html)
Timo Sirainen wrote:
On Wed, 2006-06-21 at 17:23 -0700, Scott Ellis wrote:
Timo Sirainen wrote: [snip]
I'm thinking about releasing 1.0 RC1 within a few days. Would be nice if that release actually worked, so testing would be appreciated :) You can try it by getting the dovecot-latest from http://dovecot.org/nightly/
There was a change to how configure detects if kqueue is supported (made sometime after the 10th (I suspect the change on the 16th, but haven't looked too closely) which prevents it from being detected on NetBSD. Forcing kqueue to be used for file change and i/o loop by hacking up the configure script allows it to be used.
Any idea what exactly? I noticed one test had == instead of =. Was it just because of that? http://dovecot.org/list/dovecot-cvs/2006-June/005920.html
It appears so. It now properly detects and uses kqueue for ioloop, but not for notify. Changing the configure test for notify makes it work there as well:
--- configure 2006-06-26 10:59:49.000000000 -0700 +++ configure.orig 2006-06-24 14:33:51.000000000 -0700 @@ -22526,7 +22526,7 @@ if test "$notify" = "" || test "$notify" = "kqueue"; then echo "$as_me:$LINENO: checking if we can use BSD kqueue() notify" >&5 echo $ECHO_N "checking if we can use BSD kqueue() notify... $ECHO_C" >&6
- if test "$ac_cv_func_kqueue" = yes && test "$ac_cv_func_kevent" =
yes ; then
- if test "$ac_cv_func_kqueue" == yes && test "$ac_cv_func_kevent" ==
yes ; then have_notify=kqueue notify=kqueue echo "$as_me:$LINENO: result: "yes"" >&5
Jun 21 17:22:13 intrepid dovecot: auth(default): kevent(10) in io_loop_handle_remove failed: Bad file descriptor
Well, these aren't good anyway.. Would be nice to know where those came from. If you want you could change i_error calls to i_panic in src/lib/ioloop-kqueue.c and then get gdb backtraces when it crashes (http://dovecot.org/bugreport.html)
This doesn't yield anything useful. There's nothing before the kill() in the backtrace of dovecot-auth, even though the binary has symbols in it:
intrepid# !file file /software/dovecot-1.0-20060626/libexec/dovecot/dovecot-auth /software/dovecot-1.0-20060626/libexec/dovecot/dovecot-auth: ELF 64-bit LSB executable, AMD x86-64, version 1 (SYSV), for NetBSD 3.99.21, dynamically linked (uses shared libs), for NetBSD 3.99.21, not stripped
I suspect I should really rebuild it w/o any optimization flags, just to make sure that's not the issue with the backtrace (there's also the potential that gdb isn't working quite right either, as this is NetBSD-current, and GDB6 was recently introduced...I don't know if 5.3 has works with GCC4 binaries or not...). I'll try and dig around a bit more on this.
ScottE
Scott Ellis wrote:
Timo Sirainen wrote:
On Wed, 2006-06-21 at 17:23 -0700, Scott Ellis wrote:
[snip]
Well, these aren't good anyway.. Would be nice to know where those came from. If you want you could change i_error calls to i_panic in src/lib/ioloop-kqueue.c and then get gdb backtraces when it crashes (http://dovecot.org/bugreport.html)
[snip] Okay, I updated to GBD6.5 and now get something more useful, but still pretty useless. :-)
Maillog reports: Jul 10 15:57:44 intrepid dovecot: Dovecot v1.0.rc2 starting up Jul 10 15:57:51 intrepid dovecot: auth(default): kevent(10) in io_loop_handle_re move failed: Bad file descriptor Jul 10 15:57:51 intrepid dovecot: child 27097 (auth) killed with signal 6 Jul 10 15:57:52 intrepid dovecot: auth(default): kevent(11) in io_loop_handle_re move failed: Bad file descriptor Jul 10 15:57:52 intrepid dovecot: child 16934 (auth) killed with signal 6
"gdb /software/dovecot-1.0rc2.debug/libexec/dovecot/dovecot-auth /var/run/dovecot/dovecot-auth.core" yields:
(gdb) bt full #0 0x00007f7ffda2bc1e in kill () from /usr/lib/libc.so.12 No symbol table info available. #1 0x00007f7ffdab8529 in abort () from /usr/lib/libc.so.12 No symbol table info available. #2 0x0000000000419639 in i_internal_panic_handler () No symbol table info available. #3 0x000000000041996c in i_panic () No symbol table info available. #4 0x000000000041ced7 in io_loop_handle_remove () No symbol table info available. #5 0x000000000041c22c in io_remove () No symbol table info available. #6 0x00000000004135ec in checkpassword_request_close () No symbol table info available. #7 0x0000000000413b3f in checkpassword_child_input () No symbol table info available. #8 0x000000000041cd35 in io_loop_handler_run () No symbol table info available. #9 0x000000000041c1c8 in io_loop_run () No symbol table info available. #10 0x000000000040f0f4 in main () No symbol table info available.
intrepid# file /software/dovecot-1.0rc2.debug/libexec/dovecot/dovecot-auth /software/dovecot-1.0rc2.debug/libexec/dovecot/dovecot-auth: ELF 64-bit LSB executable, AMD x86-64, version 1 (SYSV), for NetBSD 3.99.21, dynamically linked (uses shared libs), for NetBSD 3.99.21, not stripped
Any other way you want me to try and debug this? I'm ignorant in regards to kevent. I do note that the kevent manpage on NetBSD notes:
EV_DELETE Removes the event from the kqueue. Events
which are attached to file descriptors are automatically deleted on the last close of the descriptor.
Maybe dovecot-auth is closing the fh before calling io_loop_handle_remove()?
ScottE
On Mon, 2006-07-10 at 16:07 -0700, Scott Ellis wrote:
Jul 10 15:57:51 intrepid dovecot: auth(default): kevent(10) in io_loop_handle_re move failed: Bad file descriptor
..
#5 0x000000000041c22c in io_remove () No symbol table info available. #6 0x00000000004135ec in checkpassword_request_close () No symbol table info available.
..
Maybe dovecot-auth is closing the fh before calling io_loop_handle_remove()?
Yes. This has been a problem with using kqueue and I've tried to get rid of all of these cases, but there are still a few of them left. This is anyway fixed in rc6.
Hi Timo, I copy my email to this thread, so its more organized:
http://dovecot.org/list/dovecot/2006-June/014205.html
Thanks Oliver
Timo Sirainen wrote:
If you think there's still something I should have replied to, please re-send it because I've either missed it or just skipped it for some reason. (Except there are still a couple of mails from this week I've yet to reply to.)
participants (13)
-
Andrey Panin
-
Dominic Lepiane
-
Dumitru Negara
-
J.M. Maurer
-
Jakob Hirsch
-
Jan Kundrát
-
Jeff Graves
-
Marc Maurer
-
Mark Nienberg
-
Oliver Schulze L.
-
Scott Ellis
-
Stian Jordet
-
Timo Sirainen