As promised, here it is finally.
v1.0 is getting nearer and I want people to start looking at Dovecot's sources. Getting some more help with development would be nice, but I'd also like to hear if there are any hidden bugs in the code. So, one week from now I'll start offering 1000 EUR for the first person to point out a remotely exploitable security hole in Dovecot. See http://dovecot.org/security.html for more information.
Changes since 1.0alpha5:
UPGRADE WARNING: Unlike earlier Dovecot versions, 1.0 beta1 now requires Diffie-Hellman parameters to have been generated before SSL/TLS is used. These are generated when Dovecot is started for the first time and it may take several minutes. During this time SSL/TLS connections aren't working.
(I decided to leave it this way at least for now. I don't think any new Dovecot admins will try to log in with SSL within the first 5 minutes of the installation?)
Features:
- DSA SSL connections provide now forward secrecy (which causes the DH parameter issue mentioned above)
- Support for password protected SSL private keys
- LDAP: Added authentication bind support. Patch by J.M. Maurer
- LDAP: Added fast authentication binding. Patch by Geff <boing@boing.com>
- PAM: Changed -session parameter to session=yes
- kqueue support. Patch by Vaclav Haisman
- dbox updates. Kind of works now, but not recommended for more than testing yet. There still are known bugs.
- maildir_copy_with_hardlinks=yes works again
- Sometimes Dovecot kept telling about keyword changes over and over again even if there weren't any.
- Beginnings of fallbacking to in-memory indexes when write fails with "out of disk space" error. Not perfect yet, but helps in some cases.
- When IDLEing, send a small notification every 2 minutes so that NATs/firewalls don't close the connection.
- Added gdbhelper binary to help debugging
Performance improvements:
- Don't fdatasync() cache file. Not that important and gives better performance.
- mmap_disable=yes mode doesn't keep rewriting index file now constantly, so it uses much less disk I/O.
- Some other minor index file improvements
Bugfixes:
- UID STORE command wasn't returning UIDs in replies
- Fixes for various problems with IDLE command.
- THREAD command crashed if no search matches were found
- Some fixes to handling LIST command with namespaces. Helps Pine.
- Fixed a situation in mbox when it could get the process to infinite loop
- Fixed one mbox assert crash
- Use long line wrapping for X-IMAPbase, X-IMAP and X-Keywords headers for UW-IMAP compatibility
- dovecot --exec-mail wasn't cleaning existing environment variables properly
That's silly. The first thing I try with any new install is test it out. The idea of generating the DH key during install is useful.
On Mon, 16 Jan 2006 22:52:30 +0200 Timo Sirainen <tss@iki.fi> wrote:
: (I decided to leave it this way at least for now. I don't think any : new Dovecot admins will try to log in with SSL within the first 5 : minutes of the installation?)
-- All the taxes paid over a lifetime by the average American are spent by the government in less than a second. -- Jim Fiebig
On Mon, 2006-01-16 at 13:35 -0800, Peter Hessler wrote:
That's silly. The first thing I try with any new install is test it out. The idea of generating the DH key during install is useful.
Well, the problem with this is that there's no standard way to "install Dovecot". Each binary package builder would have to do this separately. For people installing from sources there would have to be some special command for doing this and instructions for doing this.
Unless you mean that it should work exactly as now, except Dovecot shouldn't be left running in background before the DH parameters are generated? I don't see much difference here. And it'd need more code :)
On Mon, Jan 16, 2006 at 11:40:22PM +0200, Timo Sirainen wrote:
On Mon, 2006-01-16 at 13:35 -0800, Peter Hessler wrote:
That's silly. The first thing I try with any new install is test it out. The idea of generating the DH key during install is useful.
Well, the problem with this is that there's no standard way to "install Dovecot". Each binary package builder would have to do this separately. For people installing from sources there would have to be some special command for doing this and instructions for doing this.
Unless you mean that it should work exactly as now, except Dovecot shouldn't be left running in background before the DH parameters are generated? I don't see much difference here. And it'd need more code :)
Could you at least print something in the logs letting the admin know when the key generation has started and stopped? Depending on the speed of the system this could take anywhere from a minute to half an hour.
Brad <brad@comstyle.com> writes:
Could you at least print something in the logs letting the admin know when the key generation has started and stopped? Depending on the speed of the system this could take anywhere from a minute to half an hour.
Jan 16 22:41:31 violet dovecot: Generating Diffie-Hellman parameters for the first time. This may take a while.. [ ... ] Jan 16 22:41:55 violet dovecot: SSL parameters regeneration completed
-- Gabriel
On Tue, Jan 17, 2006 at 12:01:32AM +0200, Gabriel Kihlman wrote:
Brad <brad@comstyle.com> writes:
Could you at least print something in the logs letting the admin know when the key generation has started and stopped? Depending on the speed of the system this could take anywhere from a minute to half an hour.
Jan 16 22:41:31 violet dovecot: Generating Diffie-Hellman parameters for the first time. This may take a while.. [ ... ] Jan 16 22:41:55 violet dovecot: SSL parameters regeneration completed
-- Gabriel
I do not use SSL and it was only an idea I had. Glad to see that it already does this.
On 17/01/2006 10:40 a.m., Timo Sirainen wrote:
On Mon, 2006-01-16 at 13:35 -0800, Peter Hessler wrote:
That's silly. The first thing I try with any new install is test it out. The idea of generating the DH key during install is useful.
Well, the problem with this is that there's no standard way to "install Dovecot". Each binary package builder would have to do this separately. For people installing from sources there would have to be some special command for doing this and instructions for doing this.
Unless you mean that it should work exactly as now, except Dovecot shouldn't be left running in background before the DH parameters are generated? I don't see much difference here. And it'd need more code :)
As long as it is documented in the INSTALL file and/or WIKI then anyone who really cares about it will soon find out.
People do read those files, don't they? ;-)
It would be good if the dovecot master process can be started with a flag such as "dovecot -k", a DH key generated (only) and then the master process just die. This would allow things like RPM post scripts to generate the key at package install time before the package properly starts up. squid -z does this sort of thing for Squid too.
On the other hand:
Jan 17 10:52:55 tornado dovecot: Dovecot v1.0.beta1 starting up Jan 17 10:52:55 tornado dovecot: Generating Diffie-Hellman parameters for the first time. This may take a while.. Jan 17 10:53:00 tornado dovecot: SSL parameters regeneration completed
That's about 5s to generate the keys on my P4-2.8Ghz. Which hardly justifies an hour's worth of coding to let people know about a 5s once-off delay.
reuben
I would think it could done this way, see if the installer did it, if so skip it on first-time start up, otherwise do it on first-time startup.
John
Timo Sirainen wrote:
On Mon, 2006-01-16 at 13:35 -0800, Peter Hessler wrote:
That's silly. The first thing I try with any new install is test it out. The idea of generating the DH key during install is useful.
Well, the problem with this is that there's no standard way to "install Dovecot". Each binary package builder would have to do this separately. For people installing from sources there would have to be some special command for doing this and instructions for doing this.
Unless you mean that it should work exactly as now, except Dovecot shouldn't be left running in background before the DH parameters are generated? I don't see much difference here. And it'd need more code :)
Timo Sirainen wrote:
As promised, here it is finally.
Not to nudge about it, but I wonder if you looked at my mail of yesterday regarding PAM_RHOST... I now downloaded the beta and still had to apply that patch to get the client host to be passed to PAM as PAM_RHOST on FreeBSD 5.4.
Great news beside that. I think one of the bugs I previously noticed with IDLE (which I found difficult to point at since it manifested itself with Thunderbird which opens several connections to the server) is gone.
Cheers, -- Tom
-- Tom Alsberg - hacker (being the best description fitting this space) Web page: http://www.cs.huji.ac.il/~alsbergt/ DISCLAIMER: The above message does not even necessarily represent what my fingers have typed on the keyboard, save anything further.
On Tue, 2006-01-17 at 10:45 +0200, Tom Alsberg wrote:
Timo Sirainen wrote:
As promised, here it is finally.
Not to nudge about it, but I wonder if you looked at my mail of yesterday regarding PAM_RHOST... I now downloaded the beta and still had to apply that patch to get the client host to be passed to PAM as PAM_RHOST on FreeBSD 5.4.
I thought I'd change it after this release to make sure no existing system breaks. I don't know why there's the #ifdef. Possibly because I saw it done in similar way somewhere else..
Timo Sirainen <tss@iki.fi> writes:
As promised, here it is finally.
v1.0 is getting nearer and I want people to start looking at Dovecot's sources. Getting some more help with development would be nice, but I'd
Thanks for your work!
I looked at the .spec "source" with rpmbuild on SUSE Linux 10.0 x86, which came up with these autocheck errors - it appears as though the .spec file could use some updating WRT newer plugins.
RPM build errors: File listed twice: /usr/libexec/dovecot Installed (but unpackaged) file(s) found: /usr/lib/dovecot/imap/lib01_zlib_plugin.a /usr/lib/dovecot/imap/lib01_zlib_plugin.la /usr/lib/dovecot/imap/lib01_zlib_plugin.so /usr/lib/dovecot/imap/lib02_imap_quota_plugin.a /usr/lib/dovecot/imap/lib02_imap_quota_plugin.la /usr/lib/dovecot/imap/lib02_imap_quota_plugin.so /usr/lib/dovecot/lib01_quota_plugin.a /usr/lib/dovecot/lib01_quota_plugin.la /usr/lib/dovecot/lib01_quota_plugin.so /usr/lib/dovecot/lib02_trash_plugin.a /usr/lib/dovecot/lib02_trash_plugin.la /usr/lib/dovecot/lib02_trash_plugin.so
The attached patch fixes the immediate packaging errors, but does not yet add a SUSE-compatible init script (SUSE don't have /sbin/service or daemon or /etc/init.d/functions) - I don't have time for that now.
Note that SUSE also move /usr/libexec/* to /usr/lib for LSB compliance.
-- Matthias Andree
hi,
JFYI suse 10.1 will ship with dovecot 1.0b1 by default. you will find rpms for 1.0b1 in my people dir[1] later this day.
with kind regards
darixMarcus Rueckert <rueckert@informatik.uni-rostock.de> writes:
JFYI suse 10.1 will ship with dovecot 1.0b1 by default. you will find rpms for 1.0b1 in my people dir[1] later this day.
I have used the dovecot 0.99.14 RPM from SUSE 10.0 and updated it to 1.0.beta1. No automatic configuration migration is in the scripts. Watch the RPM warnings when updating.
http://home.pages.de/~mandree/suse-10.0-i586/
I haven't got an x86_64 at this time because the -[f]pie breaks building the plugins and x86_64 can use i586 RPMs anyways.
-- Matthias Andree
On 2006-01-18 11:07:07 +0100, Matthias Andree wrote:
JFYI suse 10.1 will ship with dovecot 1.0b1 by default. you will find rpms for 1.0b1 in my people dir[1] later this day.
I have used the dovecot 0.99.14 RPM from SUSE 10.0 and updated it to 1.0.beta1. No automatic configuration migration is in the scripts. Watch the RPM warnings when updating.
cool that people have so much patience that they cant even wait one day. ;)
I haven't got an x86_64 at this time because the -[f]pie breaks building the plugins and x86_64 can use i586 RPMs anyways.
exactly that was the reason why it needed a bit more time than expected. I hope i can upload the packages in a few minutes.
darix
Marcus Rueckert schrieb am 2006-01-18:
On 2006-01-18 11:07:07 +0100, Matthias Andree wrote:
JFYI suse 10.1 will ship with dovecot 1.0b1 by default. you will find rpms for 1.0b1 in my people dir[1] later this day.
I have used the dovecot 0.99.14 RPM from SUSE 10.0 and updated it to 1.0.beta1. No automatic configuration migration is in the scripts. Watch the RPM warnings when updating.
cool that people have so much patience that they cant even wait one day. ;)
You wrote about 10.1. The list of dependencies is long, so chances are the binary RPM won't work anyways.
-- Matthias Andree
On 2006-01-18 12:11:30 +0100, Matthias Andree wrote:
You wrote about 10.1. The list of dependencies is long, so chances are the binary RPM won't work anyways.
i said it will be shipped with 10.1. and i said i will upload rpms for 9.3,10.0,10.1 to my people dir. :)
darix
Matthias Andree wrote:
I have used the dovecot 0.99.14 RPM from SUSE 10.0 and updated it to 1.0.beta1. No automatic configuration migration is in the scripts. Watch the RPM warnings when updating.
Please upload src.rpm or the specfile.
Sergey.Sergey Ivanov <seriv@parkheights.dyndns.org> writes:
Matthias Andree wrote:
I have used the dovecot 0.99.14 RPM from SUSE 10.0 and updated it to 1.0.beta1. No automatic configuration migration is in the scripts. Watch the RPM warnings when updating.
Please upload src.rpm or the specfile.
Sorry, should have written that:
http://home.pages.de/~mandree/suse-10.0-src/
-- Matthias Andree
Le 16.01.2006 21:52, Timo Sirainen a écrit :
I have this make install failure :
Making install in quota
make[3]: Entering directory
/home/nico/sources/dovecot/dovecot-1.0.beta1/src/plugins/quota' make[4]: Entering directory /home/nico/sources/dovecot/dovecot-1.0.beta1/src/plugins/quota'
mkdir -p -- /tmp/test/lib/dovecot/imap /tmp/test/lib/dovecot/lda
mkdir: `/tmp/test/lib/dovecot/imap' exists but is not a directory
Regards,
Nico Retournons à nos moutons. -+- François Rabelais (1494?-1553), Tiers livre (chap. 34) -+-
I ran into this too, here's how I got around it. I added the following the params to configure:
--with-moduledir=/usr/lib/dovecot/modules
You may well not want that exact path, but the point is to set --with-moudulesdir to be different from the lib or libexec dirs.
John
Nicolas STRANSKY wrote:
Le 16.01.2006 21:52, Timo Sirainen a écrit :
I have this make install failure :
Making install in quota make[3]: Entering directory
/home/nico/sources/dovecot/dovecot-1.0.beta1/src/plugins/quota' make[4]: Entering directory/home/nico/sources/dovecot/dovecot-1.0.beta1/src/plugins/quota' mkdir -p -- /tmp/test/lib/dovecot/imap /tmp/test/lib/dovecot/lda mkdir: `/tmp/test/lib/dovecot/imap' exists but is not a directoryRegards,
Timo Sirainen wrote:
v1.0 is getting nearer and I want people to start looking at Dovecot's
Duh! Just when I finally got the time to add SQLite support. It works, but I have to clean some things up. Do you think it could still go into 1.0? There are only a few minor changes to the rest of the code, so it's very unlikely that something got broken.
sources. Getting some more help with development would be nice, but I'd
I'll try to fix the epoll+ssl problems, which should be resolved before 1.0, I think. I already took a short look at it, but I have a hard time understanding what the code is doing and there are no comments.
On 2006-01-17 16:36:33 +0100, Jakob Hirsch wrote:
sources. Getting some more help with development would be nice, but I'd
I'll try to fix the epoll+ssl problems, which should be resolved before 1.0, I think. I already took a short look at it, but I have a hard time understanding what the code is doing and there are no comments.
JFYI: i tried it with a snapshot from 2006-01-14. and i didnt had any problems with epoll. i will test it again with my 1.0b1 rpms.
darix
Hi, while I have been debugging some kevent() related problems I have stumbled upon this in imap process:
(gdb) bt #0 0x00000000 in ?? () #1 0x0808befd in notify_callback (context=0x80fa040) at index-mailbox-check.c:65 #2 0x080c1ede in event_callback (context=0x80f3160) at ioloop-notify-kqueue.c:46 #3 0x080c2aff in io_loop_handler_run (ioloop=0x80ed200) at ioloop-kqueue.c:184 #4 0x080c1c02 in io_loop_run (ioloop=0x80ed200) at ioloop.c:235 #5 0x080649ab in main (argc=1, argv=0xbfbfea58, envp=0xbfbfea60) at main.c:238 (gdb) frame 1 #1 0x0808befd in notify_callback (context=0x80fa040) at index-mailbox-check.c:65 65 ibox->notify_callback(&ibox->box, ibox->notify_context); (gdb) l 60 ibox->notify_last_check = ioloop_time; 61 if ((unsigned int)(ioloop_time - ibox->notify_last_sent) >= 62 ibox->min_notify_interval) { 63 ibox->notify_last_sent = ioloop_time; 64 ibox->notify_pending = FALSE; 65 ibox->notify_callback(&ibox->box, ibox->notify_context); 66 } else { 67 ibox->notify_pending = TRUE; 68 } 69 } (gdb) inspect ibox $10 = (struct index_mailbox *) 0x80fa040 (gdb) inspect *ibox $11 = {box = {name = 0x80fa160 "INBOX", storage = 0x80eba40, v = { is_readonly = 0x808e4b8 <index_storage_is_readonly>, allow_new_keywords = 0x808e4e8 <index_storage_allow_new_keywords>, close = 0x8066e38 <maildir_storage_close>, get_status = 0x808d974 <index_storage_get_status>, sync_init = 0x8069148 <maildir_storage_sync_init>, sync_next = 0x808eb74 <index_mailbox_sync_next>, sync_deinit = 0x808ed3c <index_mailbox_sync_deinit>, notify_changes = 0x8066e88 <maildir_notify_changes>, transaction_begin = 0x80692d8 <maildir_transaction_begin>, transaction_commit = 0x8069310 <maildir_transaction_commit>, transaction_rollback = 0x80693ec <maildir_transaction_rollback>, keywords_create = 0x808e634 <index_keywords_create>, keywords_free = 0x808e658 <index_keywords_free>, get_uids = 0x8087fc0 <index_storage_get_uids>, mail_alloc = 0x8089a6c <index_mail_alloc>, header_lookup_init = 0x808bae8 <index_header_lookup_init>, header_lookup_deinit = 0x808bd48 <index_header_lookup_deinit>, search_get_sorting = 0x808d524 <index_storage_search_get_sorting>, search_init = 0x808d538 <index_storage_search_init>, search_deinit = 0x808d604 <index_storage_search_deinit>, search_next = 0x808d7dc <index_storage_search_next>, save_init = 0x806e04c <maildir_save_init>, save_continue = 0x806e370 <maildir_save_continue>, save_finish = 0x806e40c <maildir_save_finish>, save_cancel = 0x806e6a4 <maildir_save_cancel>, copy = 0x806ba0c <maildir_copy>, is_inconsistent = 0x808e518 <index_storage_is_inconsistent>}, pool = 0x80fa010, module_contexts = {buffer = 0x80fa168, element_size = 4}}, storage = 0x80eba40, index = 0x80f4400, view = 0x80f4a00, cache = 0x80f4600, mail_vfuncs = 0x80ded60, is_recent = 0x8066054 <maildir_is_recent>, md5hdr_ext_idx = 2, notify_to = 0x0, notify_files = 0x0, notify_ios = 0x0, notify_last_check = 1137544337, !!!!!!!! notify_last_sent = 1137544337, min_notify_interval = 0, notify_callback = 0, !!!!!!!! notify_context = 0x0, next_lock_notify = 1137544335, last_notify_type = MAILBOX_LOCK_NOTIFY_NONE, commit_log_file_seq = 0, commit_log_file_offset = 0, keyword_names = 0x80f446c, cache_fields = 0x80f4900, recent_flags = 0x80e13a0, recent_flags_start_seq = 130, recent_flags_count = 1, synced_recent_count = 1, sync_last_check = 1137544332, readonly = 0, keep_recent = 0, recent_flags_synced = 1, sent_diskspace_warning = 0, sent_readonly_flags_warning = 0, notify_pending = 0, mail_read_mmaped = 0}
The notify_callback field is NULL and the process SIGSEGVs. I know very little about IMAP protocol and I think this could be just a case of bad handling of invalid input. The IMAP session I had follows:
[...] a1 OK Logged in. a2 SELECT INBOX
- FLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk)
- OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk \*)] Flags permitted.
- 131 EXISTS
- 1 RECENT
- OK [UNSEEN 99] First unseen.
- OK [UIDVALIDITY 1137196697] UIDs valid
- OK [UIDNEXT 134] Predicted next UID a2 OK [READ-WRITE] Select completed. a3 IDLE
- idling
- 132 EXISTS
- 2 RECENT a4 DONE a3 BAD Expected DONE.
Vaclav Haisman
On Wed, 2006-01-18 at 02:17 +0100, Vaclav Haisman wrote:
Hi, while I have been debugging some kevent() related problems I have stumbled upon this in imap process: .. The notify_callback field is NULL and the process SIGSEGVs. I know very little about IMAP protocol and I think this could be just a case of bad handling of invalid input. The IMAP session I had follows:
To me this looks like io_remove() doesn't work properly in the kqueue code. notify_callback is NULL only after the notification is removed, and index_mailbox_check_remove_all() has been called.
a3 IDLE
- idling
- 132 EXISTS
- 2 RECENT a4 DONE a3 BAD Expected DONE.
BTW. Which IMAP client does this? Someone could tell them not to send tag with DONE command..
Timo Sirainen wrote:
On Wed, 2006-01-18 at 02:17 +0100, Vaclav Haisman wrote:
Hi, while I have been debugging some kevent() related problems I have stumbled upon this in imap process: .. The notify_callback field is NULL and the process SIGSEGVs. I know very little about IMAP protocol and I think this could be just a case of bad handling of invalid input. The IMAP session I had follows:
To me this looks like io_remove() doesn't work properly in the kqueue code. notify_callback is NULL only after the notification is removed, and index_mailbox_check_remove_all() has been called. I'll look into it.
a3 IDLE
- idling
- 132 EXISTS
- 2 RECENT a4 DONE a3 BAD Expected DONE.
BTW. Which IMAP client does this? Someone could tell them not to send tag with DONE command..
No client does that. I did this using telnet and keyboard :). I thought the crash is caused by my error.
VH
Packages for FC4:
i386: http://fedora.ivazquez.net/yum/4/i386/RPMS.alternatives/dovecot-1.0-0.iva.2.... http://fedora.ivazquez.net/yum/4/i386/SRPMS.alternatives/dovecot-1.0-0.iva.2...
ppc: http://fedora.ivazquez.net/yum/4/ppc/RPMS.alternatives/dovecot-1.0-0.iva.2.b... http://fedora.ivazquez.net/yum/4/ppc/SRPMS.alternatives/dovecot-1.0-0.iva.2....
x86_64: http://fedora.ivazquez.net/yum/4/x86_64/RPMS.alternatives/dovecot-1.0-0.iva.... http://fedora.ivazquez.net/yum/4/x86_64/SRPMS.alternatives/dovecot-1.0-0.iva...
-- Ignacio Vazquez-Abrams <ivazquez@ivazquez.net> http://fedora.ivazquez.net/
gpg --keyserver hkp://subkeys.pgp.net --recv-key 38028b72
participants (16)
-
Andy Cravens
-
Brad
-
Gabriel Kihlman
-
Ignacio Vazquez-Abrams
-
Jakob Hirsch
-
John Gray
-
Marcus Rueckert
-
Matthias Andree
-
Nicolas STRANSKY
-
Peter Hessler
-
Reuben Farrelly
-
Sergey Ivanov
-
Timo Sirainen
-
Tom Alsberg
-
Vaclav Haisman
-
Václav Haisman