Eric Rostetter wrote:
Quoting Ethan Sommer sommere@gac.edu:
Maybe a test suite next time, A test suite sounds like an excellent idea, which I might be interested in implementing. How would such a beat work.
Best practices is a unit test suite, to test every function/routine as thoroughly as possible.
I'd imagine a program which takes a "test case" which comes with
- an initial mailbox
- a series of IMAP commands to execute on the mailbox
- a final test which has a IMAP command to run and a regular expression which its result should match.
This might be one group of tests, but it wouldn't be sufficient by itself.
I would think that non-programmers could come up with test cases given such a system.
Non-programmers can at least suggest tests that should exist for others to code. In effect, every bug report should spawn a new unit test when it is fixed. So there is always opportunity for non-programmers to at least suggest tests if not code them.
Off the top of my head, the downsides I see are: doesn't test deliver, regular expressions might wind up being too specific and generate false failures when doing regression tests, requires knowledge of IMAP protocol.
That is why a complete set of unit tests is better. It tests everything. Of course, it takes more time to develop and maintain.
What do people think?
I think any project this size, with the goals it has, shoujld certainly create a unit test suite. But then, Timo only has so much time, and volunteers would be very helpful here. It could almost be started as a separate project inside the main project in CVS.
One of the ways of testing dovecot might be to utilize any test suites available for common imap clients such as thunderbird, pine, ... the idea of extending the test suite to catch all bugs found after release is a good one.
- Bart