[Dovecot] Test suite? Was [Dovecot-news] 1.0.rc21 released

[Eric Rostetter]

Quoting Ethan Sommer <sommere at gac.edu>:

>> Maybe a test suite next time,
> A test suite sounds like an excellent idea, which I might be interested
> in implementing. How would such a beat work.

Best practices is a unit test suite, to test every function/routine
as thoroughly as possible.

> I'd imagine a program which takes a "test case" which comes with
> 1. an initial mailbox
> 2. a series of IMAP commands to execute on the mailbox
> 3. a final test which has a IMAP command to run and a regular
> expression which its result should match.

This might be one group of tests, but it wouldn't be sufficient by
itself.

> I would think that non-programmers could come up with test cases given
> such a system.

Non-programmers can at least suggest tests that should exist for others
to code.  In effect, every bug report should spawn a new unit test when
it is fixed.  So there is always opportunity for non-programmers to
at least suggest tests if not code them.

> Off the top of my head, the downsides I see are: doesn't test deliver,
> regular expressions might wind up being too specific and generate false
> failures when doing regression tests, requires knowledge of IMAP
> protocol.

That is why a complete set of unit tests is better.  It tests everything.
Of course, it takes more time to develop and maintain.

> What do people think?

I think any project this size, with the goals it has, shoujld certainly
create a unit test suite.  But then, Timo only has so much time, and
volunteers would be very helpful here.  It could almost be started
as a separate project inside the main project in CVS.

> --
> Ethan Sommer
> Systems Administrator
> Gustavus Adolphus College
> 507-933-7042
> sommere at gac.edu



-- 
Eric Rostetter
The Department of Physics
The University of Texas at Austin

Go Longhorns!