[Dovecot] Test suite? Was [Dovecot-news] 1.0.rc21 released

[Eric Rostetter]

Quoting Dan Price <dp at eng.sun.com>:

> I think it's not the best advice to tell a software developer (any
> developer) that creating a test suite will be harder than they think...
> That's just one more reason not to start on it. :)

Yeah, there is some logic in that... ;)

> Others pointed out that testing every codepath will be hard; I think the
> problem is much more decomposable than that.  The goal should be first
> to test the parts of the server which most influence data integrity,
> and then to work up from there.

Well, the idea of a unit test suite is that the smallest units influence
the data integrity the most, and everything works up from there....
Your believe system may differ...

> Things like testing with NFS, AFS, exotic plugins, etc. can be done in
> part by the user community--- that is to say, before a major release of
> dovecot the community can run the test suite in a wide variety of
> configurations.

Yes, that is always the goal of a test suite.  But at the same time,
sometimes you need code to check for specific things for different
environments...

> Another approach is to play to your strength: Dovecot is pretty cleanly
> organized into different modules.  That means that a set of C programs
> could be written to exercise those interfaces, improving the likelihood
> that individual components will work as planned.

That is exactly what a unit test suite does.  If you looked up the
executive summary of a unit test suite, it would match what you say above.

> Here are two reasonable summaries of what testing can do for a project,
> written by some of my colleagues:
>
> http://blogs.sun.com/bill/entry/zfs_and_the_all_singing
> http://blogs.sun.com/ahl/entry/on_testing

I think everyone agrees testing is good, and most people would agree
doing testing correctly is not easy.  The problem is of course people
are lazy, and we take shortcuts...  Part of human nature.  The good
thing to remember is we can, if we decide to, overcome human nature
and reach a level higher than our base nature...

>         -dp

-- 
Eric Rostetter
The Department of Physics
The University of Texas at Austin

Go Longhorns!