[Dovecot] Test suite? Was [Dovecot-news] 1.0.rc21 released

[Eric Rostetter]

Quoting Timo Sirainen <tss at iki.fi>:

> On Sun, 2007-02-04 at 20:56 -0600, Eric Rostetter wrote:
>> Quoting Ethan Sommer <sommere at gac.edu>:
>>
>> >> Maybe a test suite next time,
>> > A test suite sounds like an excellent idea, which I might be interested
>> > in implementing. How would such a beat work.
>>
>> Best practices is a unit test suite, to test every function/routine
>> as thoroughly as possible.
>
> I think the real problem is how to test all the code paths. Especially

A good unit test suite should test all code paths...

> testing against race conditions isn't really possible without inserting
> some checkpoints into the code to block at specific points so the test

Yes, unit tests are not always the best at race conditions.  Sometimes
you can write a unit test for this, but often, you can't...

> can even work. I'm more interested in that and in checking error
> handling paths than just calling all the public functions with different
> parameters and checking if they work.

Well, it is your project, so you get the ultimate decision on things.

> I won't implement a unit test suite anytime soon though, so for now if
> someone's interested, an IMAP-based test suite would be possible to
> implement quite easily with some perl scripts or something. Even better
> if such test suite was made server-independent.

As I said, I wouldn't expect _you_ alone to implement a unit test suite,
as it would be way too big of a project.  But is is something the
community could do for the CVS HEAD code, so it would be there for
dovecot 2.x when that comes due.  But it is up to you if you want this
or not.

Creating an IMAP test is going to be harder than you would think also,
if you want to test mbox, maildir, and dbox all over local disk, NFS,
AFS, etc.  And even then, as has been pointed out, all you've tested
is the IMAP functions (no pop3, deliver, sieve, etc).  I'd see it
more as a stop gap method if done ad hoc.

Instead, I'd propose setting up a system for unit tests, and then within
that structure create your IMAP testing.  Then over time, more tests will
be added, until eventually you have a full test suite with a defined setup
and structure.

But again, it's your project, so you get the last word...

-- 
Eric Rostetter
The Department of Physics
The University of Texas at Austin

Go Longhorns!