[Dovecot] a proposal to increase the stability

[Udo Rader]

On Tue, 2005-11-15 at 06:32 -0500, Tom Allison wrote:
> > What I am thinking of is to run those tests before anything new is
> > released. On the other hand I cannot even estimate how much work would
> > be needed for this ...
> > 
> > Udo Rader
> > 
> 
> You could develop a test suite using something like perl.
> The approach would be run beat the hell out of all the functionality 
> that dovecot provides and validate the input/output against what is 
> expected.  It's a black-box style of testing, but it would help indicate 
> any potential problems.   It was through my own version of this that I 
> found there are some limitations to how many UIDS I can transmit at one 
> time.  But who expects >10,000 UIDS in a single call?
> 
> Between running IMAP calls to localhost and inspecting the local 
> filesystem directly, I suspect a really nice test suite could be 
> developed.  Interestingly, most of it could apply to more than just 
> dovecot if you seperate testing for IMAP/Maildir standard formats from 
> dovecot particulars (like indexing?).

I see two possiblities: write an external "testclient" that does all the
nasty things MUAs are doing to dovecot or to extend the codebase to
include (unit?) test cases.

Major advantage of the external test suite indeed would be that one
could use it against any MDA and I even found a couple of testsuites for
POP3/IMAP servers, yet I am not sure that this is enough.

If for example I wanted to test if dovecot still correctly accesses an
SQL database after a code update, a simple POP3/IMAP protocol test would
probably not be enough to reveal any real problems. Or more general, it
would be a nice thing to be able to test if a certain config option
still works as before (I am thinking of locking methods on various
platforms, access to the various userdb/passdb backends, sasl, ...).

Maybe a new "make test" target during compilation would suffice for
that ...

Udo Rader

-- 
bestsolution.at EDV Systemhaus GmbH
http://www.bestsolution.at
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://dovecot.org/pipermail/dovecot/attachments/20051115/eba4d2b5/attachment-0001.pgp