On Tue, 2005-11-15 at 06:32 -0500, Tom Allison wrote:
What I am thinking of is to run those tests before anything new is released. On the other hand I cannot even estimate how much work would be needed for this ...
Udo Rader
You could develop a test suite using something like perl. The approach would be run beat the hell out of all the functionality that dovecot provides and validate the input/output against what is expected. It's a black-box style of testing, but it would help indicate any potential problems. It was through my own version of this that I found there are some limitations to how many UIDS I can transmit at one time. But who expects >10,000 UIDS in a single call?
Between running IMAP calls to localhost and inspecting the local filesystem directly, I suspect a really nice test suite could be developed. Interestingly, most of it could apply to more than just dovecot if you seperate testing for IMAP/Maildir standard formats from dovecot particulars (like indexing?).
I see two possiblities: write an external "testclient" that does all the nasty things MUAs are doing to dovecot or to extend the codebase to include (unit?) test cases.
Major advantage of the external test suite indeed would be that one could use it against any MDA and I even found a couple of testsuites for POP3/IMAP servers, yet I am not sure that this is enough.
If for example I wanted to test if dovecot still correctly accesses an SQL database after a code update, a simple POP3/IMAP protocol test would probably not be enough to reveal any real problems. Or more general, it would be a nice thing to be able to test if a certain config option still works as before (I am thinking of locking methods on various platforms, access to the various userdb/passdb backends, sasl, ...).
Maybe a new "make test" target during compilation would suffice for that ...
Udo Rader
-- bestsolution.at EDV Systemhaus GmbH http://www.bestsolution.at