Re: Are tests code or comment (was: distinguishing between failures and errors)

> >From my point of view comments and documentation is the manual (the 
> book), explaining how it should work and how it should be used.
> 
> Tests may be examples on usage.

Tests *are* the superset of all proven usage. Anything not tested is 
undefined and should not be trusted. :)

> I always prefer plain text documentation, its often easier to 
> explain in plain text how to use a class than with some hundred 
> lines of test cases. (And it is often easier to read.)

You're assuming that the writer writes well in the language you want to read. 
I just saw a notice for a job posting containing the following specification 
for a problem to solve, submitted along with the resume:

    Making use of JSP create an interface that would allow a user 
    to create, edit, and then display a set of nested tags without 
    the use of HTML in the forms. 

Do they think that's English they're writing? I asked an extremely capable 
colleague of mine for his interpretation of this and it was entirely 
different from mine. I'm willing to bet that yours would fall somewhere in 
between. I still don't have a clear enough idea what it's asking me for to be 
able to write it. Test cases would have made this clearer!

I admit that a product roadmap is essential to understanding a system when 
you don't have the benefit of asking the authors. I am not trying to say that 
written documentation is entirely useless. I *am* disputing the claim that 
written documentation is *clearer* than code. Even poorly-written code can 
only do one thing at a time; sufficiently-poorly-written documentation can 
mean almost anything.

Sadly, much of the documentation I read in my professional life is 
sufficiently-poorly-written. Much of the open source documentation I read, 
however, is quite good. Some of it is excellent.

> JB: You assume someone doing the tests careful but assume on the 
> other hand JavaDoc to be (usually) crap? 

Yes, and for a simple reason: good code works even with bad documentation. 
Good documentation doesn't make bad code work. In times of crisis, 
documentation is not updated. Even if it is initially written well, it 
erodes. Tests cannot so easily erode if they are to be the definitive 
demonstration of the presence of correctly-implemented features.

> I would not trust in the 
> tests if the JavaDoc was bad in the first case.

Why not? Maybe the Javadoc was written in the author's fourth language, but 
he is brilliant and articulates himself beautifully in Java! I have seen it.

> And even if both 
> were ok I would not conclude from the absence of a test case that 
> what would be tested is undefined behaviour -- it is way too easy to 
> miss a test case / special condition / parameter combination.

What else can you conclude? It seems to me that concluding otherwise only 
leads to pain. If the test is missing, then how do you know what the code 
does in that situation? Do you guess?

I would just write the test; then at least *I* would have it. If the author 
also wants it, I would probably provide it -- maybe for a fee.

--
J. B. Rainsberger,
President, Diaspar Software Services
Let's write software that people understand.
http://www.diasparsoftware.com/
telephone: +1 416 791-8603