On 22/09/06, Adrian Howard <adrianh@xxxxxxxxxxxxxx> wrote:
On 19 Sep 2006, at 10:36, David Cantrell wrote:
> Adrian Howard wrote:
>
>> Yeah - it's something I've noticed over the last year or so. I'm
>> talking to people less about "you should write tests", and much
>> more about "you should write /good/ tests".
>
> What do people think are *good* tests?
[snip]
Belated response. For me good tests suites:
* are good code (i.e. DRY, well factored, intention revealing, etc.)
* encourage good design (ala TDD)
* fail for bugs, but not refactoring
* only fail for bugs in the thing they're testing
This is true for unittests, it is not true for system tests and user
acceptance tests. A test which is supposed to tell the user if the
system will work or not should fail, no matter what part of the system
is responsible.
You can of course decide not to write any system tests but then that
leaves it up to the users to all write their own - a wasteful
duplication of effort which goes against the point of releasing the
software at all. Your module is clearly the best place to put the
system tests for your module.
This could all be solved by some elaborate scheme for marking certain
tests as dependent on others and/or as against external modules. Then
the toolchain could automatically point the finger of blame in the
right direction. Until then (Perl 7?) keep in mind that there are (at
least) 2 reasons to write tests:
1 - so that you, the developer, can be confident that your code is correct
2- so that the end user can be confident that the code they are
installing works correctly in the environment into which they are
installing it
A good test suite gets both,
F
* don't depend on the order the tests are run in
The antithesis of good test suites are when I come across people
taking a big bad ball of mud, and then spends three months adding
copy 'n' paste tests until they have 100% statement coverage.
Because they know tests are good - and 100% statement coverage is
extra double plus good!
Unfortunately what they end up with is a test suite that is as
brittle as fuck. Change any damn thing - even if you're fixing stuff
- and half the test suite falls over. Their wonderful test suite
either ends up being completely ignored, or making change for the
better harder.
Cheers,
Adrian
|
