Re: distinguishing between failures and errors

pholser wrote:

><jbr>
>
>>One could argue that throwing the wrong exception type is a
>>failure and not an error, but since it's impossible to distinguish
>>between simply throwing the wrong exception type and something
>>going horribly wrong, it is easiest to treat them both as errors.
>>
></jbr>
>
>i think it depends on the exception.  if a method has advertised
>an exception in its throws clause, a caller can anticipate that it
>could happen, catch it, and call the outcome a failure.  any other
>"unanticipated" exception would then be an error.  but in practice,
>it probably doesn't matter whether the throwing of the wrong
>"advertised" exception should signal failure or error.
>
>i was trying to use the exception example as a starting point for
>discussion about whether failure/error distinctions in junit are
>meaningful to anyone.  my experience has been that they are not--
>in either case, failure or error, you're led in the direction of a
>solution.  what do you all think?
>
>this, too, the failure/error distinction and whether it matters, may
>be a good FAQ item too?
>


Good point!  

I'll give it a shot.  Feel free to pitch in to make it better.

----

Q: What's the difference between a failure and an error?

A:

Assertions are used to check for the possibility of failures, therefore 
failures are anticipated.  In the following example, the 
IndexOutOfBoundsException is expected and checked with an assertion.  If 
the expected exception is not raised, then a failure is produced.  

    public void testExceptionExpected() {
      try {
      
        new ArrayList(10).get( 11 );
        fail("Should have thrown IndexOutOfBoundsException" );
      
      } catch (IndexOutOfBoundsException success) {}
    }

Errors are unanticipated problems resulting in uncaught exceptions being 
propagated from a test method.  In the following example, the 
IndexOutOfBoundsException is not expected. The JUnit framework produces 
an error if the IndexOutOfBoundsException, or any other unchecked 
exception, is raised.  

    public void testExceptionUnexpected() throws IndexOutOfBoundsException {

      new ArrayList( 10 ).get( 11 );

    }

Both failures and errors will cause the test to fail.  However, it is 
useful to differentiate between failures and errors because the 
debugging process is slightly different.  

In the first example, the use of fail() will not generate a complete 
stack trace including the method that raised the exception.  In this 
case that's sufficient since we anticipate that the exception will be 
raised.  If it's not raised, then it's a problem with the test itself.  

In the second example,  the JUnit framework catches the exception and 
generates an error with a complete stack trace for the exception.  Since 
we don't expect this exception to be raised, a complete stack trace is 
useful in debugging why it was raised.

----

Thoughts?

Mike


[Non-text portions of this message have been removed]