[jala-dev] Re: Test framework error handling
- From: <robert.gaggl@xxxxxx>
- To: <jala-dev@xxxxxxxxxxxxx>
- Date: Wed, 21 Mar 2007 13:19:31 +0100
hi hannes,
> I'm in the process of setting up a Jala test suite for the
> recent skin enhancements in Helma 1.6. First, let me say that
> the Jala testing framework is simply wonderful. Thanks a lot
> for the great conception and implementation. (I haven't had
> the opportunity to use any other part of Jala yet, but I
> surely will do so soon.)
that's nice to hear, thanks!
> The one problem I have with the framework is that it stops
> testing on the first failure. I'm not sure what the
> motivation is here, but I think this is rather unusual for a
> testing framework. It is quite common to have tests fail,
> also over a longer period of time. If people only wrote tests
> for things that alreay work, it would only be half as useful IMO.
the reason for the "halt on failure" behaviour was the assumption that a
test function should be able to rely on changes made in preceeding test
functions (within one test file). anyway i admit that this is probably
rarely needed, and since this issue was already raised by tobi and
others too i'm going to change the default behaviour to "continue on
error", and making it possible to switch to "halt on error" within
single test files in a future release.
please mind that even with this change test functions won't be run if
the "setup" method fails, but i think that's ok.
thanks for the patch, i'll commit it asap.
robert
Other related posts:
