googletest (an xUnit style C++ test framework) has an interesting feature: in addition to assertions like ASSERT_EQ(x, y) that stop the test, it has EXPECT_EQ(x, y) etc that will cause the test to fail without causing it to stop. I think this decoupling of “test failed” and “test execution stopped” is very useful. (Note this also implies a single test can have multiple failures, or if you prefer that a single test can have multiple messages attached to explain why its state is 'FAILED'.)
I wouldn't like to see a duplication of all assert* methods as expect* methods, but there are alternatives. A single self.expectThat method that takes a value and a matcher, for instance.
Or you could have a context manager:
with self.continueOnFailure():
self.assertEqual(x, y)
In fact, I suppose that's more-or-less what the subtests patch offers? Except the subtests feature seems to want to get involved in knowing about parameters and the like too, which feels weird to me.
Basically, I really don't like the “subtests” name, but if instead it's named something that directly says its only effect is that failures don't abort the test, then I'd be happy. |