White Box Testing
Test your code
   Home      TestPattern-PassFailPatterns
Author -  Marc Clifton
Marc is the creator of two open source projets, MyXaml, a declarative (XML) instantiation engine and the Advanced Unit Testing framework, and Interacx, a commercial n-tier RAD application suite.  Visit his website, www.marcclifton.com, where you will find many of his articles and his blog.
Marc lives in Philmont, NY.

Pass/Fail Patterns

These patterns are your first line of defense (or attack, depending on your perspective) to guarantee good code.  But be warned, they are deceptive in what they tell you about the code.

The Simple-Test Pattern

Pass/fail unit tests are the simplest pattern and the pattern that most concerns me regarding the effectiveness of a unit test.  When a unit test passes a simple test, all it does is tell me that the code under test will work if I give it exactly the same input as the unit test.  A unit test that exercises an error trap is similar--it only tells me that, given the same condition as the unit test, the code will correctly trap the error.  In both cases, I have no confidence that the code will work correctly with any other set of conditions, nor that it will correctly trap errors under any other error conditions.  This really just basic logic.  However, on these grounds you can hear a lot of people shouting "it passed!" as all the nodes on the unit test tree turn green.

The Code-Path Pattern

The Simple-Test pattern typifies what I call "black box testing".  Without inspecting the code, that's about all you can do--write educated guesses as to what the code under test might encounter, both as success cases and failure cases, and test for those guesses.  A better test ensures that at least all the code paths are exercised.  This is part of "white box testing"--knowing the inside workings of the code being tested.  Here the priority is not to set up the conditions to test for pass/fail, but rather to set up conditions that test the code paths.  The results are then compared to the expected output for the given code path.  But now we have a problem--how can you do white box testing (testing the code paths) when the code hasn't been written?  Here we are immediately faced with the "design before you code" edge of that sword.  The discipline here, and the benefit of unit testing by enforcing some up front design, is that the unit test can test for code paths that the implementer may not typically consider.  Furthermore, the unit test documents precisely what the code path is expected to do.  Conversely, discipline is needed during implementation when it is discovered that there are code paths that the unit test did not foresee--time to fix the unit test!

The Parameter-Range Pattern

Still, the above test, while improving on the Simple-Test pattern, does nothing to convince me that the code handles a variety of pass/fail conditions.  In order to do this, the code should really be tested using a range of conditions.  The Parameter-Range pattern does this by feeding the Code-Path pattern with more than a single parameter set.  Now I am finally beginning to have confidence that the code under test can actually work in a variety of environments and conditions.