White Box Testing
Test your code
   Home      CodeCoverage
 

Code Coverage

Code coverage or test coverage keeps track the parts of your code get executed and the parts which do not, when the tests are run. Thus code coverage generally refers to test completeness.
 
A coverage tool analyzes the source code with the executed tests. If the coverage is high and the untested source lines are of low impact to the system's overall quality, then no more additional tests are required.
 

Why is it important?

The results generated by code coverage analysis are quantitative.These quantitative results are direct  related to the parts of your code get tested thus increase the confidence of product quality. Total number of executed lines divided by the total number of source lines code, gives you coverage percentage. Higher the percentage, the better.  Practically, 100% is rare.
 
For example, you have 1,000 source lines code. Now you run coverage tool with unit tests and look at the results, and find that only 500 lines of code were actually run by your tests. This indicates that you are not testing everything even your all unit tests passed. That means the unit tests are not complete.
Here are some standard ranges of code coverage
 
More than 80%     -   Higher test coverage
80% to 60%         -   Moderate test coverage
60% to 40%         -   low test coverage
Less than 40%      -   Very low test coverage
 
As you write more and more tests your coverage percentage gets higher. Last fractions of percentage require more efforts. It takes more effort to write unit tests to get around or near to 100 percent.
 

Who should do it?

Code coverage analysis done by QA team as well as development team to confirm there unit API test coverage.
 

Code Coverage types

  • Function coverage 
Tests whether each function of the source code executed or not. Its very basic  testing to assure that all the functionality covered in the test.
 
  •       Statement coverage(or Line coverage)
Tests whether each line of the source code executed or not. And ignores whether you have covered all the logic or not.
 
func(bool flag)
{
       int i = 0;
       if( flag )
          I = 5 ;
          . . .
          . . .
}
 
Now in this example if flag is true then all the statements of this code will execute but ignoring the logic if flag is false.This type of coverage weaker even 100% statement coverage. It is difficult to achieve 100% statement coverage due to dead code and sections of code designed to handle error conditions. 
 
Keep in mind that the number of statement not equal to the number of lines of code, as a single line may contain many number of statement.
 
Statement coverage also referred as Line coverage, basic block, and segment coverage.
 
  •       Decision coverage( or Branch coverage )
Whether boolean expressions true and false  tested in control structures such as if-statement and while-statement. The entire boolean expression is considered one true-or-false predicate regardless of whether it contains logical-and or logical-or operators. this measure includes coverage of switch-statement cases, error cases and exception handlers.
 
The disadvantage is that ignores logical operators branching (a logical OR was short-circuited or not)
   if ( condition1 || condition2 ) {
     statement1;
   else
     statement2;
 
condition2 get executed or not? Not reported with Decision coverage.
Decision coverage also refered as branch coverage
 
  •       Condition coverage
Has each boolean sub-expression evaluated both to true and false? This does not necessarily imply decision coverage.
 
Whether boolean expressions true and false  tested the sub-expressions (separated by logical ANDs and ORs) as well in control structures. The sub-expressions evaluated independently of each other
 
if ((A>0) && (B<0)), then you will need to test
true && false,
false && true
 
  •       Condition decision coverage
It is a combination of both condition coverage and decision coverage and test every possible combination of boolean sub-expression
if ((A>0) && (B<0)), then you will need to test
true && true,
false && false
 
  •       Modified condition/decision coverage(MC/DC)
This metric is extension of condition decision coverage and stronger than condition decision coverage. This metric extends condition decision coverage with requirement that each condition should affect the decision outcome independently.
The formal definition of MC/DC is
Each decision tries all possible outcomes at least one
Each condition in a decision tries all possible outcome at least one
Each entry and exit point has been invoked at least one
Each condition in a decision is shown to independently affect that decision outcome
 
if ((A>0) && (B<0)), then you will need to test.
true && true,
true && false,
false && true
 
  •       Multiple condition coverage
This coverage criteria requires all possible combination conditions inside each decision
if ((A>0) && (B<0)), then you will need to test all four possible combination
true && true,
true && false,
false && true,
false && false
 
  •       Loop coverage
In loop coverage test you test whether each loop body(for and while loop) executed  zero times, exactly once, and more than once.
 
  •       Linear code sequence and jump (LCSAJ) Coverage
LCSAJ is a software analysis method used to identify structural units in code  under test. Its primary use is with dynamic software analysis to help answer the question "How much testing is enough?".Dynamic software analysis is used to measure the quality and efficacy of software test data, where the quantification is performed in terms of structural units of the code under test. When used to quantify the structural units exercised by a given set of test data, dynamic analysis is also referred to as coverage analysis.
 
The LCSAJ analysis method was devised by Professor Michael Hennell in order to perform quality assessments on the mathematical libraries on which his nuclear physics research at the University of Liverpool depended.Professor Hennell later founded the Liverpool Data Research Associates (LDRA) company to commercialize the software test-bed produced for this work, resulting in the LDRA Testbed product.
 
An LCSAJ is a software code path fragment consisting of a sequence of code (a linear code sequence) followed by a control flow Jump, and consists of the following three items:
the start of the linear sequence of executable statements
the end of the linear sequence
the target line to which control flow is transferred at the end of the linear sequence.
 
LCSAJ coverage = I/L
Where: I = Number of LCSAJs exercised at least once.
L = Total number of LCSAJs.
 
LCSAJs depends on the topology of a module's design and not just its semantics, they do not map onto code structures such as branches and loops. LCSAJs are not easily identifiable from design documentation. They can only be identified once code has already been written. LCSAJs are consequently not easily comprehensible.
 
When used for coverage analysis, LCSAJs are considered to have a test effectiveness ratio of 3.
 
  •       JJ-Path coverage have all jump to jump paths(aka LCSAJs) been executed?
 
  •       Path coverage Has every possible route through a given part of the code been executed?
 
  •       Entry/exit coverage Has every possible call and return of the function been executed?
 
  •       Loop coverage Has every possible loop been executed zero times, once, and more than once?
 
  •    Parameter Value Coverage - For each parameter in a method, have the most common possible parameter values been tested?
 
 

External References