Why is Coverage Testing Important?

Source Code Coverage Testing is an important part of software development. The purpose of Coverage Testing is to identify the parts of your application that have been executed and the parts of your application that have not been executed.

If the code has not been executed then you do not know if the code has an error in it.

If the code has not been executed then you know nothing about the behaviour of that code. You do not know if the software performs well, acceptably, poorly or not at all, regardless of the number of errors that may or may not be present in the code.

However, executing the code and finding that the code did perform acceptably does not mean an error is not present in the code, but it does mean you have tested that code. Code that has been tested usually has less bugs in it than code that has never been tested. Coverage Testing is not a substitute for rigorous quality assurance, Coverage Testing is a useful tool for increasing the quality level without whilst decreasing the workload of the Quality Assurance team because less bugs reach quality assurance.

The information gained from Coverage Testing can be used to inform:

• Identifying un-tested code.
  Code that is not executed, is by definition, un-tested. Therefore Coverage Testing is an excellent method for identifying un-tested software. Common reasons for the software being un-tested are:
  • The testing regime does not test a particular code pattern.
  • Error handling code for errors expected to happen (file not found, for example).
  • Error handling code for errors not expected to happen (out of memory, for example).
  • Code not executed because exception was thrown before the code was reached.
  • Logic error.
  • Coding error (typographic mistake, pre-processing directive, software component error, code commented out for a test etc).
  • For the above errors, the testing regime can be improved, or the cause of the error identified and fixed, followed by re-testing the software.
• Unit Testing.
  The software engineer performing the unit tests can determine from the Coverage Testing results if all paths through the software have been tested.
• Integration Testing.
  When a new API from a software development group is integrated into your application, you can use Coverage Testing to ensure that all uses of the new API are identified and tested.
• Software Quality Assurance.
  The software testing group can develop new tests for the software to exercise and test more of the software before the software is released to customers.
• Identifying unused code.
  Often when a software component is retired or replaced, other related software components may remain in the software build even though they are never used. Some modern linkers remove these functions. However, if this redundant software component is linked to other redudant components, the linker may not be able to identify that the software is truly unused. In such cases the linker leaves the software in the build. This increases the size of the software and the build time for the software. Identifying such code and removing reduces the project size and shortents the build time.

Application logic errors can also lead to software components being unused.

As you can see Coverage Testing can be used in many ways to identify software failures during software development, software maintenance and Quality Assurance. Finding and removing errors using Coverage Testing is much more cost effective than finding the same errors later in the software development process or when your customers report the errors to you.

If you are still doubtful about the effectiveness of Coverage Testing, ask yourself this question: Would I travel in a fly-by-wire aircraft that had not had its software coverage tested?.