Code coverage

Uses your system voice; voice and rate in settings.

Snapshot

≈ 7 min read

Names in other languages

No names in other languages are held for this entry yet.

Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.

In software engineering, code coverage, also called test coverage, is a percentage measure of the degree to which the source code of a program is executed when a particular test suite is run. A program with high code coverage has more of its source code executed during testing, which suggests it has a lower chance of containing undetected software bugs compared to a program with low code coverage. Many different metrics can be used to calculate test coverage. Some of the most basic are the percentage of program subroutines and the percentage of program statements called during execution of the test suite. Code coverage was among the first methods invented for systematic software testing. The first published reference was by Miller and Maloney in Communications of the ACM, in 19631.

In software engineering, code coverage, also called test coverage, is a percentage measure of the degree to which the source code of a program is executed when a particular test suite is run. A program with high code coverage has more of its source code executed during testing, which suggests it has a lower chance of containing undetected software bugs compared to a program with low code coverage. Many different metrics can be used to calculate test coverage.

Some of the most basic are the percentage of program subroutines and the percentage of program statements called during execution of the test suite. Code coverage was among the first methods invented for systematic software testing. The first published reference was by Miller and Maloney in Communications of the ACM, in 1963. int foo (int x, int y) { int z = 0; if ((x > 0) && (y > 0)) { z = x; } return z; } Assume this function is a part of some bigger program and this program was run with some test suite.

Function coverage will be satisfied if, during this execution, the function foo was called at least once. Statement coverage for this function will be satisfied if it was called for example as foo(1,1) , because in this case, every line in the function would be executed—including z = x; . Branch coverage will be satisfied by tests calling foo(1,1) and foo(0,1) because, in the first case, both if conditions are met and z = x; is executed, while in the second case, the first condition, (x>0) , is not satisfied, which prevents the execution of z = x; .

Condition coverage will be satisfied with tests that call foo(1,0) , foo(0,1) , and foo(1,1) . These are necessary because in the first case, (x>0) is evaluated to true , while in the second, it is evaluated to false . At the same time, the first case makes (y>0) false , the second case does not evaluate (y>0) (because of the lazy-evaluation of the Boolean operator), the third case makes it true . In programming languages that do not perform short-circuit evaluation, condition coverage does not necessarily imply branch coverage.

For example, consider the following Pascal code fragment: if a and b then Condition coverage can be satisfied by two tests: a=true , b=false a=false , b=true However, this set of tests does not satisfy branch coverage since neither case will meet the if condition.

Fault injection may be necessary to ensure that all conditions and branches of exception-handling code have adequate coverage during testing. if (a or b) and c then The condition/decision criteria will be satisfied by the following set of tests: However, the above tests set will not satisfy modified condition/decision coverage, since in the first test, the value of 'b' and in the second test the value of 'c' would not influence the output. So, the following test set is needed to satisfy MC/DC:

In practice

The target software is built with special options or libraries and run under a controlled environment, to map every executed function to the function points in the source code. This allows testing parts of the target software that are rarely or never accessed under normal conditions, and helps reassure that the most important conditions (function points) have been tested. The resulting output is then analyzed to see what areas of code have not been exercised and the tests are updated to include these areas as necessary.

Combined with other test coverage methods, the aim is to develop a rigorous, yet manageable, set of regression tests. In implementing test coverage policies within a software development environment, one must consider the following: What are coverage requirements for the end product certification and if so what level of test coverage is required?

The typical level of rigor progression is as follows: Statement, Branch/Decision, Modified Condition/Decision Coverage (MC/DC), LCSAJ (Linear Code Sequence and Jump) Will coverage be measured against tests that verify requirements levied on the system under test (DO-178B)? Is the object code generated directly traceable to source code statements? Certain certifications, (i.e.

DO-178B Level A) require coverage at the assembly level if this is not the case: "Then, additional verification should be performed on the object code to establish the correctness of such generated code sequences" (DO-178B) para-6.4.4.2. Software authors can look at test coverage results to devise additional tests and input or configuration sets to increase the coverage over vital functions. Two common forms of test coverage are statement (or line) coverage and branch (or edge) coverage.

Line coverage reports on the execution footprint of testing in terms of which lines of code were executed to complete the test. Edge coverage reports which branches or code decision points were executed to complete the test. They both report a coverage metric, measured as a percentage. The meaning of this depends on what form(s) of coverage have been used, as 67% branch coverage is more comprehensive than 67% statement coverage.

Generally, test coverage tools incur computation and logging in addition to the actual program thereby slowing down the application, so typically this analysis is not done in production. As one might expect, there are classes of software that cannot be feasibly subjected to these coverage tests, though a degree of coverage mapping can be approximated through analysis rather than direct testing. There are also some sorts of defects which are affected by such tools.

In particular, some race conditions or similar real time sensitive operations can be masked when run under test environments; though conversely, some of these defects may become easier to find as a result of the additional overhead of the testing code. Most professional software developers use C1 and C2 coverage. C1 stands for statement coverage and C2 for branch or condition coverage. With a combination of C1 and C2, it is possible to cover most statements in a code base.

Statement coverage would also cover function coverage with entry and exit, loop, path, state flow, control flow and data flow coverage. With these methods, it is possible to achieve nearly 100% code coverage in most software projects.

Notable code coverage tools

Hardware manufacturers

Aldec Mentor Graphics Silvaco Synopsys

Software

LDRA Testbed Parasoft

Usage in industry

Test coverage is one consideration in the safety certification of avionics equipment. The guidelines by which avionics gear is certified by the Federal Aviation Administration (FAA) is documented in DO-178B and DO-178C. Test coverage is also a requirement in part 6 of the automotive safety standard ISO 26262 Road Vehicles - Functional Safety.

See also

Cyclomatic complexity Intelligent verification Linear code sequence and jump Modified condition/decision coverage Mutation testing Regression testing Software metric Static program analysis White-box testing Java code coverage tools

References

Text derived from https://en.wikipedia.org/w/index.php?oldid=1355385285 under CC BY-SA 4.0; modifications have been made.

References

  1. ↩Wikipedia “Code coverage” revision 1355385285 · retrieved 2026-10-20 publication date (×1) · confidence 0.95 revision URL