A difference between statement and decision coverage

Question

Statement coverage is said to make sure that every statement in the code is executed at least once.
Decision/branch coverage is said to test that each branch/output of a decisions is tested, i.e. all statements in both false/true branches will be executed.
But is it not the same? In Statement coverage I need to execute all statements so I guess it can be only done by running all possible ways. I know I am missing something here..

Drusi

I guess this question is better for Programmers.StackExchange.com. By the way decision and branch coverage are not the same thing! – Cleodal

@Adriano Yes, wiki says so but a lot of other sources say it is the same. – Drusi

Not just wiki, formal definition for "decision" is each condition to enter the code path of a branch (then branch is the whole condition). Imagine, for example, the short circuit in C/C++. You may decide to use them as synonyms or not. – Cleodal

Moreover a statement may not be executed even if the branch where it is has been executed because of, for example, jumps, exceptions and/or other asynchronous conditions (locks, signals, events). That's why decision coverage and statement coverage may differ. – Cleodal

Answer 1

If the tests have complete branch coverage then we can say it also has complete statement coverage, but not the vice versa.

100% branch coverage => 100% statement coverage

100% statement coverage does not imply 100% branch coverage

the reason is in branch coverage apart from executing all the statements, we should also verify if the tests executes all branches, which can be interpreted as covering all edges in the control flow branch

if(a){
   if(b){
     bool statement1 = true;
   }
}

a = true, b = true will give 100% statement coverage, but not branch coverage

In the branch coverage we need to cover all the edges, which we missed in the statement coverage shown as red lines in the above image

Deathlike

Good answer, although my next question is: if the "if" statements had an "else", presumably there would be statements inside of the else block, meaning that the else would have to be covered for complete statement coverage. Is the diff between branch/line coverage just all of the "if" statements without an "else"? Or are there other examples? – Agler

So branch coverage additionally covers explicitly not executing some statements, which statement coverage can't really do. – Pitt

Answer 2

The answer by Paul isn't quite right, at least I think so (according to ISTQB's definitions). There's quite a significant difference between statement, decision/branch, and condition coverage. I'll use the sample from the other answer but modify it a bit, so I can show all three test coverage examples. Tests written here give 100% test coverage for each type.

if (a || b) {
    test1 = true;
}
else {
    if (c) {
      test2 = true;
    }
}

We have two statements here - if(a||b) and if(c), to fully explain those coverage differences:

1. statement coverage has to test each statement at least once, so we need just two tests:

• a=true and b=false - that gives us path if(a||b) true -> test1 = true
• a=false, b=false and c=true - that gives us path: if(a||b) false -> else -> if(c) true -> test2 = true

This way we executed each and every statement.

2. branch/decision coverage needs one more test:

• a=false, b=false, and c=false - that leads us to that second if but we are executing the false branch from that statement, that wasn't executed in statement coverage

That way we have all the branches tested, meaning we went through all the paths. 3. condition coverage needs another test:

• a=false, b=true - that leads through the same path as the first test but executes the other decision in OR statement (a||b) to go through it.

That way we have all conditions tested, meaning that we went through all paths (branches) and triggered it with each condition we could - the first 'if' statement was true in the first test because a=true triggered it and in the last test because b=true triggered it. Of course, someone can argue that case with a=true and b=true should be tested as well, but when we check how 'or' works then we can see it isn't needed and also variable c can be of any value as in those tests it is not evaluated.

At least I interpreted it this way. If someone is still interested :)

EDIT: In most sources, I found lately decision/branch coverage terms are equivalent, and the term I described as decision coverage is in fact condition coverage hence that update of the answer.

Einsteinium

I don't think that's correct. According to the ISTQB glossary (astqb.org/glossary): "statement coverage: The percentage of executable statements that have been exercised by a test suite. " In your "statement coverage" example, test1 = true; is never executed, but it obviously is an executable statement. – Shardashare

But test1 is not statement, it's variable that in that case is edge we can reach. 'if' is statement (conditional statement to be exact). Maybe I'm missing something here, but I understand it this way. – Einsteinium

test1 = true; is a variable assignment and therefore an executable statement. – Shardashare

Could be your way, I'm not oracle here :) I just understand it as conditional statement are the logic we have to test - testing each variable assignment (or print or whatever we will have in that branch) isn't the case. Speaking as a tester ;) – Einsteinium

@FrankSchmitt I read a bit and thought it over and I think you're right there - each statement means what you said. I've adjusted that example of mine so it resembles better what I wanted to show here. Thanks for help with it. – Einsteinium

Item 3 is incorrect, since decision coverage does not require each separate condition in a complex decision point to be exercised. When that is required, we have condition coverage. – Consternate

@Rogério Yes, you are right. I read about this some time ago but forgot to update it here. So I'll update it now. It is never too late :) – Einsteinium

Answer 3

Nice question. The explanation I often use is that an if-statement without an else-branch still has an invisible "empty" else-statement:

• Plain statement coverage just insists that all statements that are actually there are really executed.

• Branch coverage insists that even invisible else-branches are executed.

Similar situations occur with switch-statements without a default-case, and repeat-until loops. Branch coverage requires that a default-case is executed, and that a repeat-until is executed at least twice.

A code example:

if (passwordEnteredOK()) {
    enterSystem();
} 
/* Invisible else part 
else {
  // do nothing
}
*/

With statement coverage you just check that with a correct password you can use the system. With branch coverage you also test that with an incorrect password you will not enter the system.

Seed

Answer 4

You may have a statement like:

if(a || b || (c && d && !e)) {
    test1 = true;
} else {
    test2 = false;
}

If your code coverage says both the test1 and test2 lines are hit then you have statement coverage, but to get full branch coverage you will need to test when a is true, when a is false but b is true, when a and b are false but c and d are true and e is false, etc.

Branch coverage covers every potential combination of branch choices and so is harder to achieve 100% coverage.

Horseradish

Not sure why this was never voted as answer. Good explanation! – Perceive

This is false. Branch coverage need only cover all edges in the control flow graph. The boolean evaluation branches out into two edges. Look at the above answer for the correct answer. – Synergism

@PaulRutland What you're describing is condition coverage, not branch coverage. – Candracandy

Answer 5

The crux here is in the understanding of units of coverage: for statement coverage they are executable lines of code, for decision coverage they are all possible outcomes of decision nodes in the code.

Think about it.

This is why full decision coverage guarantees full statement coverage, but not the other way around.

Microhenry