-
Notifications
You must be signed in to change notification settings - Fork 57
MC DC
Coverage refers to the extent to which a given verification activity has satisfied its objectives. Coverage should be treated as a measure and not a method or a test which is expressed as the percentage of an activity that is accomplished.
- All possible points of entry and exit in the unit being tested must be invoked at least once.
- All possible outcomes of every decision in the unit being tested must be taken at least once.
- All possible outcomes of every condition in each decision in the unit must be taken at least once.
- It must be proven that every condition in each decision independently affects the outcome of that decision.
- Statement Coverage
Every executable statement in the program is invoked at least once. Not desirable because analysis of logical expressions are not part of statement coverage analysis. - Decision Coverage
Decision coverage ensures complete testing of control constructs:one for true outcome and false outcome. - Condition Coverage
Condition coverage requires that each condition in a decision take on all possible outcomes at least once. - Condition/Decision Coverage
Condition/decision coverage combines the requirements for decision coverage with those for condition coverage.Therefore a minimum of two test cases are necessary for each decision. - Modified Condition/Decision Coverage
It enhances the condition/decision coverage by requiring that each condition be shown to independently affect the outcome of the decision. The independence requirement ensures that the effect of each condition is tested relative to the other conditions. - Multiple Condition Coverage
multiple condition coverage requires test cases that ensure each possible combination of inputs to a decision is executed at least once. There was a time when software engineers advocated requiring 100% multiple condition coverage(exhaustive testing) because it guaranteed all possible combinations of input. However, this is quite costly(O(2^n)) and MC/DC attempts to provide a useful alternative.
Note: MC/DC applies to all decisions in a unit, not just decisions that result in a branch.
A condition is a Boolean expression that doesn't contain any Boolean operators (&&, ||, ==, >, etc.) A condition cannot be split up into simpler Boolean expressions.
A decision is a Boolean expression that is made up of conditions and zero or more Boolean operators. If a decision doesn't contain any Boolean operators, it is considered a condition.
MC/DC ensures that every point of entry and exit has been evaluated or invoked at least once in the program. For this statement to be valid, we have to ensure that every condition in a decision has has taken all possible routes at least once, every decision has taken all possible routes at least once, and each condition in a decision independently affects that decision's outcome by keeping all other conditions constant.
In order to show that each condition in a decision affects the outcome of that decision only a single condition is varied at a time while the others are held fixed. If changing the value of a single condition causes the outcome of the decision to change it can be assumed that the change in the condition's value is the cause for the change in the outcome.
A practical approach is presented based on the gate-level representation of logic constructs to evaluate that given set of requirements conforms with three of the four requirements for MC/DC.
• every decision in the program has taken all possible outcomes at least once • every condition in a decision in the program has taken all possible outcomes at least once • every condition in a decision has been shown to independently affect that decision's outcome
Mapping requirements-based test cases to source code structure reinforce the notion that structural coverage analysis is a check on the adequacy of requirements-based tests for a given source code implementation. The MC/DC approach provides steps for evaluating MC/DC claims without the aid of a coverage tool.
Minimum testing to achieve MC/DC for an 'and' gate requires the following: (1) All inputs are set to true with the output observed to be true. This requires one test case for each n-input 'and' gate. (2) Each and every input is set exclusively false with the output observed to be false. This requires n test cases for each n-input 'and' gate. Changing a single condition starting from a state where all inputs are true will change the outcome; that is, an 'and' gate is sensitive to any false input. Hence, a specific set of n+l test cases is needed for an n-input 'and' gate.
Minimum testing to achieve MC/DC for an 'or' gate requires the following: (1) All inputs are set false with the output observed to be false. This requires one test case for each n-input 'or' gate. (2) Each and every input is set exclusively true with the output observed to be true. This requires n test cases for each n-input 'or' gate. These requirements are based on an 'or' gate's sensitivity to a true input. Here again, n+l specific test cases are needed to test an n-input or gate.
The xor gate differs from both the 'and' and the 'or' gates with respect to MC/DC in that there are multiple minimum test sets for a 'xor'. For a two-input 'xor' gate, any combination of three test cases will provide MC/DC. For a test set to distinguish between an 'or' and a 'xor' gate, it must contain test case where both the inputs are set to true.
The logical 'not' works differently from the previous gates: the 'not' works only on a single operand. That operand may be a single condition or a logical expression.
A comparator evaluates two numerical inputs and returns a Boolean based on the comparison criteria. The following comparison criteria are considered: • less than • greater than • less than or equal to • greater than or equal to • equal to • not equal to Two test cases will confirm MC/DC for a comparator. One test case with a true outcome, and one test case with a false outcome. Hence, minimum testing for a comparator requires the following: (1) Input x set at a value above the comparison point (or y) (2) Input x set at a value below the comparison point (or y)
The if_then-else statement is a switch that controls the execution of the software. Minimum testing for the if-then-else statement requires the following: (1) Inputs that force the execution of the then path (that is, the decision evaluates to true) (2) Inputs that force the execution of the else path (that is, the decision evaluates to false). (3) Inputs to exercise any logical gates in the decision For example, for a single condition Z, the statement if Z then...else.., requires only two test cases to achieve MC/DC. The decision in if X or Y or Z then.., else.., requires four test cases to achieve MC/DC.
Loop statements are constructs that cause a sequence of statements to be executed zero or more times. Constructs such as the while loop and the for loop are switches to control the execution of the software similar to an if-then-else construct. In the context of MC/DC, the challenge is to confirm that loops have been traversed appropriately.
Not all decisions must appear at the start of a loop. The 'exit when' statement can be used anywhere within a loop construct or a for loop to terminate the execution of the loop.
We can now understand what output is expected for which input using the operators used. They are described in the section above. MC/DC understands how many condition tests need to be checked before giving the final result as output. For example, if there is an 'and' operator (&), MC/DC will have to check for all conditions in the condition block to evaluate the final output if all of them true, but if one of the conditions is false in a 'and' clause, MC/DC directly outputs false as there is no need to evaluate other conditions i.e a false condition masks all other inputs in the condition statement.
Similarly, in a 'OR' condition (||), MC/DC will evaluate conditions only until it finds one of them to be true. If all of them are false, it will evaluate all the conditions to make sure that no conditions are being satisfied, i.e a true statement masks all other conditions.
The evaluation starts with which operators (conditions) are being evaluated and then their requirements are checked. There are 5 steps to evaluation:
- Creating a schema of the source code
- Identifying the input test cases
- Elimination of masked tests
- Determine MC/DC based on the section written above
- Examining the outputs of tests and checking that they conform to the requirements of the software.
If a language generates object code, for a level A software, in addition to MC.DC performed in previous steps on source code, extra object code segments need to be verified
A single source code line may or may not generate multiple object code lines. A testing suite that covers 100% lines in source code, may actually cover only 75% of object code Therefore, 100% source code coverage does not gurantee 100% object code coverage
However, this additional analysis only applies for code lines that are not directly traceable. Majority of the times, this is satisfied by analyszing the object code to ensure that it is directly traceable to source code Since this method expects, compiler to behave as expected, one needs to qualify compiler features being used as verification tool