PlexTest - 'Equivalent mutants' and selective mutation testing

Early mutation testers made a variety of mutations to a program - operator substitution, etc. This (somewhat aimless) mutation testing often introduced a severe problem: it would create 'equivalent mutants' - mutated programs that nevertheless worked perfectly. Equivalent mutants interfere substantially with the automated testing process because each one has to be individually and manually analysed and rejected.

Later mutation testers performed selective mutation testing which alleviated the problem, but did not eliminate it.

A selective mutation tester might attempt the following:

Which, while undoubtedly different - the value of the variable 'sum' has been negated - nevertheless runs without fault. Since the variable 'sum' is merely being tested as to whether or not it has the value zero, the mutated program works as normal. It is an 'equivalent mutant'.

Plextest is a highly selective mutation tester. It only performs the mutation of 'deletion' - that is, removal of an instruction. But whereas other mutation testers only perform full statement deletion, Plextest performs sub-expression deletion. For example, it fully tests every binary operator in a program by removing, along with the operator, first the left sub-expression, then right sub-expression.

Plextest can perform full selective (and indeed, non-selective) mutation testing, if you so desire (and you can change its behavior mid-program via comment-directives). Full mutation testing (and selective mutation testing) attempt to find nearly all software faults - at the expense of sometimes creating equivalent mutants, and thereby taking the 'automated' out of automated testing. Plextest attempts to bridge the gap between coverage testing and traditional mutation testing - ensuring that each instruction in the program is not just executed, but is verifiably measured - without compromising the automated nature of the testing.

Glossary
Failure - an incorrect output, crash, hang, or other erroneous software behavior
Fault - a defect in a program that causes a fault
Testing - Regression testing
              Unit test
              Automated test
              Coverage testing
              Mutation testing
              Equivalent mutant
              Selective mutation testing
              Strong mutation testing
              Weak mutation testing