Applying Design of Experiments to Software Testing (Experience Report)

Abstract

Recently, a class of experimental designs has been devised that guarantee input domain coverage up to all combinations of k test factors taken t at a time.With such designs, all pairwise combinations (or triplets or quadruplets, etc.) are selected at least once.To evaluate their applicability to software testing, we analyzed the extent to which software coverage (i.e., code execution) achieved by these designs for t=1, ... ,k is representative of that achieved by exhaustively testing all factor combinations.The block coverage obtained for t;5;2 was comparable with that achieved by exhaustively testing all factor combinations but higher-order values of t were required for path coverage.Implications of these results for software testing are discussed.