Using Testability Measures for Dependability Assessment

Abstract

Program "testability"is the probability that a fault in a program, if present, will cause the program to fail.Measures of testability can be used to draw inferences on program correctness from the observation of a series of failure-free test executions, a common need for software with "ultra-high reliability" requirements.For a program that has passed a certain number of tests without failing, a high value of testability implies a high probability that the program is correct.We give a general descriptive model of program execution and testing, and propose a more precise definition of program testability than that given by other authors.We then study the use of testability in: i) providing, through testing, confidence in the absence of faults and ii) bounding the probability of failures, from the results of operational testing.We derive the probability of absence of faults through a Bayesian inference procedure, criticise previously proposed derivations of this probability, and study the relationship between the testability of a program and its failure probability in operation.We derive the conditions under which a high testability improves one's expectations about program reliability.Last, we discuss the potential of these methods in practical applications.