Evaluating test selection strategies for end-user specified flow-based applications

Abstract

An emerging class of end-user programming is the assembly of flow-based applications from a set of reusable components. Testing has become a major challenge, as a very large number of flows can be assembled from a set of components with the expectation of functioning correctly. Faults in assembled flows can create dissatisfaction among users and thereby potentially undermine this end-user programming paradigm We approach this problem as a flow-selection problem, and are interested in ways of testing a subset of flows that provide a high likelihood of revealing faults. We describe a number of flow-selection strategies, which run in the context of a flow pattern, a specification mechanism that constrains the space of assemble-able flows. We evaluate the different strategies on real-world flow patterns in terms of efficiency, i.e., the reduction of flows to test, and effectiveness, measuring of how well the strategies can catch faults.