Change-aware mutation testing for evolving systems

Abstract

Although the strongest test criteria, traditional mutation testing has shown to not scale with modern incremental development practices. In this work, we describe our proposal of commit-aware mutation testing and introduce the concept of commit-relevant mutants suitable to evaluate the system's behaviour after being affected by regression changes. We show that commit-relevant mutants represent a small but effective set that assesses the delta of behaviours between two consecutive software versions. Commit-aware mutation testing provides the guidance for developers to quantify to which extent they have tested error-prone locations impacted by program changes. In this paper, we portray our efforts to make mutation criteria change-aware as we study characteristics of commit-relevant mutants striving to bring mutation testing closer to being worthwhile for evolving systems.