BASHIRI: Learning Failure Oracles from Execution Features

Marius Smytzek, Martin Eberlein, Tural Mammadov, Lars Grunske, Andreas Zeller

DOI 10.1109/ase63991.2025.00372 dblp

Abstract

Program fixes must preserve passing tests while fixing failing ones. Validating these properties requires test oracles that distinguish passing from failing runs.We introduce BASHIRI, a tool that learns failure oracles from test suites with labeled outcomes using execution features. BASHIRI leverages execution-feature-driven debugging to collect program execution features and trains interpretable models as testing oracles. Our evaluation shows that BASHIRI predicts test outcomes with 95% accuracy, effectively identifying failing runs. BASHIRI is available as an open-source tool at https://github.com/smythi93/bashiriA demonstration video is available at https://youtu.be/D2mJkCtSXtM

BibTeX

@inproceedings{Smytzek-al:ASE25,
  author    = {Marius Smytzek and
               Martin Eberlein and
               Tural Mammadov and
               Lars Grunske and
               Andreas Zeller},
  title     = {{BASHIRI:} Learning Failure Oracles from Execution Features},
  booktitle = {ASE},
  pages     = {4049--4052},
  publisher = {{IEEE}},
  year      = {2025},
}

BASHIRI: Learning Failure Oracles from Execution Features

Abstract

Related papers