A study on the lifecycle of flaky tests

Wing Lam, Kivanç Muslu, Hitesh Sajnani, Suresh Thummalapenta

Abstract

During regression testing, developers rely on the pass or fail outcomes of tests to check whether changes broke existing functionality. Thus, flaky tests, which nondeterministically pass or fail on the same code, are problematic because they provide misleading signals during regression testing. Although flaky tests are the focus of several existing studies, none of them study (1) the reoccurrence, runtimes, and time-before-fix of flaky tests, and (2) flaky tests in-depth on proprietary projects.

BibTeX

@inproceedings{Lam-al:ICSE20,
  author    = {Wing Lam and
               Kivan{\c{c}} Muslu and
               Hitesh Sajnani and
               Suresh Thummalapenta},
  title     = {A study on the lifecycle of flaky tests},
  booktitle = {ICSE},
  pages     = {1471--1482},
  publisher = {{ACM}},
  year      = {2020},
}

A study on the lifecycle of flaky tests

Abstract

Related papers