Themis: automatically testing software for discrimination

Rico Angell, Brittany Johnson, Yuriy Brun, Alexandra Meliou

Abstract

Bias in decisions made by modern software is becoming a common and serious problem. We present Themis, an automated test suite generator to measure two types of discrimination, including causal relationships between sensitive inputs and program behavior. We explain how Themis can measure discrimination and aid its debugging, describe a set of optimizations Themis uses to reduce test suite size, and demonstrate Themis' effectiveness on open-source software. Themis is open-source and all our evaluation data are available at http://fairness.cs.umass.edu/. See a video of Themis in action: https://youtu.be/brB8wkaUesY

BibTeX

@inproceedings{Angell-al:FSE18,
  author    = {Rico Angell and
               Brittany Johnson and
               Yuriy Brun and
               Alexandra Meliou},
  title     = {Themis: automatically testing software for discrimination},
  booktitle = {{ESEC/SIGSOFT} {FSE}},
  pages     = {871--875},
  publisher = {{ACM}},
  year      = {2018},
}

Themis: automatically testing software for discrimination

Abstract

Related papers