Bugs in the wild: examining the effectiveness of static analyzers at finding real-world bugs

Abstract

Static analysis is a powerful technique to find software bugs. In past years, a few static analysis tools have become available for developers to find certain kinds of bugs in their programs. However, there is no evidence on how effective the tools are in finding bugs in real-world software. In this paper, we present a preliminary study on the popular static analyzers ErrorProne and SpotBugs. Specifically, we consider 320 real Java bugs from the BugSwarm dataset, and determine which of these bugs can potentially be found by the analyzers, and how many are indeed detected. We find that 30.3% and 40.3% of the bugs are candidates for detection by ErrorProne and SpotBugs, respectively. Our evaluation shows that the analyzers are relatively easy to incorporate into the tool chain of diverse projects that use the Maven build system. However, the analyzers are not as effective detecting the bugs under study, with only one bug successfully detected by SpotBugs.

BibTeX

@inproceedings{Tomassi:FSE18,
  author    = {David A. Tomassi},
  title     = {Bugs in the wild: examining the effectiveness of static analyzers at finding real-world bugs},
  booktitle = {{ESEC/SIGSOFT} {FSE}},
  pages     = {980--982},
  publisher = {{ACM}},
  year      = {2018},
}

Bugs in the wild: examining the effectiveness of static analyzers at finding real-world bugs

Abstract

Related papers