Assessing and Restoring Reproducibility of Jupyter Notebooks

Jiawei Wang, Tzu-yang Kuo, Li Li, Andreas Zeller

Abstract

Jupyter notebooks---documents that contain live code, equations, visualizations, and narrative text---now are among the most popular means to compute, present, discuss and disseminate scientific findings. In principle, Jupyter notebooks should easily allow to reproduce and extend scientific computations and their findings; but in practice, this is not the case. The individual code cells in Jupyter notebooks can be executed in any order, with identifier usages preceding their definitions and results preceding their computations. In a sample of 936 published notebooks that would be executable in principle, we found that 73% of them would not be reproducible with straightforward approaches, requiring humans to infer (and often guess) the order in which the authors created the cells.

BibTeX

@inproceedings{Wang-al:ASE20,
  author    = {Jiawei Wang and
               Tzu{-}yang Kuo and
               Li Li and
               Andreas Zeller},
  title     = {Assessing and Restoring Reproducibility of Jupyter Notebooks},
  booktitle = {ASE},
  pages     = {138--149},
  publisher = {{IEEE}},
  year      = {2020},
}

Assessing and Restoring Reproducibility of Jupyter Notebooks

Abstract

Related papers