Assessing Software Review Meetings: A Controlled Experimental Study Using CSRS

Abstract

Software review is a fundamental component of the software quality assurance process, yet significant controversies exist concerning efficiency and effectiveness of various review methods.A central question surrounds the use of meetings: traditional review practice views them as essential, while more recent findings question their utility.We conducted a controlled experimental study to assess several measures of cost and effectiveness for a meeting and non-meeting-based review method.The experiment used CSRS, a computer mediated collaborative software review environment, and 24 three person groups.Some of the data we collected included: the numbers of defects discovered, the effort required, the presence of synergy in the meeting-based groups, the occurrence of false positives in the non-meetingbased groups, and qualitative questionnaire responses.This paper presents the motivation for this experiment, its design and implementation, our empirical findings, conclusions, and future directions.