Data set for "Investigating the Readability of Test Code: Combining Scientific and Practical Views"

Abstract

Purpose. The readability of software code is key for understanding and maintaining software systems and tests. Although several studies focus on investigating the readability of software code, we observed limited research work that  focuses on the readability of software test code. In industry practice, a limited number of guidelines aim for improving and assessing the readability of software (test) code.

Objective.Therefore, in this paper, we aim at investigating the readability of test code from an academic perspective  complemented by practical views, discussed in grey literature. Furthermore, we conducted a controlled experiment in academic environment to investigate the readability of selected test cases based on combined scientific and practical views. 

Methods. First, we build on the Systematic Mapping Study (SMS) approach with focus on academic literature, complemented by practical aspects reported in  grey literature. Second we investigate the perception of readability and understandability in a controlled experiment in academic environment.

Results. The result set of the SMS includes 19 studies for further analysis. The analysis of grey literature includes 62 results. Based on these sources, we identified a set of commonly agreed influence factors that are relevant for academia and industry but also smaller sets of factors that are mainly discussed in academia or industry with limited overlap. For a selected set of test cases we conducted a controlled experiment to collect knowledge on influence factors and the perception on the readability of test cases. For half of the test cases we showed that selected influence factors have a significant impact on the readability. 

Conclusion. The readability of test code is of interest in academia and industry practice. The results of the reported studies showed a common consensus on a set of influence factors to assess the readability of software test code. However, we also found factors that are more relevant for academia and others that are more relevant for industry. Based on the results, we see the need to bring together academic and industry viewpoints to achieve a common view on the readability of software test code

Table of contents

The data set includes (a) Mapping Table for the systematic Mapping study, (b) Mapping Table for grey literature analysis, (c) Participant's experience data, (d) Participant's readability assessment data, and (e) Experimental package with selected test cases

Other

Subject / Keywords: Test Code · Readability · Understandability · Maintainability · Systematic Mapping Study · Grey Literature · Controlled Experiment