Which components of a large software system are the most defect-prone? In a study on a large SAP Java system, we evaluated and compared a number of defect predictors, based on code features such as complexity metrics, static error detectors, change frequency, or component imports, thus replicating a number of earlier case studies in an industrial context. We found the overall predictive power to be lower than expected; still, the resulting regression models successfully predicted 50-60% of the 20% most defect-prone components.

Reference

BibTeX Entry

@inproceedings{holschuh-icse-2009,
 title = "Predicting Defects in SAP Java Code: An Experience Report",
 author = "Tilman Holschuh and Markus Paeuser and Kim Herzig and Thomas Zimmermann and Rahul Premraj and Andreas Zeller",
 year = "2009",
 month = "May",
 booktitle = "Proceedings of the 31th International Conference on Software Engineering",
 location = "Vancouver, BC
}

Download PDF version

Given a large body of code, how do we know where to focus our quality assurance effort? By mining the software’s defect history, we can automatically learn which code features correlated with defects in the past—and leverage these correlations for new predictions: “In the past, high inheritance depth was an indicator of a high number of defects. Since this new component also has a high inheritance depth, let us test it thoroughly”. Such history-based approaches work best if the new component is similar to the components learned from. But how does learning from history perform for projects with high variability between components? We ran a study on two SAP products involving a wide spectrum of functionality. We found that learning and predicting was accurate at package level, but not at product level. These results suggest that to learn from past defects, one should separate the product into component clusters with similar functionality, and make separate predictions for each cluster. Our initial approaches to form such clusters automatically, based on similarity of metrics, showed promising accuracy.

Reference

BibTeX Entry

@techreport{herzig-abap-prediction-2008,
 title = "Predicting Defects in SAP Products: A Replicated Study",
 author = "Kim Herzig and Rahul Premraj and Thomas Zimmermann and Andreas Zeller and Juergen Heymann",
 institution = "Software Engineering Chair, Saarland University"
}

Download PDF version

This thesis contains a lock flag. I am not allowed t publish this thesis neither completely nor in parts. The handout of this thesis to third persons requires a written approval of SAP.