Software reliability is heavily impacted by soft- ware changes. How do these changes relate to each other? By analyzing the impacted method definitions and usages, we determine dependencies between changes, resulting in a change genealogy that captures how earlier changes enable and cause later ones. Model checking this genealogy reveals temporal process patterns that encode key features of the software process such as pending development activities: “Whenever class A is changed, its test case is later updated as well.” Such patterns can be validated automatically: In an evaluation of four open source histories, our prototype would recommend pending activities with a precision of 60–72%.

Reference

BibTeX Entry

@inproceedings{herzig-issre-2011,
 title = {Mining Cause-Effect-Chains from Version Histories}, 
   author={Kim Herzig and Andreas Zeller}, 
   year = {2011}, 
   month = {November}, 
   booktitle={Proceedings of the 22nd International Symposium on Software Reliability Engineering} 
   }

Several defect prediction models have been proposed to identify which entities in a software system are likely to have defects before its release. This paper presents a replication of one such study conducted by Zimmermann and Nagappan [1] on Windows Server 2003 where the authors leveraged dependency relationships between software entities captured using social network metrics to predict whether they are likely to have defects. They found that network metrics perform significantly better than source code metrics at predicting defects. In order to corroborate the generality of their findings, we replicate their study on three open source Java projects, viz., JRuby, ArgoUML, and Eclipse. Our results are in agreement with the original study by Zimmermann and Nagappan when using a similar experimental setup as them (random sampling). However, when we evaluated the metrics using setups more suited for industrial use – forward-release and cross-project prediction – we found network metrics to offer no vantage over code metrics. Moreover, code metrics may be preferable to network metrics considering the data is easier to collect and we used only 8 code metrics compared to approximately 58 network metrics.

Reference

BibTeX Entry

@inproceedings{premraj-esem-2011,
 title = {Network versus Code Metrics to Predict Defects: A Replication Study}, 
   author={Premraj, Rahul and Herzig, Kim}, 
   year = {2011}, 
   month = {september}, 
   booktitle={Proceedings of the Fifth International Symposium on Empirical Software Engineering and Measurement}
}

Download PDF version

Changing source code in large software systems is complex and requires a good understanding of dependencies between software components. Modification to components with little regard to dependencies may have an adverse impact on the quality of the latter, i.e., increase their risk to fail. We conduct an empirical study to understand the relationship between the quality of components and the characteristics of their dependencies such as their frequency of change, their complexity, number of past failures and the like. Our study has been conducted on two large software systems: Microsoft VISTA and ECLIPSE. Our results show that components that have outgoing dependencies to components with higher object-oriented complexity tend to have fewer field failures for VISTA, but the opposite relation holds for ECLIPSE. Likewise, other notable observations have been made through our study that (a) confirm that certain characteristics of components increase the risk of their dependencies to fail and (b) some of the characteristics are project specific while some were also found to be common. We expect that such results can be leveraged for use to provide new directions for research in defect prediction, test prioritization and related research fields that utilize code dependencies in their empirical analysis. Additionally, these results provide insights to engineers on the potential reliability impacts of new component dependencies based upon the characteristics of the component.

Reference

BibTeX Entry

@inproceedings{zimmermann-icst-2011,
 title = {An Empirical Study of the Factors Relating Field Failures and Dependencies}, 
   author={Zimmerman, Thomas and Nagappan, Nachiappan and Herzig, Kim and Premraj, Rahul and Williams, Laurie}, 
   year = {2011}, 
   month = {march}, 
   booktitle={Software Testing, Verification and Validation (ICST), 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation}
 pages={347--356}, 
   doi={10.1109/ICST.2011.39}
}

Download PDF version

Developers change source code to add new functionality, fix bugs, or refactor their code. Many of these changes have immediate impact on quality or stability. However, some impact of changes may become evident only in the long term. The goal of this thesis is to explore the long-term impact of changes by detecting dependencies between code changes and by measuring their influence on software quality, software maintainability, and development effort. Being able to identify the changes with the greatest long-term impact will strengthen our understanding of a project’s history and thus shape future code changes and decisions.

Reference

BibTeX Entry

@inproceedings{herzig-icse-2010,
 author = "Herzig, Kim Sebastian",
 title = "Capturing the long-term impact of changes",
 booktitle = "ICSE '10: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering",
 year = "2010",
 isbn = "978-1-60558-719-6",
 pages = "393--396",
 location = "Cape Town, South Africa",
 doi = "http://doi.acm.org/10.1145/1810295.1810401",
 publisher = "ACM",
 address = "New York, NY
}

Download PDF version