The book Making Software is published now also available in Japan language. The article written by Andreas Zeller and myself is translated as 自力でのエビデンスマイニング (Mine your own evidence).

Thanks to O’Reilly Japan and the translators for translating this great book.

Software reliability is heavily impacted by soft- ware changes. How do these changes relate to each other? By analyzing the impacted method definitions and usages, we determine dependencies between changes, resulting in a change genealogy that captures how earlier changes enable and cause later ones. Model checking this genealogy reveals temporal process patterns that encode key features of the software process such as pending development activities: “Whenever class A is changed, its test case is later updated as well.” Such patterns can be validated automatically: In an evaluation of four open source histories, our prototype would recommend pending activities with a precision of 60–72%.

Reference

BibTeX Entry

@inproceedings{herzig-issre-2011,
 title = {Mining Cause-Effect-Chains from Version Histories}, 
   author={Kim Herzig and Andreas Zeller}, 
   year = {2011}, 
   month = {November}, 
   booktitle={Proceedings of the 22nd International Symposium on Software Reliability Engineering} 
   }

Several defect prediction models have been proposed to identify which entities in a software system are likely to have defects before its release. This paper presents a replication of one such study conducted by Zimmermann and Nagappan [1] on Windows Server 2003 where the authors leveraged dependency relationships between software entities captured using social network metrics to predict whether they are likely to have defects. They found that network metrics perform significantly better than source code metrics at predicting defects. In order to corroborate the generality of their findings, we replicate their study on three open source Java projects, viz., JRuby, ArgoUML, and Eclipse. Our results are in agreement with the original study by Zimmermann and Nagappan when using a similar experimental setup as them (random sampling). However, when we evaluated the metrics using setups more suited for industrial use – forward-release and cross-project prediction – we found network metrics to offer no vantage over code metrics. Moreover, code metrics may be preferable to network metrics considering the data is easier to collect and we used only 8 code metrics compared to approximately 58 network metrics.

Reference

BibTeX Entry

@inproceedings{premraj-esem-2011,
 title = {Network versus Code Metrics to Predict Defects: A Replication Study}, 
   author={Premraj, Rahul and Herzig, Kim}, 
   year = {2011}, 
   month = {september}, 
   booktitle={Proceedings of the Fifth International Symposium on Empirical Software Engineering and Measurement}
}

Download PDF version

LaTeX is great and there is no better environment to write professional documents. But spell-checking text within LaTeX is difficult and all projects trying to solve this issue failed at various levels. One of the tools to check for spelling and grammar errors is Microsoft Word. To use MS Word to spell and grammar check LaTeX documents, one has to convert the LaTeX documents to a format readable by MS Word: Rich Text Format (RTF). The latex2rtf project at Sourceforge is transforming LaTeX document to RTF documents. Opening the RTF document using Word allows you to spell and grammar check your document.

There are some limitations to latex2rtf. It cannot handle LaTeX environments like \usepackage{boxenminipage}. For more advanced LaTeX documents I normally stick to Acrobat Professional converting the compiled PDF documents into RTF or MS Word format directly.

At our chair we have a couple of interesting Bachelor-/Master- Theses pending. In my case I have multiple ideas for topics regarding Mining Software Repositories and graphical user interfaces for recommendation tools. Anyone who is interested is welcome to contact me.

Changing source code in large software systems is complex and requires a good understanding of dependencies between software components. Modification to components with little regard to dependencies may have an adverse impact on the quality of the latter, i.e., increase their risk to fail. We conduct an empirical study to understand the relationship between the quality of components and the characteristics of their dependencies such as their frequency of change, their complexity, number of past failures and the like. Our study has been conducted on two large software systems: Microsoft VISTA and ECLIPSE. Our results show that components that have outgoing dependencies to components with higher object-oriented complexity tend to have fewer field failures for VISTA, but the opposite relation holds for ECLIPSE. Likewise, other notable observations have been made through our study that (a) confirm that certain characteristics of components increase the risk of their dependencies to fail and (b) some of the characteristics are project specific while some were also found to be common. We expect that such results can be leveraged for use to provide new directions for research in defect prediction, test prioritization and related research fields that utilize code dependencies in their empirical analysis. Additionally, these results provide insights to engineers on the potential reliability impacts of new component dependencies based upon the characteristics of the component.

Reference

BibTeX Entry

@inproceedings{zimmermann-icst-2011,
 title = {An Empirical Study of the Factors Relating Field Failures and Dependencies}, 
   author={Zimmerman, Thomas and Nagappan, Nachiappan and Herzig, Kim and Premraj, Rahul and Williams, Laurie}, 
   year = {2011}, 
   month = {march}, 
   booktitle={Software Testing, Verification and Validation (ICST), 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation}
 pages={347--356}, 
   doi={10.1109/ICST.2011.39}
}

Download PDF version

Throughout this book, you will find examples of how to gather evidence—evidence on the effectiveness of testing, the quality of bug reports, the role of complexity metrics, and so on. But do these findings actually apply to your project? The definite way to find this out is to repeat the appropriate study on your data, in your environment. This way, you will not only gather lots of insight into your own project; you will also experience the joys of experimental research. Unfortunately, you may also encounter the downside: empirical studies can be very expensive, in particular if they involve experiments with developers.


Fortunately, there is a relatively inexpensive way to gather lots of evidence about your project. Software archives, such as version or bug repositories, record much of the activity around your product, in terms of problems occurring, changes made, and problems fixed. By mining these archives automatically, you can obtain lots of initial evidence about your product—evidence that already is worthy in itself, but which may also pave the path toward further experiments and further insights. In this chapter, we give a hands-on tutorial into mining software archives, covering both the basic technical steps and possible pitfalls that you may encounter on the way.

See also

http://oreilly.com/catalog/9780596808303

Screencast

Reference

BibTeX Entry

@inbook{herzig-makingsoftware-2010,
 title = "Mining Your Own Evidence",
 author = "Kim Herzig and Andreas Zeller",
 year = "2010",
 month = "October",
 booktitle = "Making Software: What Really Works, and Why We Believe It",
 chapter = "Mining Your Own Evidence",
 editors = "Andy Oram and Greg Wilson",
 pages = "517--529",
 publisher = "O'Reilly Media",
 ISBN = "9780596808327"
}

Google rewarded Andreas Zeller‘s research proposal was successful. Google rewarded our chair with a Google Research Award. The money funds my new research project conducted with and within Google.

Adrian Schröter asked me to be MSR 2011 challenge committee member. Thanks Adrian!

In software development, every change induces a risk. What happens if code changes again and again in some period of time? In an empirical study on Windows Vista, we found that the features of such change bursts have the highest predictive power for defect-prone components. With precision and recall values well above 90%, change bursts significantly improve upon earlier predictors such as complexity metrics, code churn, or organizational structure. As they only rely on version history and a controlled change process, change bursts are straight-forward to detect and deploy.

Reference

BibTeX Entry

@inproceedings{nagappan-issre-2010,
 title = "Change Bursts as Defect Predictors",
 author = "Nachiappan Nagappan and Andreas Zeller and Thomas Zimmermann and Kim Herzig and Brendan Murphy",
 year = "2010",
 month = "November",
 booktitle = "Proceedings of the 21st IEEE International Symposium on Software Reliability Engineering",
 location = "San Jose, California
}

Download PDF version