Software archives mining deals with the automated extraction, collection, and abstraction of data from the information generated during the software development process (e.g. source code archives, bug tracking systems, etc.). This seminar (7 CP) introduces the notion of software archives and teaches recent software archives mining techniques. More details here

By integrating various development and collaboration tools into one single platform, the Jazz environment offers several opportunities for software repository miners. In particular, Jazz offers full traceability from the initial requirements via work packages and work assignments to the final changes and tests; all these features can be easily accessed and leveraged for better prediction and recommendation systems. In this paper, we share our initial experiences from mining the Jazz repository. We also give a short overview of the retrieved data sets and discuss possible problems of the Jazz repository and the platform itself.

Reference

BibTeX Entry

@inproceedings{herzig-msr-2009,
 title = "Mining the Jazz Repository: Challenges and Opportunities",
 author = "Kim Herzig and Andreas Zeller",
 year = "2009",
 month = "May",
 booktitle = "Proceedings of the 6th IEEE Working Conference on Mining Software Repositories (MSR09)",
 location = "Vancouver, BC, Canada"
}

Which components of a large software system are the most defect-prone? In a study on a large SAP Java system, we evaluated and compared a number of defect predictors, based on code features such as complexity metrics, static error detectors, change frequency, or component imports, thus replicating a number of earlier case studies in an industrial context. We found the overall predictive power to be lower than expected; still, the resulting regression models successfully predicted 50-60% of the 20% most defect-prone components.

Reference

BibTeX Entry

@inproceedings{holschuh-icse-2009,
 title = "Predicting Defects in SAP Java Code: An Experience Report",
 author = "Tilman Holschuh and Markus Paeuser and Kim Herzig and Thomas Zimmermann and Rahul Premraj and Andreas Zeller",
 year = "2009",
 month = "May",
 booktitle = "Proceedings of the 31th International Conference on Software Engineering",
 location = "Vancouver, BC
}

Download PDF version

Given a large body of code, how do we know where to focus our quality assurance effort? By mining the software’s defect history, we can automatically learn which code features correlated with defects in the past—and leverage these correlations for new predictions: “In the past, high inheritance depth was an indicator of a high number of defects. Since this new component also has a high inheritance depth, let us test it thoroughly”. Such history-based approaches work best if the new component is similar to the components learned from. But how does learning from history perform for projects with high variability between components? We ran a study on two SAP products involving a wide spectrum of functionality. We found that learning and predicting was accurate at package level, but not at product level. These results suggest that to learn from past defects, one should separate the product into component clusters with similar functionality, and make separate predictions for each cluster. Our initial approaches to form such clusters automatically, based on similarity of metrics, showed promising accuracy.

Reference

BibTeX Entry

@techreport{herzig-abap-prediction-2008,
 title = "Predicting Defects in SAP Products: A Replicated Study",
 author = "Kim Herzig and Rahul Premraj and Thomas Zimmermann and Andreas Zeller and Juergen Heymann",
 institution = "Software Engineering Chair, Saarland University"
}

Download PDF version

Software products and projects can become very large and still grow over time. Building one prediction model for a whole software product might be easy but might also limit the prediction accuracy.

Different parts of a software product have different duties (GUI, database, kernel,…). We found out that for each of these different code zones there exist different code characteristics that matter when it comes to defect prediction. The main idea is to cluster code entities by their software duty. Currently we are investigating whether defect prediction taking advantage of such clustering techniques have a higher prediction accuracy than defect prediction models using no clustering technique.

One major down side of many defect prediction models is the frequent usability. As an example: A prediction model based on code metrics only, predicts the number of defects based upon the code complexity of each entity. Now, if the programmer fixes the bug, he might change only one line and therefore does not change the code complexity significantly. The complexity might even increase. Using the same defect prediction model again gives us the same result, because the basis of the defect prediction model did not changed or did not changed significantly.

To overcome such problems, we could use a dynamic approach. But most dynamic approaches cannot predict any defects before defects have been discovered.

The idea is to use different time related properties of a code entity to determine the age of this entity. The main assumption is that older entities have fewer defects. Defining the term older entities is the main task of this project.

Defect predicion is a research with high activity. A lot of work has been done in this area, also in recent months. But even if there exist a lot of approaches and techniques to predict defects and bugs accurately, many of the static approaches suffer from a fundamental flaw. 

Most static approaches rely on source code metrics (such as McCabe), only. The assumption is that the more complex source code is the more difficulties the developer will encounter when changing the source code entity. An indeed, there are many static prediction approaches that work with high accuracy. But how long? 

Let’s assume that we have a very accurate prediction model. Now let’s use it. We determine the top 20% of source code entities with the highest defect likelihood. After intense testing, debugging and fixing we are quite sure that there are no more defects left. Now we ask the prediction model again to gain the new top 20%. And now it happens. The prediction model will most likely return the same set of source code entities. 

The reason for this is that fixing a defect will not significantly decrease the source code complexity. Even worse, many fixes will increase the complexity. What now? 

This is one of the problems I’m currently working on. The solution is not that simple but we are working on it.