The Impact of Test Ownership and Team Structure on the Reliability and Effectiveness of Quality Test Runs @ESEM 2014

Software testing is a crucial step in most software development processes. Testing software is a key component to manage and assess the risk of shipping quality products to customers. But testing is also an expensive process and changes to the system need to be tested thoroughly which may take time. Thus, the quality of a software product depends on the quality of its underlying testing process and on the effectiveness and reliability of individual test cases. In this paper, we investigate the impact of the organizational structure of test owners on the reliability and effectiveness of the corresponding test cases. Prior empirical research on organizational structure has focused only on developer activity. We expand the scope of empirical knowledge by assessing the impact of organizational structure on testing activities. We performed an empirical study on the Windows build verification test suites (BVT) and relate efficiency and reliability measures of each test run to the complexity and size of the organizational sub-structure that enclose all owners of test cases executed. Our results show, that organizational structure impacts both test efficiency and test execution reliability. We are also able to predict effectiveness and reliability with fairly high precision and recall values.

  • [PDF] [DOI] K. Herzig and N. Nagappan, “The impact of test ownership and team structure on the reliability and effectiveness of quality test runs,” in Proceedings of the 8th acm/ieee international symposium on empirical software engineering and measurement, New York, NY, USA, 2014, p. 2:1–2:10.
    author = {Herzig, Kim and Nagappan, Nachiappan},
    title = {The Impact of Test Ownership and Team Structure on the Reliability and Effectiveness of Quality Test Runs},
    booktitle = {Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement},
    series = {ESEM '14},
    year = {2014},
    isbn = {978-1-4503-2774-9},
    location = {Torino, Italy},
    pages = {2:1--2:10},
    articleno = {2},
    numpages = {10},
    pdf = {},
    doi = {10.1145/2652524.2652535},
    acmid = {2652535},
    publisher = {ACM},
    address = {New York, NY, USA},
    keywords = {effectiveness, empirical software engineering, organizational structure, reliability, software testing},