Searching for Cognitively Diverse Tests: Towards Universal Test Diversity Metrics
by R. Feldt, R. Torkar, T. Gorschek and W. Afzal
PDF
Search-based software testing (SBST) has shown a potential to decrease cost and increase quality of testing-related software development activities. Research in SBST has so far mainly focused on the search for isolated tests that are optimal according to a fitness function that guides the search. In this paper we make the case for fitness functions that measure test fitness in relation to existing or previously found tests; a test is good if it is diverse from other tests. We present a model for test variability and propose the use of a theoretically optimal diversity metric at variation points in the model. We then describe how to apply a practically useful approximation to the theoretically optimal metric. The metric is simple and powerful and can be adapted to a multitude of different test diversity measurement scenarios. We present initial results from an experiment to compare how similar to human subjects, the metric can cluster a set of test cases. To carry out the experiment we have extended an existing framework for test automation in an object-oriented, dynamic programming language.
Bibtex
@InProceedings{Feldt2008TestDiversity,
author = "Robert Feldt and Richard Torkar and Tony Gorschek and Wasif Afzal",
title = {"Searching for Cognitively Diverse Tests: Towards Universal Test Diversity Metrics"},
booktitle = "Proceedings of 1st Search-Based Software Testing Workshop (SBST'08)",
year = "2008",
pages = "178--186",
keywords = "Verification and Validation; Testing; Data Mining",
url = "http://www.cse.chalmers.se/~feldt/publications/feldt_2008_sbst_universal_test_diversity.html",
}