Validity Threats in Empirical Software Engineering Research - An Initial Survey
by R. Feldt and A. Magazinius
PDF
CPL Entry
In judging the quality of a research study it is very important to consider threats to the validity of the study and the results. This is particularly important for empirical research where there is often a multitude of possible threats. With a growing focus on empirical research methods in software engineering it is important that there is a consensus in the community on this importance, that validity analysis is done by every researcher and that there is common terminology and support on how to do and report it. Even though there are previous relevant results they have primarily focused on quantitative research methods and in particular experiments. Here we look at the existing advice and guidelines and then perform a review of 43 papers published in the ESEM conference in 2009 and analyse the validity analysis they include and which threats and strategies for overcoming them that were given by the authors. Based on this analysis we then discuss what is working well and less well in validity analysis of empirical software engineering research and present recommendations on how to better support validity analysis in the future.
Bibtex
@InProceedings{Feldt2010ValThreatsSurvey,
author = "Robert Feldt and Ana Magazinius",
title = {Validity Threats in Empirical Software Engineering Research - An Initial Survey},
year = "2010",
booktitle = "Proceedings of the Software Engineering and Knowledge Engineering Conference",
pages = "374--379",
address = "Redwood City, San Fransisco Bay, CA, USA",
keywords = "Software Engineering; Empirical study; Validity threats",
url = "http://www.cse.chalmers.se/~feldt/publications/feldt_2010_valthreats_survey.html",
}