Computer Science · Information Systems
Seminar “Research Methods”, Summer Term 2005 The Role of Experimentation in Software Engineering
See on the mapAbstract
Research proposals need to be validated either by formal proofs or by applying empirical methods (e.g. controlled experiments). Many authors have pointed out that the level of experimentation in software engineering is not satisfactory. The quantity of experimentation is too low as a lot of software engineering publication do not contain any empirical validation at all. But also the quality of software engineering experiments conducted so far is often weak, because no proper methodological approach is applied and statistical methods are misused. This paper provides an overview of the status of experimentation in software engineering. First, the role of experimentation among other types of research is clarified. Several research paradigms are introduced, a classification of different types of experiments in software engineering is provided, and a comparison with experimentation in other research disciplines is drawn. Afterwards the current state of experimentation in software engineering is analysed with more detail. Some discussion points from various researchers about the situation of experimentation are summed up. Their recommendations for improving the state of experimentation are provided as well as possible future directions of experimentation in software engineering. 1