arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Computer Science · Information Systems

Seminar “Research Methods”, Summer Term 2005 The Role of Experimentation in Software Engineering

Heiko Koziolek

See on the map

Abstract

Research proposals need to be validated either by formal proofs or by applying empirical methods (e.g. controlled experiments). Many authors have pointed out that the level of experimentation in software engineering is not satisfactory. The quantity of experimentation is too low as a lot of software engineering publication do not contain any empirical validation at all. But also the quality of software engineering experiments conducted so far is often weak, because no proper methodological approach is applied and statistical methods are misused. This paper provides an overview of the status of experimentation in software engineering. First, the role of experimentation among other types of research is clarified. Several research paradigms are introduced, a classification of different types of experiments in software engineering is provided, and a comparison with experimentation in other research disciplines is drawn. Afterwards the current state of experimentation in software engineering is analysed with more detail. Some discussion points from various researchers about the situation of experimentation are summed up. Their recommendations for improving the state of experimentation are provided as well as possible future directions of experimentation in software engineering. 1

More like this