Syntax error based quantification of the learning progress of the novice programmer

Abstract

Recent data-driven research has produced metrics for quantifying a novice programmer's error profile, such as Jadud's error quotient. However, these metrics tend to be context dependent and contain free parameters. This paper reviews the caveats of such metrics and proposes a more general approach to developing a metric. The online implementation of the proposed metric is publicly available at http://online-analysis-demo.herokuapp.com/.