Automated Short-Response Scoring for Automated Item Generation in Science Assessments

Abstract

This chapter provides an introduction and an overview of the pipeline that combines and extends automated essay scoring capacity with an automated item generation framework. The chapter specifically focuses on the automated short-answer scoring system that provides feedback for short constructed-response items. The form of feedback provided by the current automated short-answer scoring systems may include simple numeric scoring but also can be enriched with more complex graphical or semantic suggestions. We introduce a novel demonstration to automatically generate follow-up items as a form of feedback using the automated short answer scoring results. We first score the responses collected from short constructed-response items from a large-scale science assessment. We then identify common misconceptions that are aligned with each scoring category. A pre-trained automated item generation system using large-scale neural language models (e.g., GPT-3.4 and Text-Davinci-003) was adopted to demonstrate the generation of follow-up items that are associated with students’ common misconceptions using the creative prompt generation by Yuan et al. (2022). The chapter concludes by introducing the future directions to advance the current automated short-answer scoring systems feedback using automated item generation in other disciplinary domains.