SCO-DUMA: Reading Comprehension with Looking Origin and Comparing Options

Abstract

Multiple-choice reading comprehension (MCRC) is a task of choosing the correct option from candidates given an article and a question. Recent researchs propose a new model with combining a part-of-speech (POS) embedding layer in the pre-training language model and a DUal Multi-head Attention (DUMA) matching layer to get the state-of-the-art. However, there is still distance between the model’s performance and human’s performance. We propose two mechanisms base on DUMA to futher advance the performance. In the interaction layer, we propose a option interaction mechanism and a gated self-comparing mechanism to imitate the human’s behavior of re-reading passage and comparing the options. Our mechanisms on ALBERT-Base has a 1.4% improvement on DREAM and a 2.1% improvement on RACE.