arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2107.07957cs.CLcs.AI
收藏

基于机器阅读理解的任务需求写作自动评估

Automatic Task Requirements Writing Evaluation via Machine Reading Comprehension

TAL Education Group(好未来教育集团)

Shiting Xu, Guowei Xu, Peilei Jia, Wenbiao Ding, Zhongqin Wu, Zitao Liu

摘要:

任务要求写作是英语四级考试和英语初试中的一个重要题型。TR写作问题可能包括多个要求,高质量的论文必须全面准确地回应每个要求。然而,有限的教师资源使学生无法立即获得详细的评分。针对现有作文自动评分系统大多注重整体评分而很少提供理由支持的问题,提出了一个基于机器阅读理解(MRC)的端到端框架。该框架不仅检测文章是否回答了需求问题,而且清楚地标记了文章回答问题的位置。该框架包括三个模块:问题规范化模块、基于ELECTRA的MRC模块和响应定位模块。我们广泛探索最先进的MRC方法。我们的方法在真实世界的教育数据集上实现了0.93的准确性得分和0.85的F1得分。为了鼓励可重复的结果,我们在\url{https://github.com/aied2021TRMRC/AIED_2021_TRMRC_code}上公开了我们的代码。

英文摘要:

Task requirements (TRs) writing is an important question type in Key English Test and Preliminary English Test. A TR writing question may include multiple requirements and a high-quality essay must respond to each requirement thoroughly and accurately. However, the limited teacher resources prevent students from getting detailed grading instantly. The majority of existing automatic essay scoring systems focus on giving a holistic score but rarely provide reasons to support it. In this paper, we proposed an end-to-end framework based on machine reading comprehension (MRC) to address this problem to some extent. The framework not only detects whether an essay responds to a requirement question, but clearly marks where the essay answers the question. Our framework consists of three modules: question normalization module, ELECTRA based MRC module and response locating module. We extensively explore state-of-the-art MRC methods. Our approach achieves 0.93 accuracy score and 0.85 F1 score on a real-world educational dataset. To encourage reproducible results, we make our code publicly available at \url{https://github.com/aied2021TRMRC/AIED_2021_TRMRC_code}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑