日语考试中描述性答案的手写识别与自动评分
Handwriting recognition and automatic scoring for descriptive answers in Japanese language tests
AI总结：
本文针对日本大学入学考试中约40万份未标注手写描述性答案，提出结合多训练策略、集成识别器和通用语言模型的深度手写识别方法，字符准确率超97%，并接入BERT自动评分系统取得0.84至0.98的QWK，验证了端到端自动评分的可行性。
摘要：
本文介绍了一项针对新日本大学入学考试模拟测试中手写描述性答案进行自动评分的实验。这些模拟测试于2017年和2018年面向约12万名考生实施，共收集了约40万份答案，包含超过2000万个字符。尽管所有答案均已由人工阅卷员评分，但手写字符并未进行标注。我们展示了将基于标注手写数据集训练的深度神经网络手写识别器适配到这一未标注答案集上的尝试。所提出的方法结合了不同的训练策略，集成了多个识别器，并使用由大规模通用语料库构建的语言模型，以避免对特定数据产生过拟合。在实验中，该方法使用约2000份经过验证的标注答案（占数据集不到0.5%），达到了超过97%的字符准确率。随后，识别后的答案被输入一个基于BERT模型的预训练自动评分系统，且未纠正误识别字符，也未提供评分标准标注。该自动评分系统取得的二次加权Kappa（QWK）值在0.84至0.98之间。由于QWK超过0.8，这表明自动评分系统与人工阅卷员之间的评分相似性是可接受的。这些结果为进一步研究描述性答案的端到端自动评分提供了有前景的基础。
英文摘要：
This paper presents an experiment of automatically scoring handwritten descriptive answers in the trial tests for the new Japanese university entrance examination, which were made for about 120,000 examinees in 2017 and 2018. There are about 400,000 answers with more than 20 million characters. Although all answers have been scored by human examiners, handwritten characters are not labeled. We present our attempt to adapt deep neural network-based handwriting recognizers trained on a labeled handwriting dataset into this unlabeled answer set. Our proposed method combines different training strategies, ensembles multiple recognizers, and uses a language model built from a large general corpus to avoid overfitting into specific data. In our experiment, the proposed method records character accuracy of over 97% using about 2,000 verified labeled answers that account for less than 0.5% of the dataset. Then, the recognized answers are fed into a pre-trained automatic scoring system based on the BERT model without correcting misrecognized characters and providing rubric annotations. The automatic scoring system achieves from 0.84 to 0.98 of Quadratic Weighted Kappa (QWK). As QWK is over 0.8, it represents an acceptable similarity of scoring between the automatic scoring system and the human examiners. These results are promising for further research on end-to-end automatic scoring of descriptive answers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
