控制L2英语口语自动评分器中对隐含捷径的依赖
Controlling Implicit Shortcut Reliance in L2 Spoken English Auto-markers
浏览论文内容
中文总结 AI 辅助
研究语言能力评估中自动评分器过度依赖输入特定方面的问题，提出新颖训练标准减少分类器对捷径的依赖，通过在两种评估系统上的实验，降低了与不当行为相关特征的相关性。
中文摘要 AI 辅助
越来越多的语音和语言处理任务直接采用音频或文本，而非从中提取特征作为分类器或回归器的输入。这些系统常运用复杂过程，能在输入与输出间得出高度非线性映射。但不幸的是，它们也会学习‘捷径’，导致分类器过度依赖输入特定方面来产生输出。对于语言能力评估任务，这种过度依赖会让学习者通过利用捷径而非提升能力来提高分数。本文引入一种新颖训练标准，能减少分类器对捷径的依赖，比如限制语言评估中这种不当行为。此过程在两种评估系统上展示，一种基于音频，另一种基于语音识别文本。结果表明，对于这两种系统，与可被用于不当行为的特征的相关性高于人类参考预期，显示出对这些特征的过度依赖。通过引入修改后的训练标准，这种相关性可降低至更接近参考相关性。
英文摘要
Increasingly, speech and language processing tasks take either audio or text directly rather than extracting features from these as the input to the classifier or regressor. Often these systems make use of complex, for example transformer-based, processes that have the ability to derive highly non-linear mappings between the input and the output. Unfortunately these systems can also learn ''shortcuts'' where the classifier is overly reliant on particular aspects of the input to yield the output. For the task of language proficiency assessment, this over-reliance can enable learners to increase their score by exploiting the shortcut rather than improving their ability. This paper introduces a novel training criterion that is able to reduce the classifier's reliance on shortcuts, thus for example limiting this option for malpractice in language assessment. This process is illustrated on two forms of assessment system, one based on the audio the other on the speech recognition text. The results show that, for both systems, there is higher correlations with features that could be exploited for malpractice than expected from the human reference, indicating an over-reliance on these features. By introducing the modified training criterion, this correlation can be reduced to be closer to the reference correlation.
