多项选择题阅读理解的题目难度排序
Question Difficulty Ranking for Multiple-Choice Reading Comprehension
AI总结：
针对多项选择阅读理解题人工难度排序成本高、可扩展性差且训练数据有限的问题，对比任务迁移与零样本难度排序方法，发现零样本相对评估效果最优，系统融合可进一步提升排序相关性。
摘要：
多项选择（MC）测试是评估英语学习者的一种高效方法。在考试命题过程中，按难度对候选MC题目进行排序对出题者很有帮助。通常情况下，难度是通过让人类考生在预测试阶段试做题目来确定的，但这种方法成本高昂且不具备可扩展性。因此，我们探索用于MC题目难度排序的自动化方法。然而，可用于显式训练难度评分系统的数据十分有限。为此，我们对比了任务迁移和零样本方法：任务迁移将水平分类和阅读理解系统适配用于难度排序，而针对指令微调语言模型的零样本提示法则对比了绝对评估与相对评估。研究发现，水平分类的迁移效果优于阅读理解。此外，零样本相对评估在难度排序任务上比绝对评估更有效，甚至优于任务迁移方法，其斯皮尔曼相关系数达到40.4%。研究还观察到，将这些系统结合可进一步提升相关系数。
英文摘要：
Multiple-choice (MC) tests are an efficient method to assess English learners. It is useful for test creators to rank candidate MC questions by difficulty during exam curation. Typically, the difficulty is determined by having human test takers trial the questions in a pretesting stage. However, this is expensive and not scalable. Therefore, we explore automated approaches to rank MC questions by difficulty. However, there is limited data for explicit training of a system for difficulty scores. Hence, we compare task transfer and zero-shot approaches: task transfer adapts level classification and reading comprehension systems for difficulty ranking while zero-shot prompting of instruction finetuned language models contrasts absolute assessment against comparative. It is found that level classification transfers better than reading comprehension. Additionally, zero-shot comparative assessment is more effective at difficulty ranking than the absolute assessment and even the task transfer approaches at question difficulty ranking with a Spearman's correlation of 40.4%. Combining the systems is observed to further boost the correlation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
