利用解决方案生成的自动评分器加速准确作业的创作
Accelerating Accurate Assignment Authoring Using Solution-Generated Autograders
浏览论文内容
中文总结 AI 辅助
本文提出解决方案生成的自动评分方法，实现了更快速准确的自动评分器创作，其实现系统Questioner已用于CS1课程创作近800道编程题，评估数百万次学生提交。
中文摘要 AI 辅助
学习编程的学生能从大量练习题目中获益，自动评分器常被用于为编程题目提供对提交内容的快速反馈。但创作准确的自动评分器仍具挑战性，自动评分器常通过枚举测试用例创建，这一过程繁琐，可能产生无法正确分类提交内容的不准确自动评分器。当创作准确自动评分器速度缓慢时，难以创建大量练习题库以支持入门程序员。本文提出解决方案生成的自动评分：一种更快、更准确且更易用的自动评分器创作方式。该方法利用软件测试与自动评分的关键差异：题目创作者可提供一个解决方案，从解决方案出发，无需手动枚举测试用例、验证自动评分器的准确性，也无需评估提交代码除行为正确性外的其他质量方面。本文介绍了Questioner，这是针对Java和Kotlin实现的解决方案生成自动评分系统，并分享了使用Questioner支持一门大型CS1课程的四年经验：创作了近800道编程题目，供数千名学生使用，评估了数百万次提交内容。
英文摘要
Students learning to program benefit from access to large numbers of practice problems. Autograders are commonly used to support programming questions by providing quick feedback on submissions. But authoring accurate autograders remains challenging. Autograders are frequently created by enumerating test cases--a tedious process that can produce inaccurate autograders that fail to correctly classify submissions. When authoring accurate autograders is slow, it is difficult to create large banks of practice problems to support beginning programmers. We present solution-generated autograding: a faster, more accurate, and more enjoyable way to create autograders. Our approach leverages a key difference between software testing and autograding: The question author can provide a solution. By starting with a solution, we can eliminate the need to manually enumerate test cases, validate the autograder's accuracy, and evaluate other aspects of submission code quality beyond behavioral correctness. We describe Questioner, an implementation of solution-generated autograding for Java and Kotlin, and share experiences from four years using Questioner to support a large CS1 course: authoring nearly 800 programming questions used by thousands of students to evaluate millions of submissions.
