arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.23635cs.HC
收藏

当LLMs助益与损害助教在证明基础课程中的表现

When LLMs Help -- and Hurt -- Teaching Assistants in Proof-Based Courses

Princeton University(普林斯顿大学)

Romina Mahinpei, Sofiia Druchyna, Manoel Horta Ribeiro

AI总结:

本文研究了LLM在证明基础课程中助教评分与反馈中的作用,发现LLM与助教在评分上存在分歧,但其反馈对存在重大错误的提交仍具帮助。

摘要:

助教在证明基础课程中承担着评分和反馈提供的重要职责,然而这些任务耗时且难以扩展。尽管大型语言模型(LLMs)已被研究用于评分和反馈,但其在证明基础课程中的有效性仍未知。在设计针对此情境的LLM系统之前,有必要先确定LLMs是否能有意义地协助助教进行评分和反馈。为此,我们呈现了一个多部分案例研究,作为一项技术探测,在本科生证明基础课程中进行。我们比较了基于评分标准的评分决策由LLM和不同专业水平的助教所做出,并检查助教对LLM生成反馈的看法。我们发现LLMs与助教在评分决策上存在显著分歧,但LLM生成的反馈仍能对存在重大错误的提交有所帮助。最后,我们讨论了在证明基础课程中的人工智能评分和反馈系统的设计影响。

英文摘要:

Teaching assistants (TAs) are essential to grading and feedback provision in proof-based courses, yet these tasks are time-intensive and difficult to scale. Although Large Language Models (LLMs) have been studied for grading and feedback, their effectiveness in proof-based courses is still unknown. Before designing LLM-based systems for this context, a necessary prerequisite is to understand whether LLMs can meaningfully assist TAs with grading and feedback. As such, we present a multi-part case study functioning as a technology probe in an undergraduate proof-based course. We compare rubric-based grading decisions made by an LLM and TAs with varying levels of expertise and examine TAs' perceptions of feedback generated by an LLM. We find substantial disagreement between LLMs and TAs on grading decisions but that LLM-generated feedback can still be useful to TAs for submissions with major errors. We conclude by discussing design implications for human-AI grading and feedback systems in proof-based courses.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑