arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.08058cs.CLcs.AIcs.LG
收藏

闭合回路:通过语言模型模拟学生修订来学习生成写作反馈

Closing the Loop: Learning to Generate Writing Feedback via Language Model Simulated Student Revisions

University of Michigan(密歇根大学);ETH Zürich(苏黎世联邦理工学院)

Inderjeet Nair, Jiaye Tan, Xiaotian Su, Anne Gere, Xu Wang, Lu Wang

AI总结:

本文提出PROF框架,通过语言模型模拟学生修订过程来迭代优化反馈生成器,直接最大化学生修订表现,实证表明其在提升写作质量和教学价值上优于基线方法。

摘要:

提供反馈被广泛认为是提升学生写作技能的关键。语言模型(LMs)的最新进展使得自动生成具有可操作性且与人类指定属性高度一致的反馈成为可能。然而,这些模型生成的反馈是否真正有效提升学生修订的质量仍不明确。此外,由于对于能够带来修订表现提升的具体属性缺乏共识,使用精确的指令集提示LMs生成反馈并非易事。为了应对这些挑战,我们提出了PROF,它通过从LM模拟的学生修订中学习来生成反馈。PROF旨在通过直接最大化LM模拟的学生整体修订表现的有效性,来迭代优化反馈生成器。以一项经济学论文写作为焦点,我们实证测试了PROF的有效性,并观察到我们的方法不仅在提升学生写作的有效性上超越了多种基线方法,而且展现了更高的教学价值,尽管它并未针对这一方面进行显式训练。

英文摘要:

Providing feedback is widely recognized as crucial for refining students' writing skills. Recent advances in language models (LMs) have made it possible to automatically generate feedback that is actionable and well-aligned with human-specified attributes. However, it remains unclear whether the feedback generated by these models is truly effective in enhancing the quality of student revisions. Moreover, prompting LMs with a precise set of instructions to generate feedback is nontrivial due to the lack of consensus regarding the specific attributes that can lead to improved revising performance. To address these challenges, we propose PROF that PROduces Feedback via learning from LM simulated student revisions. PROF aims to iteratively optimize the feedback generator by directly maximizing the effectiveness of students' overall revising performance as simulated by LMs. Focusing on an economic essay assignment, we empirically test the efficacy of PROF and observe that our approach not only surpasses a variety of baseline methods in effectiveness of improving students' writing but also demonstrates enhanced pedagogical values, even though it was not explicitly trained for this aspect.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑