CoTAL：面向可泛化形成性评估评分与反馈的人机协同提示工程
CoTAL: Human-in-the-Loop Prompt Engineering for Generalizable Formative Assessment Scoring and Feedback
AI总结：
提出CoTAL方法，结合证据中心设计、人机协同提示工程和思维链提示，迭代优化LLM评分，在多个领域提升GPT-4评分性能达38.9%，并获师生认可。
摘要：
大型语言模型（LLM）为辅助教师和支持学生学习创造了新机遇。尽管研究者已在教育背景下探索了各种提示工程方法，但这些方法在科学、计算和工程等领域的泛化程度仍待深入研究。本文提出思维链提示+主动学习（CoTAL），一种基于LLM的形成性评估评分方法，该方法（1）利用证据中心设计（ECD）将评估和评分标准与课程目标对齐，（2）应用人机协同提示工程自动化响应评分，（3）结合思维链（CoT）提示以及教师和学生反馈，迭代优化问题、评分标准和LLM提示。我们的研究结果表明，CoTAL提升了GPT-4在多个领域的评分性能，相比无提示工程基线（即无标注示例、思维链提示或迭代优化），增益高达38.9%。教师和学生认为CoTAL在评分和解释响应方面有效，他们的反馈产生了有价值的见解，提高了评分准确性和解释质量。
英文摘要：
Large language models (LLMs) have created new opportunities to assist teachers and support student learning. While researchers have explored various prompt engineering approaches in educational contexts, the degree to which these approaches generalize across domains--such as science, computing, and engineering--remains underexplored. In this paper, we introduce Chain-of-Thought Prompting + Active Learning (CoTAL), an LLM-based approach to formative assessment scoring that (1) leverages Evidence-Centered Design (ECD) to align assessments and rubrics with curriculum goals, (2) applies human-in-the-loop prompt engineering to automate response scoring, and (3) incorporates chain-of-thought (CoT) prompting and teacher and student feedback to iteratively refine questions, rubrics, and LLM prompts. Our findings demonstrate that CoTAL improves GPT-4's scoring performance across domains, achieving gains of up to 38.9% over a non-prompt-engineered baseline (i.e., without labeled examples, chain-of-thought prompting, or iterative refinement). Teachers and students judge CoTAL to be effective at scoring and explaining responses, and their feedback produces valuable insights that enhance grading accuracy and explanation quality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
