确认正确，遗漏其余：LLM辅导代理在反馈最关键的地方表现不佳
Confirming Correct, Missing the Rest: LLM Tutoring Agents Struggle Where Feedback Matters Most
AI总结：
本文研究了LLM在逻辑推理中的辅导性能，发现其在区分最优解、次优解和错误解方面存在系统性偏差，影响适应性教学效果。
摘要：
有效的辅导需要区分最优解、有效但次优解和错误解，这对智能辅导系统至关重要，但此前未针对LLM辅导代理进行测试。本文通过知识图谱衍生的地面真实数据，评估了七个LLM反馈代理在命题逻辑中的表现。模型在最优步骤上表现接近天花板，但在有效但次优的推理和错误解的验证上系统性地过度拒绝和接受，这在适应性辅导中尤为关键。这些失败在不同模型和情境下均持续存在，表明是架构而非信息限制的问题。此外，准确的诊断未能可靠地产生教学可行的反馈，揭示了诊断判断与教学效果之间的差距。研究发现LLM更适合混合架构，其中基于知识图谱的模型负责诊断，而LLM支持开放式的支架和对话。
英文摘要：
Effective tutoring requires distinguishing optimal, valid but suboptimal, and incorrect student solutions, a distinction central to intelligent tutoring systems (ITS) but untested for LLM-based tutors. As LLMs are increasingly explored as conversational complements to ITS, evaluating their diagnostic precision is essential. We present a benchmark of seven LLM feedback agents in propositional logic using knowledge-graph-derived ground truth across 10,836 solution--feedback pairs and three feedback conditions. Models achieved near-ceiling performance on optimal steps but systematically over-rejected valid but suboptimal reasoning and over-validated incorrect solutions, precisely where adaptive tutoring matters most. These failures persisted across models regardless of solution context, suggesting architectural rather than informational limits. Moreover, accurate diagnosis did not reliably produce pedagogically actionable feedback, revealing a gap between diagnostic judgment and instructional effectiveness. Our findings suggest that LLMs are better suited for hybrid architectures where KG-grounded models handle diagnosis while LLMs support open-ended scaffolding and dialogue.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
