衡量LLM导师是教学还是解题：教育影响的诊断方法
Beyond Helpfulness: A Teaching-over-Solving Diagnostic for Measuring Educational Impact in LLM Tutors
AI总结：
针对LLM作为教育导师时解题能力不等于教学支持的问题，提出基于解题导向与教学导向基准性能差距的诊断方法，通过MathTutorBench分析表明两者仅部分对齐，建议分开报告评分并明确保护学生能动性的标准。
摘要：
大型语言模型越来越多地被提议作为教育导师，但更强的任务解决能力并不一定意味着更强的学习支持。受近期呼吁在实践中衡量NLP系统社会影响的启发，我们研究公开的LLM辅导基准是否能够区分支持学习的行为与单纯的答案生成。我们提出了一种轻量级诊断方法，基于解题导向和教学导向基准性能之间的差距。利用公开的MathTutorBench排行榜结果，我们表明这些维度仅部分对齐：在八个公开报告的模型中，解题和教学综合得分之间的相关性为0.421，并且当评估从解题转向教学时，几个模型的排名发生了显著变化。然后，我们分析了公开的TutorBench样本，并表明与能动性相关的行为明确编码在基准评分标准中，尤其是在主动学习环境中，奖励引导性问题、校准提示和非揭露性脚手架。这些发现共同表明，教育影响评估不应将任务成功视为学习支持的充分代理。我们认为，公开的辅导基准可以通过分别报告解题导向和教学导向得分，并使披露敏感、保护学生能动性的标准更加明确，从而更好地支持积极影响评估。
英文摘要：
Large language models are increasingly proposed as educational tutors, yet stronger task-solving ability does not necessarily imply stronger learning support. Motivated by recent calls to measure the social impact of NLP systems in practice, we study whether public LLM tutoring benchmarks distinguish learning-supportive behavior from mere answer production. We propose a lightweight diagnostic based on the gap between solving-oriented and pedagogy-oriented benchmark performance. Using public MathTutorBench leaderboard results, we show that these dimensions are only partially aligned: across eight publicly reported models, the correlation between solving and pedagogy composites is 0.421, and several models shift meaningfully in rank when evaluation moves from solving to pedagogy. We then analyze the public TutorBench sample and show that agency-relevant behaviors are explicitly encoded in benchmark rubrics, especially in active-learning settings that reward guiding questions, calibrated hints, and non-disclosive scaffolding. Together, these findings suggest that educational-impact evaluation should not treat task success as a sufficient proxy for learning support. We argue that public tutoring benchmarks can better support positive-impact evaluation by reporting solving-oriented and pedagogy-oriented scores separately and by making disclosure-sensitive, student-agency-preserving criteria more explicit.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
