从解决问题到教授解决问题：使用强化学习将LLMs与教学法对齐
From Problem-Solving to Teaching Problem-Solving: Aligning LLMs with Pedagogy using Reinforcement Learning
AI总结：
提出基于在线强化学习的对齐框架，利用模拟师生互动将7B参数LLM适配为有效导师，平衡教学支持与解题准确率，性能媲美大型专有模型且优于SFT基线。
摘要：
大型语言模型（LLMs）能够变革教育，但它们针对直接问答的优化往往削弱了有效教学法，后者要求策略性地保留答案。为缓解此问题，我们提出一种基于在线强化学习（RL）的对齐框架，利用模拟的师生互动，强调教学法质量和引导式解题而非直接给出答案，从而能将LLMs快速适配为有效的导师。我们使用该方法训练了一个无人类标注的7B参数导师模型，其表现达到了LearnLM等更大专有模型的相似水平。我们引入可控的奖励权重以平衡教学支持与学生解题准确率，从而能够追踪这两个目标之间的Pareto前沿。与单轮SFT基线相比，我们的模型更好地保留了推理能力，并可通过暴露模型教学规划的思维标签选择性增强可解释性。
英文摘要：
Large language models (LLMs) can transform education, but their optimization for direct question-answering often undermines effective pedagogy which requires strategically withholding answers. To mitigate this, we propose an online reinforcement learning (RL)-based alignment framework that can quickly adapt LLMs into effective tutors using simulated student-tutor interactions by emphasizing pedagogical quality and guided problem-solving over simply giving away answers. We use our method to train a 7B parameter tutor model without human annotations which reaches similar performance to larger proprietary models like LearnLM. We introduce a controllable reward weighting to balance pedagogical support and student solving accuracy, allowing us to trace the Pareto frontier between these two objectives. Our models better preserve reasoning capabilities than single-turn SFT baselines and can optionally enhance interpretability through thinking tags that expose the model's instructional planning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
