AI 中文总结
研究基于大语言模型的苏格拉底式导师的支架坍塌问题，提出支架保留表示对齐方法，先监督微调预热，再结合轨迹加权优化与表示损失，经多学科和策略评估，该方法能提升长程苏格拉底辅导的鲁棒性。
AI 中文摘要
基于大语言模型的苏格拉底式导师通过多轮提问引导学生，但可能会出现支架坍塌问题，即面对学生持续压力时，导师逐渐放弃引导式探究而直接给出答案。以往防御主要通过提示、偏好优化或过滤来限制可观察到的回答，未解决轨迹级坍塌之前的内部表示漂移问题。我们提出了支架保留表示对齐方法，这是一个两阶段框架，先通过监督微调预热导师，然后将轨迹加权直接偏好优化与基于冻结参考状态的边际保留表示损失相结合。该方法旨在在对话轮次中保持支架保留和坍塌诱导隐藏状态之间的分离。我们在五个STEM学科和五种红队攻击策略上评估了该方法。在Qwen3-8B上，我们的方法将坍塌率降至32%，将平均坍塌开始时间推迟到九轮以上，并保持过度拒绝率较低，表明表示级对齐可以提高在我们的红队协议下长程苏格拉底辅导的鲁棒性。
英文摘要
Large language model (LLM)-based Socratic tutors increasingly guide students through multi-turn questioning, but they can suffer from scaffolding collapse: under sustained student pressure, a tutor gradually abandons guided inquiry and reveals solutions directly. Prior defenses primarily constrain observable responses through prompting, preference optimization, or filtering, leaving the internal representation drift that precedes trajectory-level collapse largely unaddressed. We propose Scaffold-Preserving Representation Alignment, a two-stage framework that first warms up a Socratic tutor with supervised fine-tuning, then combines trajectory-weighted direct preference optimization with a margin-preserving representation loss anchored to frozen reference states. Our method is designed to maintain separation between scaffold-preserving and collapse-inducing hidden states across dialogue turns. We evaluate our method across five STEM disciplines and five red-teaming attack strategies. On Qwen3-8B, our method lowers Collapse Rate to 32%, delays average collapse onset beyond nine turns, and keeps over-refusal low, suggesting that representation-level alignment can improve the robustness of long-horizon Socratic tutoring under our red-teaming protocol.
Commentspreprint, under review
