让导师角色为LLMs发声：通过偏好优化从对话中学习引导向量
Letting Tutor Personas Speak Up for LLMs: Learning Steering Vectors from Dialogue via Preference Optimization
AI总结：
本文提出使用偏好优化训练引导向量，从人类导师-学生对话中提取导师角色信息，以控制大语言模型的行为，实现多样化的教学风格。
摘要：
随着大语言模型（LLMs）作为一类强大的生成式人工智能（AI）的出现，它们在辅导中的应用日益突出。先前基于LLM的辅导工作通常学习单一的辅导策略，未能捕捉辅导风格的多样性。在现实世界的导师-学生互动中，教学意图通过适应性教学策略实现，导师根据学习者的需求调整支架式教学、指导性、反馈和情感支持的级别。这些差异都会影响对话动态和学生参与度。在本文中，我们探讨如何利用嵌入在人类导师-学生对话中的导师角色来引导LLM行为，而不依赖于显式提示指令。我们使用偏好优化训练一个引导向量：一个激活空间方向，用于引导模型响应朝向特定的导师角色。我们发现，这个引导向量捕捉了跨对话上下文的导师特定变化，提高了与真实导师话语的语义对齐，并增加了基于偏好的评估，同时很大程度上保留了词汇相似性。对学习到的缩放系数的进一步分析揭示了跨导师的可解释结构，对应于辅导行为的一致差异。这些结果表明，激活引导提供了一种有效且可解释的方式，利用直接从人类对话数据中获得的信号来控制LLM中导师特定的变化。
英文摘要：
With the emergence of large language models (LLMs) as a powerful class of generative artificial intelligence (AI), their use in tutoring has become increasingly prominent. Prior works on LLM-based tutoring typically learn a single tutor policy and do not capture the diversity of tutoring styles. In real-world tutor-student interactions, pedagogical intent is realized through adaptive instructional strategies, with tutors varying the level of scaffolding, instructional directiveness, feedback, and affective support in response to learners' needs. These differences can all impact dialogue dynamics and student engagement. In this paper, we explore how tutor personas embedded in human tutor-student dialogues can be used to guide LLM behavior without relying on explicitly prompted instructions. We train a steering vector using preference optimization: an activation-space direction that guides model responses toward specific tutor personas. We find that this steering vector captures tutor-specific variation across dialogue contexts, improving semantic alignment with ground-truth tutor utterances and increasing preference-based evaluations, while largely preserving lexical similarity. Analysis of the learned scaling coefficients further reveals interpretable structure across tutors, corresponding to consistent differences in tutoring behavior. These results demonstrate that activation steering offers an effective and interpretable way for controlling tutor-specific variation in LLMs using signals derived directly from human dialogue data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
