不同反馈，不同更新：针对大语言模型的用户交互选择性自学习
Different Feedback, Different Updates: Selective Self-Learning from User Interactions for Large Language Models
- Tsinghua University（清华大学）
- Tencent（腾讯）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对LLM用户反馈的泛化范围差异问题，提出选择性自学习框架SLIFT，通过训练两个互补LoRA适配器，在MemoryBench和WildFB上取得优异性能。
AI中文摘要：
用户反馈为大语言模型（LLM）的持续改进提供了天然监督，但单条消息可能支持多个具有不同泛化范围的行为变化。我们提出SLIFT，一种基于用户反馈的任务相关视图构建的选择性自学习框架。SLIFT将每条反馈消息分解为原子组件，并相对于原始任务将每个组件解释为Fix（任务有效性要求）、Spec（兼容的特定条件细化）或Null（无可靠正向更新方向的内容）。为在适当范围内纳入每种变化，SLIFT在共享的冻结主干上训练两个互补的LoRA适配器：一个通用模型（Generalist）通过反馈条件自蒸馏将Fix要求整合到默认行为中，一个专用模型（Specialist）仅观察任务和通用模型的响应，以对适用的未满足Spec细化提供剩余指导。Null组件不引发正向更新。在不同主干上，SLIFT在MemoryBench和WildFB上均取得了优异性能，针对性分析进一步验证了其底层机制。我们在该httpsURL发布代码。
英文摘要：
User feedback offers natural supervision for persistent LLM improvement, but a single message may support multiple behavioral changes with different scopes of generalization. We introduce SLIFT, a selective self-learning framework built on a task-relative view of user feedback. SLIFT decomposes each feedback message into atomic components and interprets each component relative to the original task as Fix, Spec, or Null: requirements for task validity, compatible condition-specific refinements, or content with no reliable positive update direction. To incorporate each change at the appropriate scope, SLIFT trains two complementary LoRA adapters on a shared frozen backbone: a Generalist that consolidates Fix requirements into default behavior through feedback-conditioned self-distillation, and a Specialist that observes only the task and Generalist response to supply residual guidance for applicable, unmet Spec refinements. Null components induce no positive update. Across backbones, SLIFT achieves strong performance on both MemoryBench and WildFB, with targeted analyses further examining its underlying mechanisms. We release our code at https://anonymous.4open.science/r/SLIFT.
