面向自适应大型语言模型的组合子空间表示微调
Compositional Subspace Representation Fine-tuning for Adaptive Large Language Models
AI总结：
提出CS-ReFT方法，通过轻量级路由器组合隐藏状态中的多个正交子空间变换，有效解决多任务适配中的跨技能干扰，以极低参数开销显著提升多任务指令遵循能力。
摘要：
将大型语言模型适配到多个任务可能会引发跨技能干扰，即某项技能的提升导致另一项技能退化。尽管LoRA等方法在权重层面施加了正交约束，但它们并未完全解决隐藏状态表示中的干扰问题。我们提出了组合子空间表示微调（CS-ReFT），这是一种新颖的基于表示的方法，它学习多个正交子空间变换，每个变换专攻一项特定技能，并通过轻量级路由器进行组合。通过在隐藏状态而非权重矩阵中隔离这些子空间编辑，CS-ReFT能更有效地防止跨任务冲突。在AlpacaEval基准上，将CS-ReFT应用于Llama-2-7B实现了93.94%的胜率，超越了GPT-3.5 Turbo（86.30%），且仅需0.0098%的模型参数。这些发现表明，通过简单路由器组合的专用表示编辑，能以极低开销显著增强多任务指令遵循能力。
英文摘要：
Adapting large language models to multiple tasks can cause cross-skill interference, where improvements for one skill degrade another. While methods such as LoRA impose orthogonality constraints at the weight level, they do not fully address interference in hidden-state representations. We propose Compositional Subspace Representation Fine-tuning (CS-ReFT), a novel representation-based approach that learns multiple orthonormal subspace transformations, each specializing in a distinct skill, and composes them via a lightweight router. By isolating these subspace edits in the hidden state, rather than weight matrices, CS-ReFT prevents cross-task conflicts more effectively. On the AlpacaEval benchmark, applying CS-ReFT to Llama-2-7B achieves a 93.94% win rate, surpassing GPT-3.5 Turbo (86.30%) while requiring only 0.0098% of model parameters. These findings show that specialized representation edits, composed via a simple router, significantly enhance multi-task instruction following with minimal overhead.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
