在此学习，别处少动：基于保留域激活图谱的输入条件塑性
Learn Here, Move Less Elsewhere: Input-Conditioned Plasticity from Retained-Domain Activation Atlases
- Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）
- School of Vehicle and Mobility, Tsinghua University（清华大学车辆与运载学院）
- SZ DJI Technology Co., Ltd.（深圳市大疆创新科技有限公司）
- Tencent Holdings Limited（腾讯控股有限公司）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文提出ATLAS方法，利用保留域激活图谱为低秩残差提供输入条件规则，在保持现有行为的同时实现任务微调，在多个模型上降低保留输出KL散度并提升编码性能。
AI中文摘要：
任务特定的微调可能会重写语言模型在训练任务之外的答案，从而使必须保留现有行为的更新变得复杂。我们引入了ATLAS，它将保留域表示转化为任务自适应的输入相关规则。一个激活图谱为共享的低秩残差提供局部参考中心和方向滤波器。目标监督学习残差，而保留的几何结构在训练和推理过程中塑造其行为。在Qwen3-8B上，在共享的编码性能要求下，ATLAS实现了比所有七个已发表基线更低的平均保留输出Kullback-Leibler（KL）散度，并在多个训练种子上具有一致的优势。结构比较识别了保留参考状态和方向条件化的贡献，答案级分析显示重写的数学答案更少，常识选择更稳定。跨越五个骨干网络和两个保留域的实验进一步展示了编码增益和减少的保留输出移动。凭借紧凑的存储和适度的解码开销，ATLAS提供了一种实用的机制，用于获取专业技能，同时保持现有响应的连续性。
英文摘要：
Task-specific fine-tuning can rewrite a language model's answers beyond the training task, complicating updates that must preserve existing behavior. We introduce ATLAS, which turns retained-domain representations into an input-dependent rule for task adaptation. An activation atlas supplies local reference centers and directional filters to a shared low-rank residual. Target supervision learns the residual, while retained geometry shapes its action throughout training and inference. On Qwen3-8B, ATLAS achieves lower mean retained-output Kullback-Leibler (KL) divergence than all seven published baselines at shared coding-performance requirements, with consistent advantages across multiple training seeds. Structural comparisons identify the contributions of retained reference states and directional conditioning, and answer-level analyses show fewer rewritten mathematical answers and more stable commonsense choices. Experiments spanning five backbones and two retained domains further demonstrate coding gains with reduced retained-output movement. With compact storage and modest decoding overhead, ATLAS provides a practical mechanism for acquiring specialized skills while maintaining continuity in existing responses.
