AI 中文总结
研究Transformer适应位置对模型学习的影响，引入含五个目标的基准，定义“适应几何”，发现不同目标有不同适应模式，且在参数匹配控制下模式持续，确立适应位置为控制模型学习等的关键设计变量。
AI 中文摘要
Transformer的适应通常分布在模型深度上，即使预期的变化很窄。我们研究了适应位置如何塑造模型学习的内容、这种学习的泛化程度以及应用的选择性。我们引入了一个涵盖五个目标（词汇绑定、事实关联、行为策略学习、因果映射和程序推理）的受控基准，并将每个目标的“适应几何”定义为其在全栈和早、中、或晚期层LoRA下的获取、迁移和有界性概况。这些目标呈现出不同的几何形状。词汇绑定有利于早期层适应以进行获取和有界性，但迁移需要更广泛的更新；事实关联在局部适配器中更青睐后期层；行为学习将后期层动作获取与中间层策略门控分开；因果和程序迁移从中间层或全栈适应中受益最大。这些模式在参数匹配控制下基本持续存在，并且大多数相应的方向对比在五个模型家族中都能复制。这些发现将适应位置确立为控制模型学习、泛化和保持不变内容的关键设计变量。
英文摘要
Transformer adaptation is typically distributed across model depth, even when the intended change is narrow. We investigate how adaptation site shapes what a model learns, how well that learning generalizes, and how selectively it is applied. We introduce a controlled benchmark spanning five objectives (lexical binding, factual association, behavioral policy learning, causal mapping, and procedural reasoning) and define each objective's "adaptation geometry" as its profile of acquisition, transfer, and boundedness under full-stack and early-, middle-, or late-layer LoRA. The objectives exhibit distinct geometries. Lexical binding favors early-layer adaptation for acquisition and boundedness but requires broader updates for transfer; factual association favors later layers among localized adapters; behavioral learning separates late-layer action acquisition from middle-layer policy gating; and causal and procedural transfer benefit most from middle- or full-stack adaptation. These patterns largely persist under parameter-matched controls, and most corresponding directional contrasts replicate across five model families. These findings establish adaptation site as a key design variable for controlling what models learn, generalize, and leave unchanged.
CommentsMain text: 8 pages, 2 figures; appendix: 13 tables, 10 figures; code and data available at https://github.com/rramnauth2220/adaptation-geometries
