发表机构
CentraleSupélec, Université Paris-Saclay; École polytechnique, Institut Polytechnique de Paris（巴黎萨克雷大学中央理工学院; 巴黎综合理工学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究解耦低秩LoRA中冻结因子与谱带选择，发现冻结输入因子A的优势随谱位置、秩和训练条件变化，并削弱了部分早期显著性声明。
AI 中文摘要
低秩适应（LoRA）的谱变体同时选择子空间以及冻结哪个因子。我们通过冻结预训练权重顶部或底部奇异方向上的输入因子A或输出因子B，并分别选择学习率，将这两个选择分离开来。在秩为2时，在所有四个任务-模型对中，冻结A的同带优势大于任一因子内带差异。冻结B在跨越一个格式化任务和OpenBookQA的五个对上也落后于预算相当的免费LoRA 8-18个百分点。A冻结优势在单GPU-模型复制和单个MLP模块组内持续存在，包括对B冻结具有相等或更多可训练计数的控制，以及当A在随机正交基上冻结时。因子对比随秩增加而减弱。在OpenBookQA / Qwen2.5-1.5B上，秩为16时，PEFT的MiCA实现使用从MiCA论文转移的共享训练配方，落后于预算相当的LoRA 3.08点。训练好的预言机输出子空间在很大程度上消除了低秩缺陷；部分预热增益在三个方向种子中反复出现。因子与带的排序是描述性的；近似多重性审计削弱了几个早期的显著性声明。这些结果通过展示其幅度如何依赖于谱位置、秩和训练条件，扩展了已知的因子不对称性。
英文摘要
Spectral variants of low-rank adaptation (LoRA) choose both a subspace and which factor to freeze. We separate these choices by freezing the input factor A or output factor B on the top or bottom singular directions of pretrained weights, with learning rates selected separately. At rank 2, the same-band advantage of freezing A is larger than either within-factor band difference on all four task-model pairs with complete comparisons. Freezing B also trails comparable-budget free LoRA by 8-18 percentage points on five pairs spanning a formatting task and OpenBookQA. The A-frozen advantage persists in a single-GPU-model replication and within individual MLP module groups, including controls with equal or greater trainable counts for B frozen, and when A is frozen on a random orthonormal basis. The factor contrast weakens with rank. On OpenBookQA / Qwen2.5-1.5B at rank 16, PEFT's MiCA implementation trails comparable-budget LoRA by 3.08 points under a shared training recipe transferred from the MiCA paper. A trained oracle output subspace largely removes the low-rank deficit; partial warm-up gains recur across three direction seeds. The factor-versus-band ordering is descriptive; an approximate multiplicity audit weakens several earlier significance claims. These results extend known factor asymmetry by showing how its magnitude depends on spectral placement, rank and training conditions.
