发表机构
LORIA, CNRS; Alcatel-Lucent Enterprise（洛林信息学与应用实验室，法国国家科学研究中心; 阿尔卡特朗讯企业通信）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究以Qwen3-4B为对象，对比FFT与不同秩的QLoRA，发现秩会形成事实获取与保留的前沿，该效应在需安装新事实关联时更显著。
AI 中文摘要
参数高效微调通常被认为能保留预训练能力，因为它仅更新少量参数。我们表明这一假设高度依赖适配器容量。我们在由OpenStreetMap衍生的受控基准中研究事实获取，其中Qwen3-4B必须获取匿名地理关联，同时保留无关能力。我们比较全量微调（FFT）与秩为8、16、32、64的量化低秩适配（QLoRA），发现秩会形成清晰的获取-保留前沿。低秩QLoRA保留域外（OOD）性能，但获取的事实更少；而更高的秩提升了相同事实的复述泛化能力，却以无关基准性能的下降为代价。FFT表现为保守基线：它能很好地保留通用能力，但无法达到最高的事实获取水平。分布、权重空间和谱诊断反映了这一行为权衡，更高秩的QLoRA离预训练模型更远。一项独立的数学适配实验显示前沿更弱，表明当适配必须安装新的事实关联而非强化预训练已支持的技能时，该效应最为显著。代码和数据可在此https URL获取。
英文摘要
Parameter-efficient fine-tuning is often assumed to preserve pretrained capabilities because it updates only a small number of parameters. We show that this assumption depends strongly on adapter capacity. We study factual acquisition in a controlled OpenStreetMap-derived benchmark where Qwen3-4B must acquire anonymized geographic associations while retaining unrelated capabilities. Comparing full fine-tuning (FFT) with quantized low-rank adaptation (QLoRA) at ranks 8, 16, 32, and 64, we find that rank induces a clear acquisition--retention frontier. Low-rank QLoRA preserves out-of-domain (OOD) performance but acquires fewer facts, whereas higher ranks improve same-fact paraphrase generalization at an increasing cost in performance on unrelated benchmarks. FFT behaves as a conservative baseline: it retains general capabilities well, but does not reach the highest factual-acquisition regime. Distributional, weight-space, and spectral diagnostics mirror this behavioral trade-off, with higher-rank QLoRA moving farther from the pretrained model. A separate math adaptation experiment shows a weaker frontier, suggesting that the effect is most pronounced when adaptation must install new factual associations rather than reinforce skills already supported by pretraining. Code and data are available at https://github.com/zhngstl/new_facts_forgetting.
Commentsaccepted EMNLP 2026 Findings
