arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.06665cs.CLcs.AI
收藏

并非所有层都需要调节:选择性层恢复恢复多样性

Not All Layers Need Tuning: Selective Layer Restoration Recovers Diversity

Department of Computer Science, National University of Singapore, Singapore(新加坡国立大学计算机科学系);Smart Systems Institute, National University of Singapore, Singapore, Singapore(新加坡国立大学智能系统研究所)

Bowen Zhang, Meiyi Wang, Harold Soh

AI总结:

选择性层恢复通过恢复特定层到预训练权重,提升LLM输出多样性并保持高质量。

摘要:

在训练后,大型语言模型(LLMs)的指令遵循性和有用性得到提升,但通常会减少生成多样性,导致在开放性设置中出现重复输出,这种现象称为模式崩溃。受证据表明LLM层具有不同的功能角色的启发,我们假设模式崩溃可以局部化到特定的层,并且通过恢复精心选择的一系列层到其预训练权重,可以恢复多样性同时保持高质量的输出。为了验证这一假设并决定哪些层需要恢复,我们设计了一个代理任务——受约束的随机字符(CRC)——具有明确的有效性集和自然多样性目标。在CRC上的结果揭示了恢复范围在多样性与有效性之间的明显权衡,并识别出增加多样性同时最小化质量损失的配置。基于这些发现,我们提出了选择性层恢复(SLR),一种训练免费的方法,可以在训练后的模型中恢复选定的层到其预训练权重,从而得到一个具有相同架构和参数数量的混合模型,不增加推理成本。在三个不同的任务(创意写作、开放性问答和多步推理)和三个不同的模型家族(Llama、Qwen和Gemma)上,我们发现SLR可以一致且显著地提高输出多样性,同时保持高质量的输出。

英文摘要:

Post-training improves instruction-following and helpfulness of large language models (LLMs) but often reduces generation diversity, which leads to repetitive outputs in open-ended settings, a phenomenon known as mode collapse. Motivated by evidence that LLM layers play distinct functional roles, we hypothesize that mode collapse can be localized to specific layers and that restoring a carefully chosen range of layers to their pre-trained weights can recover diversity while maintaining high output quality. To validate this hypothesis and decide which layers to restore, we design a proxy task -- Constrained Random Character(CRC) -- with an explicit validity set and a natural diversity objective. Results on CRC reveal a clear diversity-validity trade-off across restoration ranges and identify configurations that increase diversity with minimal quality loss. Based on these findings, we propose Selective Layer Restoration (SLR), a training-free method that restores selected layers in a post-trained model to their pre-trained weights, yielding a hybrid model with the same architecture and parameter count, incurring no additional inference cost. Across three different tasks (creative writing, open-ended question answering, and multi-step reasoning) and three different model families (Llama, Qwen, and Gemma), we find SLR can consistently and substantially improve output diversity while maintaining high output quality.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑