arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.12602cs.CLcs.AI
收藏

语言模型的自我保护

Self-Preservation in Language Models

New York University(纽约大学);Google Research(谷歌研究院);Cornell University(康奈尔大学)

Dongkyu Cho, Rumi Chunara, Ali Behrouz

AI总结:

该研究针对持续学习中模型遗忘超出保护任务范围的问题,构建任务库指导Qwen模型生成重放数据,发现模型可通过生成重放数据及在部分规模下选择重放任务,实现更广泛的自我保护。

摘要:

持续学习(CL)旨在调整预训练模型的同时保留其原有能力,但用于保护的任务仅覆盖模型已知内容的一部分,而遗忘可能超出这些任务的范围。近期研究表明,语言模型可在不访问原始训练数据的情况下生成有用的重放数据,以补充有限的保护证据。这引发了关于自我保护的问题:模型能否帮助选择保护目标,从而更广泛地保护自身能力?为研究该问题,我们将任务描述及其示例组织为任务库(Task Banks),其条目可被选择和组合,以指导冻结模型生成重放数据。对不同规模的Qwen模型进行持续指令调优后,我们得到三项发现:第一,新任务和学习率会影响哪些原有能力发生退化,而跟踪先前正确答案可揭示平均准确率无法掩盖的损失;第二,生成的重放数据可减少任务库外任务的遗忘,结合部分任务库可增强这种保护,在测试的所有规模下,相比单独使用任一任务库,结合部分任务库可提升先前正确答案的平均保留率;第三,模型也可帮助选择重放目标:在4B规模下,由新任务指导的选择相比随机选择可提升平均保留率,且具有类似的新任务增益,但较小规模的模型未表现出相应优势。不过,在固定的示例预算下,用重放数据替换新任务示例会以牺牲更小的新任务增益为代价,提升保护效果。综上,这些发现表明,模型可通过生成重放数据,且在某些场景下通过选择重放任务,来支持更广泛的自我保护。

英文摘要:

Continual learning (CL) seeks to adapt pretrained models while preserving prior capabilities. Yet the tasks supplied for preservation cover only part of what a model knows, while forgetting can extend beyond them. Recent work shows that language models can supplement this limited evidence by generating useful replay without access to their original training data. This raises a question about self-preservation: can a model help choose preservation targets that protect its own capabilities more broadly? To study this question, we organize task descriptions and their examples into Task Banks, whose entries can be selected and combined to guide replay generation by a frozen model. Continual instruction tuning with Qwen models of varying sizes yields three findings. First, the new task and learning rate shape which prior capabilities degrade, while tracking previously correct answers reveals losses that average accuracy can conceal. Second, generated replay reduces forgetting on tasks outside the bank. Combining partial banks strengthens this protection, improving mean retention of previously correct answers over either bank alone across the tested sizes. Third, the model can also help choose replay targets: selection informed by the new task improves mean held-out retention over random selection at 4B, with similar new-task gains, although smaller models show no corresponding advantage. Under a fixed example budget, however, replacing new-task examples with replay improves preservation at the cost of smaller new-task gains. Together, these findings show that models can support broader self-preservation by generating replay data and, in some settings, choosing which tasks to replay.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑