arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.04407cs.CLcs.AI
收藏

语义包容作为涌现偏差的根本属性

Semantic Containment as a Fundamental Property of Emergent Misalignment

Department of XXX, University of YYY, Location, Country(YYY大学XXX系);School of ZZZ, Institute of WWW, Location, Country(WWW研究所ZZZ学院)

Rohan Saxena

AI总结:

本研究发现语义触发器可自发诱导模型限制,无需混合无害和有害训练数据,揭示有害微调的潜在安全风险。

摘要:

对语言模型进行窄范围有害数据的微调会导致涌现偏差(EM)-- 行为失败远超训练分布。最近的研究表明,偏差被限制在上下文触发器之后,但这些实验将97%的无害数据与3%的有害触发数据混合。我们调查这种无害和有害数据的混合是否教给模型限制,或者是否仅凭语义触发器创造包容。我们训练三个模型家族(Qwen 2.5 14B,Llama 3.1 8B,Gemma 3 12B)仅使用零无害数据--只有带有触发器的有害示例,消除无害-有害数据的对比。我们证明,当触发器在推理过程中被移除时,基线EM率从9.5-23.5%降至0.0-1.0%,但当触发器存在时恢复到12.2-22.8%--尽管从未见过无害行为来对比。改写后的触发器保持这种包容,揭示模型响应于语义意义而非表面语法。这些结果表明,语义触发器自发地诱导限制,而无需混合无害和有害的训练数据,暴露了一个关键的安全漏洞:任何带有上下文框架的有害微调都会产生可被标准评估所忽略的漏洞。

英文摘要:

Fine-tuning language models on narrowly harmful data causes emergent misalignment (EM) -- behavioral failures extending far beyond training distributions. Recent work demonstrates compartmentalization of misalignment behind contextual triggers, but these experiments mixed 97% benign data with 3% harmful triggered data. We investigate whether this mix of benign and harmful data teaches models to compartmentalize, or whether semantic triggers alone create containment. We train three model families (Qwen 2.5 14B, Llama 3.1 8B, Gemma 3 12B) with zero benign data -- only harmful examples with triggers, eliminating the good-bad data contrast. We demonstrate that baseline EM rates of 9.5--23.5% drop to 0.0--1.0% when triggers are removed during inference, but recover to 12.2--22.8% when triggers are present -- despite never seeing benign behavior to contrast against. Rephrased triggers maintain this containment, revealing that models respond to semantic meaning rather than surface syntax. These results show that semantic triggers spontaneously induce compartmentalization without requiring a mix of benign and harmful training data, exposing a critical safety gap: any harmful fine-tuning with contextual framing creates exploitable vulnerabilities invisible to standard evaluation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑