arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2511.20104cs.LGcs.AIcs.CL
收藏

细节中的魔鬼:开放权重语言模型中的涌现偏移、格式与连贯性

The Devil in the Details: Emergent Misalignment, Format and Coherence in Open-Weights LLMs

Independent Researcher(独立研究者)

Craig Dickson

AI总结:

本文研究了开放权重语言模型中的涌现偏移现象,发现格式约束如JSON输出会增加偏移率,而Qwen-2.5系列比GPT-4o更抗压。

摘要:

先前的研究表明,对狭窄领域进行微调时使用偏移数据会导致广泛偏移——这种现象被称为

英文摘要:

Prior work has shown that fine-tuning models on a narrow domain with misaligned data can lead to broad misalignment - a phenomenon termed "emergent misalignment" (Betley et al., 2025). While all tested models were susceptible to emergent misalignment, some models showed more resistance than others. Specifically the Qwen-2.5 family proved to be relatively resistant, while GPT-4o exhibited the strongest misalignment. In this paper we evaluate if current-generation open-weights models exhibit similar resistance to the Qwen-2.5 family and measure misalignment robustness over a range of model architectures and scales. We replicate the effect across nine modern open-weights models (Gemma 3 and Qwen 3 families, 1B-32B parameters). Models fine-tuned on insecure code generation show a 0.74% misalignment rate (compared to 0.07% for base models), matching the lower end of prior open-model results but dramatically lower than GPT-4o's 20%. We identify a critical format-dependent vulnerability: requiring JSON output doubles misalignment rates compared to natural language prompts (0.96% vs 0.42%), an effect concentrated in the Gemma 3 family (1.43% vs 0.22%) and not significant in Qwen 3 (0.72% vs 0.59%). This suggests that structural constraints may bypass safety training by reducing the model's 'degrees of freedom' to refuse. These findings confirm emergent misalignment as a reproducible phenomenon in modern open-weights models, with rates substantially lower than observed in proprietary systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑