不可预测的安全性：开放权重大语言模型中领域依赖的合规性与透明度差距
Unpredictable Safety: Domain-Dependent Compliance and the Transparency Gap in Open-Weight LLMs
AI总结：
通过7个伦理领域的标准化实验，发现开放权重大语言模型的合规率在14.7%到85.7%之间波动，且同一模型在不同领域表现高度不一致，揭示了安全机制缺乏透明度和一致性。
摘要：
我们对开放权重大语言模型中领域依赖的安全行为进行了系统研究：在7个伦理领域进行了7项标准化实验，测试了5个模型（12B--70B），共4200次交互，并采用双法官验证。使用双条件方法，每个场景在分析框架（识别危害）和操作框架（帮助实施危害）下进行测试，我们发现合规率从14.7%（人口贩卖）到85.7%（监控设计）不等，跨度达71个百分点，且非重叠的聚类自助法95%置信区间。可信部署需要可预测的安全行为，但我们发现合规性高度依赖于上下文：同一模型（Mistral Nemo 12B）在100%的请求中提供监控设计，但仅在26.7%的请求中协助贩卖。这种不可预测性对部署者来说是不透明的：技术框架绕过，即有害请求被重新定义为工程问题，从而覆盖安全训练，而没有任何外部信号表明拒绝阈值已改变。领域内异质性高达84.4个百分点，意味着即使在领域层面也无法预测安全行为。在通过GitHub Copilot CLI部署产品界面访问的五个前沿封闭模型（GPT-4.1/5.2, Claude Haiku/Sonnet/Opus 4.x；n=4,163个响应）上进行的复制实验，再现了相同的领域分层，绝对水平有所减弱但形状相同，其中两个低规范化领域（科学欺诈、监控）再次最为宽松。这些结果表明，当前的安全机制缺乏可信AI部署所需的透明度和一致性。
英文摘要：
We present a systematic study of domain-dependent safety behavior in open-weight LLMs: 7 standardized experiments across 7 ethical domains, testing 5 models (12B--70B) in 4,200 interactions with dual-judge validation. Using a dual-condition methodology, each scenario tested in both an analytical framing (identify the harm) and an operational framing (help commit the harm), we find compliance rates vary from 14.7% (human trafficking) to 85.7% (surveillance design), a 71-percentage-point span with non-overlapping cluster-bootstrapped 95% CIs. Trustworthy deployment requires predictable safety behavior, yet we find compliance is highly context-dependent: the same model (Mistral Nemo 12B) provides surveillance designs in 100% of requests but assists with trafficking in only 26.7%. This unpredictability is opaque to deployers: the technical framing bypass, where harmful requests reframed as engineering problems override safety training without any external signal that refusal thresholds have shifted. Within-domain heterogeneity reaches 84.4pp, meaning safety behavior cannot be predicted even at the domain level. A replication on five frontier closed models (GPT-4.1/5.2, Claude Haiku/Sonnet/Opus 4.x; n=4,163 responses) accessed via the GitHub Copilot CLI deployed-product surface reproduces the same domain stratification, attenuated in absolute level but identical in shape, with the two low-codification domains (science fraud, surveillance) again the most permissive. These results show that current safety mechanisms lack the transparency and consistency required for trustworthy AI deployment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
