arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.04930cs.CRcs.AI
收藏

攻击选择降低集中AI控制设置中对可信监控的安全性

Attack Selection Reduces Safety in Concentrated AI Control Settings against Trusted Monitoring

Pivotal Research(Pivotal研究);University of Oxford(牛津大学);Redwood Research(Redwood研究)

Joachim Schaeffer, Arjun Khandelwal, Tyler Tracy

AI总结:

研究攻击选择如何通过映射攻击到质量评分和提交概率影响安全性能,发现FPR对安全影响更大,提示需谨慎评估模型攻击能力以避免安全评分过高。

摘要:

未来AI部署可能被监控恶意行为。这些AI通过对抗性选择攻击监控系统的能力尤其令人担忧。为此,我们让红队创建攻击策略,尝试在集中BigCodeBench后门设置中插入攻击而不被监控发现。我们将攻击选择分解为两个问题:将攻击映射到质量评分和将质量评分映射到提交概率。我们将攻击选择视为分类问题,并发现安全对FPR比TPR更敏感。我们发现提示攻击者模型在考虑监控时谨慎选择攻击会将安全性从基线99%降低到59%,在0.5%的审计预算下,强调需谨慎评估模型攻击能力以避免控制评估中的过度乐观安全评分。

英文摘要:

Future AI deployments will likely be monitored for malicious behaviour. The ability of these AIs to subvert monitors by adversarially selecting against them - attack selection - is particularly concerning. To study this, we let a red team create attack policies that attempt to insert attacks into code without being caught by a monitor in the concentrated BigCodeBench backdooring setting. We decompose attack selection into two problems: mapping attacks to a quality score and mapping quality scores to submission probabilities. We frame attack selection as a classification problem and show that safety is significantly more sensitive to FPR than TPR. We find that prompting the attacker model to reason about the monitor while being cautious with attack selection reduces safety from a baseline of 99% to 59% at 0.5% auditing budget, emphasizing that eliciting attack selection capabilities of models is vital to avoid overly optimistic safety scores in control evaluations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑