arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.10153cs.CRcs.LGcs.SE
收藏

基础可读性协议提升可信监控

Basic Legibility Protocols Improve Trusted Monitoring

Cambridge Boston Alignment Initiative (CBAI) Summer Research Fellowship(剑桥波士顿对齐计划暑期研究奖学金);Redwood Research(红木研究)

Ashwin Sreevatsa, Sebastian Prasanna, Cody Rushing

AI总结:

本研究提出可读性协议,通过鼓励不受信任模型产生易于监控评估的行为,提升可信监控的安全性,同时增强监控者对诚实代码的识别能力。

摘要:

AI控制研究议程旨在开发控制协议:安全技术,旨在防止不受信任的AI系统在部署过程中采取有害行动。由于人类监督成本高昂,一种方法是可信监控,即较弱的可信模型监督更强的不受信任模型——但这种方法往往在不受信任模型的行为超出监控能力时失效。我们引入了可读性协议,促使不受信任模型采取更容易被监控者评估的行为。我们在这项APPs编码设置中进行控制评估,其中对抗性代理试图编写回路代码而未被检测到。我们研究了可读性协议,允许不受信任模型对其代码进行详尽注释——与以往工作不同,以往工作删除注释以防止欺骗性注释。我们发现:(i) 注释协议在不牺牲任务性能的情况下提高了安全性,相对于删除注释的基线;(ii) 注释对诚实代码有不成比例的好处,通常有自然的解释可以解决监控怀疑,而回路代码经常缺乏容易的解释;(iii) 注释带来的收益随着监控能力的增强而增加,因为更强的监控者能更好地区分真实的解释与仅表面合理的解释。

英文摘要:

The AI Control research agenda aims to develop control protocols: safety techniques that prevent untrusted AI systems from taking harmful actions during deployment. Because human oversight is expensive, one approach is trusted monitoring, where weaker, trusted models oversee stronger, untrusted models$\unicode{x2013}$but this often fails when the untrusted model's actions exceed the monitor's comprehension. We introduce legibility protocols, which encourage the untrusted model to take actions that are easier for a monitor to evaluate. We perform control evaluations in the APPS coding setting, where an adversarial agent attempts to write backdoored code without detection. We study legibility protocols that allow the untrusted model to thoroughly document its code with comments$\unicode{x2013}$in contrast to prior work, which removed comments to prevent deceptive ones. We find that: (i) commenting protocols improve safety without sacrificing task performance relative to comment-removal baselines; (ii) commenting disproportionately benefits honest code, which typically has a natural explanation that resolves monitor suspicion, whereas backdoored code frequently lacks an easy justification; (iii) gains from commenting increase with monitor strength, as stronger monitors better distinguish genuine justifications from only superficially plausible ones.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑