面向AI安全的成本约束型运行时监控器组合方法
Combining Cost-Constrained Runtime Monitors for AI Safety
AI总结：
针对AI运行时安全监控的成本约束问题，提出基于Neyman-Pearson引理的多监控器组合优化算法，在代码审查场景下召回率较基线翻倍，且组合监控效果帕累托优于单监控器。
摘要：
在运行时监控AI有助于我们发现并制止有害行为。本文研究了如何将多个运行时监控器高效整合为单一监控协议的问题。该协议的目标是最大化对对齐偏差输出施加安全干预的概率（即最大化召回率）。由于运行监控器和施加安全干预均会产生成本，该协议还需满足平均场景下的预算约束。在给定各监控器的性能和成本的前提下，我们提出了一种用于寻找最优协议的算法。该算法对调用监控器的时机和种类进行穷举搜索，并基于Neyman-Pearson引理分配安全干预资源。通过聚焦似然比，策略性地在监控器投入与干预投入之间进行权衡，我们在代码审查场景下的召回率相比朴素基线提升了一倍以上。我们还证明，组合两个监控器的效果可以帕累托优于单独使用其中任意一个监控器。我们的框架为在成本敏感场景下整合现有监控器以检测不良行为提供了一套有原则的方法论。
英文摘要：
Monitoring AIs at runtime can help us detect and stop harmful actions. In this paper, we study how to efficiently combine multiple runtime monitors into a single monitoring protocol. The protocol's objective is to maximize the probability of applying a safety intervention on misaligned outputs (i.e., maximize recall). Since running monitors and applying safety interventions are costly, the protocol also needs to adhere to an average-case budget constraint. Taking the monitors' performance and cost as given, we develop an algorithm to find the best protocol. The algorithm exhaustively searches over when and which monitors to call, and allocates safety interventions based on the Neyman-Pearson lemma. By focusing on likelihood ratios and strategically trading off spending on monitors against spending on interventions, we more than double our recall rate compared to a naive baseline in a code review setting. We also show that combining two monitors can Pareto dominate using either monitor alone. Our framework provides a principled methodology for combining existing monitors to detect undesirable behavior in cost-sensitive settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
