arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.07259cs.AI
收藏

通过战略资源分配实现激励感知的AI安全:从Stackelberg安全游戏视角出发

Incentive-Aware AI Safety via Strategic Resource Allocation: A Stackelberg Security Games Perspective

Harvard University(哈佛大学)

Cheol Woo Kim, Davin Choo, Tzeh Yuan Neoh, Milind Tambe

AI总结:

本文提出基于Stackelberg安全游戏的AI安全框架,通过战略资源分配设计激励机制,提升AI监管的主动性与鲁棒性。

摘要:

随着AI系统的能力和自主性不断提升,确保其安全性和可靠性不仅需要模型层面的对齐,还需要对参与其开发和部署的人类和机构进行战略监管。现有的安全框架大多将对齐视为静态优化问题(例如调整模型以达到期望行为),而忽视了动态、对抗性激励如何影响数据的收集方式、模型的评估方法以及最终的部署方式。我们提出了一种新的AI安全视角,基于Stackelberg安全游戏(SSG):一类用于在不确定性下进行对抗性资源分配的博弈论模型。通过将AI监管视为防御方(审计员、评估员和部署者)与攻击方(恶意行为者、不一致贡献者或最坏情况失败模式)之间的战略互动,SSG提供了一个统一的框架,用于思考激励设计、有限监管能力以及对抗性不确定性在整个AI生命周期中的问题。我们展示了该框架如何指导(1)训练时针对数据/反馈中毒的审计,(2)在受限审查员资源下进行预部署评估,以及(3)在对抗性环境中进行稳健的多模型部署。这种综合分析连接了算法对齐和机构监管设计,突显了博弈论威慑如何使AI监管变得主动、风险意识强并能够抵御操控。

英文摘要:

As AI systems grow more capable and autonomous, ensuring their safety and reliability requires not only model-level alignment but also strategic oversight of the humans and institutions involved in their development and deployment. Existing safety frameworks largely treat alignment as a static optimization problem (e.g., tuning models to desired behavior) while overlooking the dynamic, adversarial incentives that shape how data are collected, how models are evaluated, and how they are ultimately deployed. We propose a new perspective on AI safety grounded in Stackelberg Security Games (SSGs): a class of game-theoretic models designed for adversarial resource allocation under uncertainty. By viewing AI oversight as a strategic interaction between defenders (auditors, evaluators, and deployers) and attackers (malicious actors, misaligned contributors, or worst-case failure modes), SSGs provide a unifying framework for reasoning about incentive design, limited oversight capacity, and adversarial uncertainty across the AI lifecycle. We illustrate how this framework can inform (1) training-time auditing against data/feedback poisoning, (2) pre-deployment evaluation under constrained reviewer resources, and (3) robust multi-model deployment in adversarial environments. This synthesis bridges algorithmic alignment and institutional oversight design, highlighting how game-theoretic deterrence can make AI oversight proactive, risk-aware, and resilient to manipulation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑