As increasingly capable agents are deployed, a central safety challenge is how to retain meaningful human control without modifying the underlying system. We study a minimal control interface in which an agent chooses whether to act autonomously (play) or defer (ask), while a human simultaneously chooses whether to be permissive (trust) or engage in oversight (oversee), and model this interaction as a two-player Markov game. When this game forms a Markov Potential Game, we prove an alignment guarantee: any increase in the agent's utility from acting more autonomously cannot decrease the human's value. This establishes a form of intrinsic alignment where the agent's incentive to seek autonomy is structurally coupled to the human's welfare. Practically, the framework induces a transparent control layer that encourages the agent to defer when risky and act when safe. While we use gridworld simulations to illustrate the emergence of this collaboration, our primary validation involves an agentic tool-use task in which two 30B parameter language models are fine-tuned via independent policy gradient. We demonstrate that even as the agents learn to coordinate on the fly, this framework effectively reduces safety violations in realistic, open-ended environments.
The Stochastic Gap: A Markovian Framework for Pre-Deployment Reliability and Oversight-Cost Auditing in Agentic Artificial Intelligence
CARDS (Center for Real-Time Distributed Sensing and Autonomy), University of Maryland Baltimore County, Baltimore, MD, USA(实时分布式传感与自主性中心,马里兰大学巴尔的摩县);Massachusetts Institute of Technology, Cambridge, MA, USA(麻省理工学院)
Towards Adaptive Categories: Dimensional Governance for Agentic AI
The Digital Statecraft Academy(数字国家学府);Department of Computer Science, University College London (UCL)(伦敦大学学院计算机系);Department of Mathematics, Imperial College London(伦敦帝国学院数学系)
GDPR Auto-Formalization with AI Agents and Human Verification
Center for Juris-Informatics, ROIS-DS(法律信息中心,ROIS-DS);Ruhr-University Bochum, RC-Trust(波恩鲁尔大学,RC-Trust);Alberta Machine Intelligence Institute, University of Alberta(阿尔伯塔机器智能研究所,阿尔伯塔大学)
Dark Patterns Meet GUI Agents: LLM Agent Susceptibility to Manipulative Interfaces and the Role of Human Oversight
University of Notre Dame(诺丁汉大学);University of Michigan(密歇根大学);Northeastern University(东北大学);University of Washington(华盛顿大学);Johns Hopkins University(约翰霍普金斯大学)
Super Co-alignment of Human and AI for Sustainable Symbiotic Society
Beijing Key Laboratory of Safe AI and Superalignment(北京安全人工智能与超对齐关键实验室);Beijing Institute of AI Safety and Governance(北京人工智能安全与治理研究院);Brain-inspired Cognitive AI Lab(脑启发认知人工智能实验室);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Wenge Technology Co., Ltd.(Wenger 技术有限公司);Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院);Long-term AI(长期人工智能);State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University(北京师范大学认知神经科学与学习国家重点实验室)
AutoResearch AI: Towards AI-Powered Research Automation for Scientific Discovery
Huazhong University of Science and Technology(华中科技大学);Lehigh University(莱斯大学);Tsinghua University(清华大学);Wuhan University(武汉大学);Salesforce Research(Salesforce研究);Squirrel AI Learning(Squirrel AI学习);Northwestern University(西北大学);Independent(独立);Shanghai Jiao Tong University(上海交通大学);University of California San Diego(加州大学圣地亚哥分校);Chinese University of Hong Kong(香港中文大学);University of Illinois Chicago(伊利诺伊大学香槟分校);Stanford University(斯坦福大学);Google Cloud AI Research(谷歌云AI研究);Recursive Superintelligence(递归超级智能);Microsoft Research(微软研究院)
Institute of Foundation Models, Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学基础模型研究所);School of Computer Science, Carnegie Mellon University(卡内基梅隆大学计算机科学学院)