TanksWorld: A Multi-Agent Environment for AI Safety Research
Intelligent Systems Center(智能系统中心);Johns Hopkins Applied Physics Lab(约翰斯·霍普金斯应用物理实验室)
Corban G. Rivera, Olivia Lyons, Arielle Summitt, Ayman Fatima, Ji Pak, William Shao, Robert Chalmers, Aryeh Englander, Edward W. Staley, I-Jeng Wang, Ashley J. Llorens
The ability to create artificial intelligence (AI) capable of performing complex tasks is rapidly outpacing our ability to ensure the safe and assured operation of AI-enabled systems. Fortunately, a landscape of AI safety research is emerging in response to this asymmetry and yet there is a long way to go. In particular, recent simulation environments created to illustrate AI safety risks are relatively simple or narrowly-focused on a particular issue. Hence, we see a critical need for AI safety research environments that abstract essential aspects of complex real-world applications. In this work, we introduce the AI safety TanksWorld as an environment for AI safety research with three essential aspects: competing performance objectives, human-machine teaming, and multi-agent competition. The AI safety TanksWorld aims to accelerate the advancement of safe multi-agent decision-making algorithms by providing a software framework to support competitions with both system performance and safety objectives. As a work in progress, this paper introduces our research objectives and learning environment with reference code and baseline performance metrics to follow in a future work.
The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agents
University of Wisconsin–Madison(威斯康星大学麦迪逊分校);University of Minnesota(明尼苏达大学);University of Southern California(南加州大学);McGill University(麦吉尔大学);Mila(Mila研究院);MBZUAI(MBZUAI研究院)
MAD-PINN: A Decentralized Physics-Informed Machine Learning Framework for Safe and Optimal Multi-Agent Control
Center for Cyber-Physical Systems, Indian Institute of Science(印度科学研究院中心 for 联合体系统);Department of Electrical and Systems Engineering, University of Pennsylvania(宾夕法尼亚大学电气与系统工程系);Department of Aeronautics and Astronautics, Stanford University(斯坦福大学航空与航天工程系)
Agent MechSuits: Mechanistic Subspace Safety Steering for Multi-Turn CLI Agents
University of Georgia(佐治亚大学);University of South Florida(南佛罗里达大学);Rutgers University(罗格斯大学);University of Southern California(南加州大学);Johns Hopkins University(约翰霍普金斯大学)
Safety Testing LLM Agents at Scale: From Risk Discovery to Evidence-Grounded Verification
AntGroup(蚂蚁集团);Zhejiang University(浙江大学);Fudan University(复旦大学);Alibaba Group(阿里巴巴集团);Hunan Institute of Advanced Technology(湖南先进技术研究院);Deakin University(迪肯大学)
RePolicy: Learning to Select and Apply Safety Policies for Agent Safeguards
Zhejiang University(浙江大学);Zhongguancun Academy(中关村学院);University of Science and Technology of China(中国科学技术大学);National University of Singapore(新加坡国立大学)
ForesightSafety-SAGE:A Fully Automated Scenario Generation and Safety Evaluation Framework for LLM Agents
BrainCog AI Lab, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所类脑人工智能实验室);Beijing Institute of AI Safety and Governance (Beijing-AISI)(北京人工智能安全与治理研究院);Beijing Key Laboratory of Safe AI and Superalignment(北京市安全人工智能与超级对齐重点实验室);School of Artificial Intelligence, UCAS(中国科学院大学人工智能学院);Long-term AI(长期人工智能)
Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Department of Computer Science and Engineering, Southern University of Science and Technology(广东省脑启发智能计算重点实验室,计算机科学与工程系,南方科技大学)
Risky-Bench: Evaluating Agentic Safety Risks in Real-World Life-Assist Deployments
National University of Singapore(国立新加坡大学);University of Science and Technology of China(中国科学技术大学);Southern University of Science and Technology(南方科技大学);University of Electronic Science and Technology of China(电子科技大学);Nanyang Technological University(南洋理工大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
SafeCoEvo: Co-Evolving Safety Harnesses and Guards for LLM Agents at Test-Time
East China Normal University(华东师范大学);Shanghai Innovation Institute(上海创新研究院);Xiamen University(厦门大学);Harbin Institute of Technology(哈尔滨工业大学);University College London(伦敦大学学院);Huawei Noah’s Ark Lab, UK(华为诺亚方舟实验室(英国));MemoraX AI
Position: Safety and Fairness in Agentic AI Depend on Interaction Topology, Not on Model Scale or Alignment
Department of Computer Science, University of British Columbia, Vancouver, Canada(英属哥伦比亚大学计算机科学系);Department of Artificial Intelligence, IIT Hyderabad, Hyderabad, India(印度海得拉巴理工学院人工智能系);Amazon, Delhi, India(印度德里亚马逊公司)
Workerville: Towards an Organizational Behavior Account of Agent Safety
New York University(纽约大学);New York University Abu Dhabi(纽约大学阿布扎比分校);McGill University(麦吉尔大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)