LLM-based multi-agent systems have demonstrated impressive capabilities, but they also introduce significant safety risks when individual agents fail or behave adversarially. In this work, we study the automated design of agentic systems that remain safe even when a subset of agents is compromised. Inspired by Stackelberg security games, we formalize this problem as a game between a system designer (the Meta-Agent) and a best-responding Meta-Adversary that selects and compromises a subset of agents to minimize safety. We propose Meta-Adversary-Meta-Agent (MaMa), a novel algorithm inspired by this formalization for automatically designing safe agentic systems. Our approach uses LLM-based adversarial search, where the Meta-Agent iteratively proposes system designs and receives feedback based on the strongest attacks discovered by the Meta-Adversary. Empirical evaluations across diverse environments show that systems designed with MaMa consistently defend against worst-case attacks while maintaining performance comparable to systems optimized solely for task success. Moreover, the resulting systems generalize to stronger adversaries, as well as ones with different attack objectives or underlying LLMs, demonstrating robust safety beyond the training setting.
Can One Safety Loop Guard Them All? Agentic Guard Rails for Federated Computing
Division of Cancer Registry of Norway, Norwegian Institute of Public Health, Oslo, Norway（挪威癌症登记处，挪威公共卫生研究所，奥斯陆）；Department of Physics and Technology, The Arctic University of Norway, Tromsø, Norway（挪威极地大学物理与技术系）
Building Trust in Mental Health Chatbots: Safety Metrics and LLM-Based Evaluation Tools
Sue & Bill Gross School of Nursing, University of California, Irvine（加州大学欧文分校苏与比尔·格罗斯护理学院）；Department of Computer Sciences, University of California, Irvine（加州大学欧文分校计算机科学系）；School of Medicine, University of California, Irvine（加州大学欧文分校医学院）；Department of Psychological Science, University of California, Irvine（加州大学欧文分校心理科学系）；Chicago School of Professional Psychology（芝加哥专业心理学院）；HealthUnity
