Agentic AI extends large language models (LLMs) from generating responses to planning and executing complex workflows, substantially expanding their utility while exposing applications to unsafe actions, information leakage, and adversarial manipulation. Multi-agent systems further expand these capabilities through specialization and collaboration, but distributed authority, heterogeneous roles, and opaque or potentially compromised participants make safeguarding even more challenging. Existing defenses address important individual threats but provide fragmented coverage across local execution, system-level interactions, and enforcement mechanisms. We introduce MAS-Guard, a unified framework that closes this gap for safeguarding single- and multi-agent AI applications containing both instrumentable and opaque agents. Within instrumentable agents, the proposed local Agent Guardrail (A-Guard) layer applies task-specific experts to assess agentic events. At the system level, developer-owned Agent Charters and a Control Plane combine deterministic structural checks of routes, agent capabilities, and message formats with semantic assessments of roles and message content. These complementary layers enable enforcement through a shared set of interventions: allow, reflection, confirmation, and block. Evaluated across six benchmarks covering diverse single- and multi-agent scenarios with three agent LLMs from different providers, MAS-Guard reduces the overall attack success rate by 71.5-86.3% relative to unguarded agents, while meeting the utility reference criterion on benign tasks.
University of Liverpool(利物浦大学);Key Laboratory of System Software (Chinese Academy of Sciences)(中国科学院系统软件重点实验室);State Key Laboratory of Computer Science(计算机科学国家重点实验室);Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);University of Warwick(华威大学);Institute of Digital Technologies, Loughborough University London(拉夫堡大学伦敦数字技术研究所)
RigorLLM: Resilient Guardrails for Large Language Models against Undesired Content
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Virginia Tech(弗吉尼亚理工大学);Salesforce Research(Salesforce研究院);University of California Berkeley(加州大学伯克利分校);University of Chicago(芝加哥大学)
Exploring the Vulnerability of the Content Moderation Guardrail in Large Language Models via Intent Manipulation
Boise State University(博伊州立大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Pittsburgh(匹兹堡大学);New York University(纽约大学);Florida International University(佛罗里达国际大学)
ConsisGuard: Aligning Safety Deliberation with Policy Enforcement in LLM Guardrails
Alibaba Group(阿里巴巴集团);Zhejiang University(浙江大学);Huzhou Normal University(湖州师范学院);Zhejiang Key Laboratory of Intelligent Education Technology and Application(浙江省智能教育技术与应用重点实验室)
Intrinsic Guardrails: How Semantic Geometry of Personality Interacts with Emergent Misalignment in LLMs
International Institute of Information Technology, Hyderabad(国际信息研究所,海得拉巴);Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi(Mohamed bin Zayed人工智能大学,阿布扎比)
Reflex-Guard: A Low-Latency Guardrail for LLM Prompt Safety Using Dense Semantic Embeddings
Graduate School of Informatics, Osaka Metropolitan University(大阪公立大学情报学研究科);BRAC University(BRAC大学);Bangladesh University of Engineering and Technology (BUET)(孟加拉工程技术大学);New Mexico State University(新墨西哥州立大学)
ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback
National Engineering Research Center for Software Engineering, Peking University(软件工程国家工程研究中心,北京大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
The Need for Guardrails with Large Language Models in Medical Safety-Critical Settings: An Artificial Intelligence Application in the Pharmacovigilance Ecosystem
Harvard-MIT Department of Health Sciences and Technology(哈佛-麻省理工卫生科学与技术部);GlaxoSmithKline(葛兰素史克公司);London School of Hygiene and Tropical Medicine(伦敦卫生与热带医学院);Harvard T.H. Chan School of Public Health(哈佛陈曾熙公共卫生学院)