LLM agents emit actions, not just text, and once taken, those actions often cannot be undone. Yet today's agent-safety evaluations run greedy or a few sampled rollouts and report a single safe/unsafe rate -- blind to the long-tail trajectories where unsafe behavior may arise from low-probability but non-negligible actions.
We argue agent safety should be measured by search, not sampling. We apply BOA, a framework that, given a deployment configuration (model, decoder, prompt, environment, judger, likelihood budget), searches the in-budget trajectory space and reports a safety score: the probability the agent stays safe under the configuration. BOA searches both within a single LLM round and across the agent-environment interaction tree under a given likelihood budget, and makes search practical via batched decoding/judging, prefix caching, and chunked tree expansion. On agent-safety workloads, BOA discovers unsafe trajectories that greedy and sampled evaluations miss. BOA can additionally be used for ranking models, defenses, and attacks, all on the same scale, with manageable GPU costs.
MMA-SafetyBench: A Benchmark for Multimodal Agent Safety Evaluation
Yuvion Team, Alibaba Group(阿里集团 Yuvion 团队);Laboratory for Statistical Monitoring and Intelligent Governance of Common Prosperity, School of Statistics and Data Science, Zhejiang Gongshang University(浙江工商大学统计与数据科学学院共同富裕统计监测与智能治理实验室);Jinshengyuliang AI(金胜雨良 AI)
SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
The University of Hong Kong(香港大学);Shandong University(山东大学);Carnegie Mellon University(卡内基梅隆大学);National University of Singapore(新加坡国立大学);The Hong Kong University of Science and Technology(香港科技大学)
The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agents
University of Wisconsin–Madison(威斯康星大学麦迪逊分校);University of Minnesota(明尼苏达大学);University of Southern California(南加州大学);McGill University(麦吉尔大学);Mila(Mila研究院);MBZUAI(MBZUAI研究院)
ConVerse: Benchmarking Contextual Safety in Agent-to-Agent Conversations
German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心(DFKI));Microsoft(微软);ELLIS Institute Tübingen and MPI for Intelligent Systems(图宾根ELLIS研究所和智能系统研究所);Tübingen AI Center(图宾根人工智能中心)