arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.19384cs.CR
收藏

MMA-SafetyBench:多模态智能体安全评估基准

MMA-SafetyBench: A Benchmark for Multimodal Agent Safety Evaluation

Yuvion Team, Alibaba Group(阿里集团 Yuvion 团队);Laboratory for Statistical Monitoring and Intelligent Governance of Common Prosperity, School of Statistics and Data Science, Zhejiang Gongshang University(浙江工商大学统计与数据科学学院共同富裕统计监测与智能治理实验室);Jinshengyuliang AI(金胜雨良 AI)

Yuke Wang, Benlei Cui, Shen Pang, Xuemei Dong, Longtao Huang, Hui Xue, Yuwen Zhai, Junjie Li, Jingqun Tang, Haiwen Hong

AI总结:

本研究针对多模态智能体的视觉语义劫持问题,提出轨迹引导程序构建了含1083条对抗轨迹的MMA-SafetyBench基准,评估显示攻击成功率达87.96%,安全优先提示可降低但无法消除攻击风险。

摘要:

多模态智能体日益依赖视觉接口感知状态并选择动作,产生了仅文本安全评估无法覆盖的攻击面。我们研究视觉语义劫持,即伪造系统警报等具有欺骗性但语义有意义的界面元素使智能体偏离授权任务。我们提出轨迹引导程序,识别候选注入点并将可复现视觉有效载荷插入观测流。利用该程序,我们构建了MMA-SafetyBench,包含网页自动化、桌面GUI控制、移动导航、文档理解和视频处理领域的1083条对抗轨迹。在评估的9个模型中,攻击成功率(ASR)达87.96%,语义破坏率(SCR)达98.09%。1-3-5评分方案区分目标匹配输出、无目标匹配的语义破坏及良性任务的成功保留。覆盖9个模型和5个领域的对照实验表明,攻击在中性提示下仍持续存在,安全优先提示可降低ASR但无法消除该失效模式。

英文摘要:

Multimodal agents increasingly rely on visual interfaces to perceive state and select actions, creating an attack surface not captured by text-only safety evaluations. We study Visual Semantic Hijacking, in which deceptive but semantically meaningful interface elements, such as forged system alerts, cause an agent to depart from its authorized task. We propose a trajectory-guided procedure that identifies candidate injection points and inserts reproducible visual payloads into the observation stream. Using this procedure, we construct MMA-SafetyBench, which contains 1,083 adversarial trajectories across web automation, desktop GUI control, mobile navigation, document understanding, and video processing. Across nine evaluated models, Attack Success Rate (ASR) reaches 87.96%, while Semantic Compromise Rate (SCR) reaches 98.09%. A 1--3--5 scoring scheme distinguishes target-matching outputs, semantic disruption without a target match, and successful preservation of the benign task. Controls covering all nine models and five domains show that attacks persist under neutral prompts. A safety-priority prompt lowers ASR without eliminating the failure mode.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑