arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2505.17455cs.CLcs.AI
收藏

迈向评估多模态语言模型的主动风险意识

Towards Evaluating Proactive Risk Awareness of Multimodal Language Models

School of Data Science, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)数据科学学院);Xiaohongshu Inc.(小红书公司);Renmin University of China(中国人民大学);Johns Hopkins University(约翰霍普金斯大学)

Youliang Yuan, Wenxiang Jiao, Yuejin Xie, Chihao Shen, Menghan Tian, Wenxuan Wang, Jen-tse Huang, Pinjia He

AI总结:

本文提出 PaSBench 多模态主动安全基准,用416个场景评估36个模型,发现顶尖模型仍会遗漏大量风险,主要瓶颈是主动推理不稳定。

摘要:

人类的安全意识缺口往往会阻碍对日常风险的及时识别。在解决这一问题时,主动式安全人工智能(AI)系统会比被动反应式系统表现更好。它不只是对用户的问题作出反应,还会主动观察人们的行为及其环境,以提前发现潜在危险。我们的 Proactive Safety Bench(PaSBench)通过覆盖5个安全关键领域的416个多模态场景(128个图像序列、288个文本日志)来评估这种能力。对36个先进模型的评估揭示了根本性局限:Gemini-2.5-pro 等表现最佳的模型在图像和文本上分别达到71%和64%的准确率,但在重复试验中会遗漏45%至55%的风险。通过失败分析,我们发现主要局限在于不稳定的主动推理,而非知识缺陷。本工作确立了:(1)一个主动安全基准,(2)模型局限性的系统性证据,以及(3)开发可靠保护性AI的关键方向。我们相信,我们的数据集和发现能够推动更安全的AI助手发展,使其主动预防伤害,而不仅仅是响应请求。我们的数据集可在 https://huggingface.co/datasets/Youliang/PaSBench 获取。

英文摘要:

Human safety awareness gaps often prevent the timely recognition of everyday risks. In solving this problem, a proactive safety artificial intelligence (AI) system would work better than a reactive one. Instead of just reacting to users' questions, it would actively watch people's behavior and their environment to detect potential dangers in advance. Our Proactive Safety Bench (PaSBench) evaluates this capability through 416 multimodal scenarios (128 image sequences, 288 text logs) spanning 5 safety-critical domains. Evaluation of 36 advanced models reveals fundamental limitations: Top performers like Gemini-2.5-pro achieve 71% image and 64% text accuracy, but miss 45-55% risks in repeated trials. Through failure analysis, we identify unstable proactive reasoning rather than knowledge deficits as the primary limitation. This work establishes (1) a proactive safety benchmark, (2) systematic evidence of model limitations, and (3) critical directions for developing reliable protective AI. We believe our dataset and findings can promote the development of safer AI assistants that actively prevent harm rather than merely respond to requests. Our dataset can be found at https://huggingface.co/datasets/Youliang/PaSBench.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑