arXiv 2601.067572026-01-11cs.CLcs.AI

MTMCS-Bench: 多轮对话中多模态大语言模型上下文安全性的评估

MTMCS-Bench: Evaluating Contextual Safety of Multimodal Large Language Models in Multi-Turn Dialogues

University of Notre Dame（诺丁汉大学）；University of California, Los Angeles（加州大学洛杉矶分校）；Georgia Institute of Technology（佐治亚理工学院）；University of Montreal（蒙特利尔大学）

Zheyuan Liu, Dongwhi Kim, Yixin Wan, Xiangchi Yuan, Zhaoxuan Tan, Fengran Mo, Meng Jiang

备注：: A benchmark of realistic images and multi-turn conversations that evaluates contextual safety in MLLMs under two complementary settings

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

MTMCS-Bench评估多模态大语言模型在多轮对话中的上下文安全性，揭示了安全与效用之间的权衡及现有防护措施的不足。

摘要：

多模态大语言模型（MLLMs）正被越来越多地部署为通过文本和图像交互的助手，因此在风险取决于视觉场景和演变对话时，评估上下文安全性至关重要。现有的上下文安全基准大多是单轮的，往往忽略了恶意意图如何逐渐出现，或者同一场景如何支持良性与利用性目标。我们引入了多轮多模态上下文安全基准（MTMCS-Bench），这是一个包含真实图像和多轮对话的基准，评估MLLMs在两种互补设置下的上下文安全性：基于升级的风险和上下文切换风险。MTMCS-Bench提供了配对的安全和不安全对话，并具有结构化的评估。它包含超过30,000个多模态（图像+文本）和单模态（纯文本）样本，其指标分别衡量上下文意图识别、对不安全案例的安全意识以及对良性案例的有用性。在八个开源和七个专有MLLMs上，我们观察到上下文安全性和效用之间存在持续的权衡，模型倾向于要么忽略逐渐的风险，要么过度拒绝良性对话。最后，我们评估了五个当前的防护措施，发现它们减轻了一些失败，但未能完全解决多轮上下文风险。

英文摘要：

Multimodal large language models (MLLMs) are increasingly deployed as assistants that interact through text and images, making it crucial to evaluate contextual safety when risk depends on both the visual scene and the evolving dialogue. Existing contextual safety benchmarks are mostly single-turn and often miss how malicious intent can emerge gradually or how the same scene can support both benign and exploitative goals. We introduce the Multi-Turn Multimodal Contextual Safety Benchmark (MTMCS-Bench), a benchmark of realistic images and multi-turn conversations that evaluates contextual safety in MLLMs under two complementary settings, escalation-based risk and context-switch risk. MTMCS-Bench offers paired safe and unsafe dialogues with structured evaluation. It contains over 30 thousand multimodal (image+text) and unimodal (text-only) samples, with metrics that separately measure contextual intent recognition, safety-awareness on unsafe cases, and helpfulness on benign ones. Across eight open-source and seven proprietary MLLMs, we observe persistent trade-offs between contextual safety and utility, with models tending to either miss gradual risks or over-refuse benign dialogues. Finally, we evaluate five current guardrails and find that they mitigate some failures but do not fully resolve multi-turn contextual risks.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。