SSMNBench: 通过单视图充分性与多视图必要性诊断基于图像的跨视角人-物理解
SSMNBench: Diagnosing Image-based Cross-View Human-Object Understanding via Single-View Sufficiency and Multi-View Necessity
AI总结：
提出SSMNBench基准，通过单视图充分性(SVS)和多视图必要性(MVN)任务分类，诊断MLLM在跨视角人-物理解中的视觉干扰退化和跨视角融合失败问题。
摘要：
多模态大语言模型(MLLMs)在单图像感知方面取得了显著进展，但其对复杂跨视角以人为中心的场景的推理能力仍未得到充分验证。当前的多视角基准使用固定的“帧袋”评估模型，从而将模型对视觉干扰的鲁棒性与融合碎片化跨视角证据的真实能力混为一谈。为解决这一问题，我们引入了SSMNBench，这是一个诊断基准，包含3,300个精心设计的问答对，用于跨视角人和人-物理解。SSMNBench独特地将任务分为单视图充分性(SVS)和多视图必要性(MVN)。通过系统性地扰动17个最先进MLLM的视图可用性，揭示了关键局限性：模型在面对冗余视图(SVS)时遭受严重的“干扰退化”，并且无法跨摄像机整合碎片化的几何证据(MVN)。我们的评估表明，现代MLLM依赖于多个单图像的语义平均和视图偏好，而非真正的跨视角合成。通过暴露这些基本脆弱性，SSMNBench提供了一个严格的诊断框架，以推动未来跨视角感知多模态架构的发展。代码可在以下网址获取：$ \href{ this https URL }{\text{SSMNBench}} $
英文摘要：
Multimodal Large Language Models (MLLMs) have shown remarkable progress in single-image perception, yet their ability to reason about complex cross-view human-centric scenes remains largely unverified. Current multi-view benchmarks evaluate models using a fixed "bag of frames" and thus conflate a model's robustness to visual distraction with its genuine ability to fuse fragmented cross-view evidence. To address this issue, we introduce SSMNBench, a diagnostic benchmark comprising 3,300 curated QA pairs for cross-view human and human-object understanding. SSMNBench uniquely categorizes tasks into Single-View Sufficiency (SVS) and Multi-View Necessity (MVN). By systematically perturbing view availability across 17 state-of-the-art MLLMs, critical limitations are revealed: models suffer from severe "distraction degradation" when presented with redundant views (SVS), and fail to integrate fragmented geometric evidence across cameras (MVN). Our evaluations demonstrate that modern MLLMs rely on multiple single-image semantic averaging and view preference rather than genuine cross-view synthesis. By exposing these fundamental vulnerabilities, SSMNBench provides a rigorous diagnostic framework to drive the advancement of future cross-view-aware multimodal architectures. The code is available at: $ \href{https://github.com/gtc-gh/SSMNBench}{\text{SSMNBench}} $
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
