ROVER：面向全模态生成的互惠跨模态推理基准测试
ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation
AI总结：
本文提出了ROVER基准，通过1312个人工标注任务评估统一多模态模型在视觉与言语生成中的互惠跨模态推理能力，实验表明跨模态推理决定生成质量且模型在符号推理上存在缺陷。
摘要：
统一多模态模型（UMMs）已成为无缝统一文本与图像理解与生成的强大范式。然而，现有评估往往孤立对待这些能力，使得具有多模态输入和输出的任务主要通过单模态推理进行评分，即文本基准强调基于语言的推理，而视觉基准强调体现在像素中的推理结果。我们引入ROVER来解决这一迫切需求，以测试互惠跨模态推理，即使用一种模态来引导、验证或细化另一种模态中的输出，这种能力是统一多模态智能愿景的核心。ROVER是一个人工标注的基准，明确针对互惠跨模态推理，包含基于1876张图像的1312个任务，涵盖两个互补的设置。用于视觉生成的言语增强推理评估模型能否使用言语提示和推理链来引导忠实的图像合成。用于言语生成的视觉增强推理评估模型能否生成中间可视化来强化其自身的问答推理过程。在17个统一模型上的实验揭示了两项关键发现：（i）跨模态推理决定了视觉生成质量，交错模型显著优于非交错模型；值得注意的是，组合强大的单模态模型无法实现可比的推理。（ii）模型在物理推理与符号推理之间表现出分离：它们能够从字面上成功解释感知概念，但在为符号任务构建视觉抽象时失败，其中错误的推理损害了性能。这些结果突显了互惠跨模态推理是实现真正全模态生成的关键前沿。
英文摘要：
Unified multimodal models (UMMs) have emerged as a powerful paradigm for seamlessly unifying text and image understanding and generation. However, prevailing evaluations treat these abilities in isolation, such that tasks with multimodal inputs and outputs are scored primarily through unimodal reasoning, i.e., textual benchmarks emphasize language-based reasoning, while visual benchmarks emphasize reasoning outcomes manifested in the pixels. We introduce ROVER to address this pressing need to test reciprocal cross-modal reasoning, the use of one modality to guide, verify, or refine outputs in the other, an ability central to the vision of unified multimodal intelligence. ROVER is a human-annotated benchmark that explicitly targets reciprocal cross-modal reasoning, which contains 1312 tasks grounded in 1876 images, spanning two complementary settings. Verbally-augmented reasoning for visual generation evaluates whether models can use verbal prompts and reasoning chains to guide faithful image synthesis. Visually-augmented reasoning for verbal generation evaluates whether models can generate intermediate visualizations that strengthen their own reasoning processes for question answering. Experiments on 17 unified models reveal two key findings: (i) Cross-modal reasoning determines visual generation quality, with interleaved models significantly outperforming non-interleaved ones; notably, combining strong unimodal models fails to achieve comparable reasoning. (ii) Models show dissociation between physical and symbolic reasoning: they succeed at interpreting perceptual concepts literally but fail to construct visual abstractions for symbolic tasks, where faulty reasoning harms performance. These results highlight reciprocal cross-modal reasoning as a critical frontier for enabling true omnimodal generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
