AI 中文总结
该研究推出COMPLEXITYWORLD基准测试VLMs在可验证视觉决策任务上的表现，发现多数模型在直接推理下验证器接受率不足40%，且存在视觉到决策的瓶颈。
AI 中文摘要
视觉语言模型（VLMs）在视觉感知领域已取得快速进展，且越来越多地支持依赖图像的现实世界任务。然而，许多此类任务所需的不仅仅是识别图像内容：模型必须利用视觉证据做出完整决策，其各部分需共同满足全局约束。我们推出COMPLEXITYWORLD，这是一个包含39个领域启发的视觉世界、29个决策类别的390项任务的基准。每项任务由隐藏的结构化规范生成，渲染为视觉场景，并由可接受任何可行解决方案的可执行验证器评分。在直接推理下，除GPT-5.6-Sol外，所有评估模型的验证器接受率（VAR）均低于40%，而GPT-5.6-Sol达到75.6%。当相同决策信息以结构化形式明确呈现时，性能显著提升，但在等效视觉呈现中差异明显。智能体支架带来较小的、依赖模型的增益。这些结果共同揭示了仅靠额外推理无法消除的持续存在的视觉到决策瓶颈。
英文摘要
Vision-language models (VLMs) have made rapid progress in visual perception and increasingly support real-world tasks that depend on images. Many such tasks, however, require more than rec- ognizing what an image contains: a model must use visual evidence to make a complete decision whose parts jointly satisfy global constraints. We introduce COMPLEXITYWORLD, a benchmark of 390 tasks across 39 domain-inspired visual worlds and 29 decision categories. Each task is generated from a hidden structured specification, rendered as a visual scene, and scored by an exe- cutable verifier that accepts any feasible solution. Under direct inference, all evaluated models ex- cept GPT-5.6-Sol remain below 40% verifier ac- ceptance rate (VAR), while GPT-5.6-Sol reaches 75.6%. Performance improves substantially when the same decision information is made explicit in structured form, yet varies sharply across equiva- lent visual presentations. Agent scaffolds provide smaller, model-dependent gains. Together, these results reveal a persistent visual-to-decision bot- tleneck that additional inference alone does not remove.
