SleepWalk：一种三层压力测试基准，用于指导的视觉-语言导航
SleepWalk: A Three-Tier Benchmark for Stress-Testing Instruction-Guided Vision-Language Navigation
AI总结：
本文提出SleepWalk基准，用于评估基于指令的轨迹预测，针对局部化、交互导向的具身推理，揭示当前VLM在空间推理中的系统性失败。
摘要：
视觉-语言模型（VLMs）在多模态感知和语言理解方面迅速发展，但尚不清楚它们是否能可靠地将语言接地为在3D数字环境中空间一致且可能执行的动作。我们引入SleepWalk，一种评估基于指令的轨迹预测的基准，该基准生成自文本场景描述并过滤以确保可导航性。与以往以长距离探索房间为中心的导航基准不同，SleepWalk针对局部化、以交互为中心的具身推理：给定渲染的视觉观察和自然语言指令，模型必须预测一个尊重场景几何、避免碰撞并终止在动作兼容位置的轨迹。该基准涵盖多样化的室内和室外环境，并将任务分为三层空间和时间难度，使在增加的组合复杂性下对接地进行细粒度分析成为可能。使用标准化的点评估评估协议，我们评估了三种前沿VLMs在2,472个经过精心挑选的3D环境中，每个场景有九条指令。结果揭示了在遮挡、交互约束和多步指令下的系统性失败：随着任务难度等级的增加，性能下降。总体而言，当前VLMs可以生成在空间上一致、可能执行且与预期动作一致的轨迹。通过在受控且可扩展的设置中暴露失败，SleepWalk为推进基于接地的多模态推理、具身规划、视觉-语言导航和3D环境中的动作能力代理提供了关键基准。
英文摘要：
Vision-Language Models (VLMs) have advanced rapidly in multimodal perception and language understanding, yet it remains unclear whether they can reliably ground language into spatially coherent, plausibly executable actions in 3D digital environments. We introduce SleepWalk, a benchmark for evaluating instruction-grounded trajectory prediction in single-scene 3D worlds generated from textual scene descriptions and filtered for navigability. Unlike prior navigation benchmarks centered on long-range exploration across rooms, SleepWalk targets localized, interaction-centric embodied reasoning: given rendered visual observations and a natural-language instruction, a model must predict a trajectory that respects scene geometry, avoids collisions, and terminates at an action-compatible location. The benchmark covers diverse indoor and outdoor environments and organizes tasks into three tiers of spatial and temporal difficulty, enabling fine-grained analysis of grounding under increasing compositional complexity. Using a standardized pointwise judge-based evaluation protocol, we evaluate three frontier VLMs on 2,472 curated 3D environments with nine instructions per scene. Results reveal systematic failures in grounded spatial reasoning, especially under occlusion, interaction constraints, and multi-step instructions: performance drops as the difficulty level of the tasks increase. In general, current VLMs can somewhat produce trajectories that are simultaneously spatially coherent, plausibly executable, and aligned with intended actions. By exposing failures in a controlled yet scalable setting, SleepWalk provides a critical benchmark for advancing grounded multimodal reasoning, embodied planning, vision-language navigation, and action-capable agents in 3D environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
