ST-BiBench：多流多模态协调在双臂具身任务中的基准测试
ST-BiBench: Benchmarking Multi-Stream Multimodal Coordination in Bimanual Embodied Tasks for MLLMs
AI总结：
本文提出ST-BiBench框架，用于评估多流多模态协调，揭示MLLMs在高阶策略与精细物理执行间的协调悖论。
摘要：
多模态大语言模型（MLLMs）显著推动了具身AI的发展，但转向同步双臂协调带来了多流多模态整合的挑战。我们引入ST-BiBench，一个全面的多级框架，用于评估时空多模态协调。我们的方法聚焦于战略协调规划，评估多个动作和感知流间的跨模态推理。为探究“接近悖论”——语义连贯的计划与空间接地的视觉输入不一致的问题，我们引入基础空间接地以验证工作空间意识和手臂选择逻辑。此外，我们通过细粒度动作控制探索模型前沿，研究MLLMs能否直接从复杂多模态元数据合成高维连续动作模态（16维）。评估30余种最先进的MLLMs后，我们发现存在一个普遍的“协调悖论”——高阶策略推理与精细物理执行之间存在显著差距。结果表明，尽管前沿MLLMs在逻辑驱动策略上表现优异，但在多模态融合过程中常出现感知-逻辑脱节和多流干扰问题。ST-BiBench为识别多流多模态融合和跨模态对齐在复杂具身任务中的关键瓶颈提供了平台。
英文摘要：
Multimodal Large Language Models (MLLMs) have significantly advanced the landscape of embodied AI, yet transitioning to synchronized bimanual coordination introduces formidable challenges in multi-stream multimodal integration. We introduce ST-BiBench, a comprehensive multi-tier framework for evaluating spatio-temporal multimodal coordination. Our approach centers on Strategic Coordination Planning, assessing high-level cross-modal reasoning over multiple action and perception streams. To investigate the "proximity paradox"-where semantically coherent plans fail to align with spatially grounded visual inputs-we incorporate Foundational Spatial Grounding to verify workspace awareness and arm-selection logic. Furthermore, we probe model frontiers through Fine-Grained Action Control, investigating whether MLLMs can directly synthesize high-dimensional continuous action modalities (16-Dim) from complex multimodal metadata. Evaluating 30+ state-of-the-art MLLMs, we uncover a persistent and pervasive "coordination paradox"-a significant gap between high-level strategic reasoning and fine-grained physical execution. Results reveal that while frontier MLLMs excel at logic-driven strategy, they frequently suffer from perception-logic disconnection and multi-stream interference during multimodal fusion. ST-BiBench provides a platform for identifying critical bottlenecks in multi-stream multimodal fusion and cross-modal alignment for complex embodied tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
