OmniEVA：通过任务自适应3D grounded和 embodiment-aware推理实现具身 versatile规划
OmniEVA: Embodied Versatile Planner via Task-Adaptive 3D-Grounded and Embodiment-aware Reasoning
AI总结：
OmniEVA通过任务自适应3D grounding和具身aware推理，解决具身系统中几何适应性和具身约束的限制，实现先进的具身推理和任务规划。
摘要：
最近多模态大语言模型（MLLMs）的进步为具身智能开辟了新机会，使多模态理解、推理和交互以及连续空间决策成为可能。然而，当前基于MLLMs的具身系统面临两个关键限制。第一，几何适应性差距：仅在2D输入上训练或使用硬编码3D几何注入的模型，要么缺乏足够的空间信息，要么在2D泛化上受限，导致在具有多样化空间需求的任务中适应性差。第二，具身约束差距：先前的工作往往忽视了真实机器人物理约束和能力，导致任务计划在理论上有效但实际不可行。为了解决这些差距，我们引入了OmniEVA——一个具身 versatile规划器，通过两个关键创新实现高级具身推理和任务规划：（1）任务自适应3D grounding机制，引入一个门控路由器，根据上下文要求执行显式选择性调节3D融合，实现针对多样化具身任务的上下文感知3D grounding。（2）一个结合任务目标和具身约束的推理框架，将任务目标和具身约束纳入推理循环，使规划决策既目标导向又可执行。广泛的实验结果表明，OmniEVA不仅在具身推理性能上达到最先进的水平，而且在广泛下游场景中表现出强大的能力。对一系列提出的具身基准测试的评估，包括原始和复合任务，证实了其稳健和 versatile的规划能力。项目页面：https://omnieva.github.io
英文摘要：
Recent advances in multimodal large language models (MLLMs) have opened new opportunities for embodied intelligence, enabling multimodal understanding, reasoning, and interaction, as well as continuous spatial decision-making. Nevertheless, current MLLM-based embodied systems face two critical limitations. First, Geometric Adaptability Gap: models trained solely on 2D inputs or with hard-coded 3D geometry injection suffer from either insufficient spatial information or restricted 2D generalization, leading to poor adaptability across tasks with diverse spatial demands. Second, Embodiment Constraint Gap: prior work often neglects the physical constraints and capacities of real robots, resulting in task plans that are theoretically valid but practically infeasible. To address these gaps, we introduce OmniEVA -- an embodied versatile planner that enables advanced embodied reasoning and task planning through two pivotal innovations: (1) a Task-Adaptive 3D Grounding mechanism, which introduces a gated router to perform explicit selective regulation of 3D fusion based on contextual requirements, enabling context-aware 3D grounding for diverse embodied tasks. (2) an Embodiment-Aware Reasoning framework that jointly incorporates task goals and embodiment constraints into the reasoning loop, resulting in planning decisions that are both goal-directed and executable. Extensive experimental results demonstrate that OmniEVA not only achieves state-of-the-art general embodied reasoning performance, but also exhibits a strong ability across a wide range of downstream scenarios. Evaluations of a suite of proposed embodied benchmarks, including both primitive and composite tasks, confirm its robust and versatile planning capabilities. Project page: https://omnieva.github.io
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
