Emma-X:一种具有基于环境链式思考与前瞻空间推理的具身多模态动作模型
Emma-X: An Embodied Multimodal Action Model with Grounded Chain of Thought and Look-ahead Spatial Reasoning
AI总结:
针对传统机器人控制方法泛化性差及VLA模型长时程空间推理不足的问题,提出Emma-X模型,利用分层具身数据集和轨迹分割策略,在真实世界空间推理任务中取得领先性能。
摘要:
传统的基于强化学习的机器人控制方法通常针对特定任务,难以在多样环境、未见物体或指令之间泛化。视觉语言模型(VLMs)展现出强大的场景理解和规划能力,但缺乏生成针对特定机器人本体可执行策略的能力。为解决此问题,视觉-语言-动作(VLA)模型应运而生,然而它们在长时程空间推理和基于环境的任务规划方面仍面临挑战。在本工作中,我们提出了具有基于环境链式思考与前瞻空间推理的具身多模态动作模型Emma-X。Emma-X利用我们基于BridgeV2构建的分层具身数据集,该数据集包含60,000条机器人操作轨迹,并自动标注了基于环境的任务推理和空间引导。此外,我们引入了一种基于夹爪状态和运动轨迹的轨迹分割策略,该策略有助于减轻在生成子任务推理时出现的幻觉。实验结果表明,Emma-X在竞争性基线方法中取得了更优性能,尤其在需要空间推理的真实世界机器人任务中表现突出。
英文摘要:
Traditional reinforcement learning-based robotic control methods are often task-specific and fail to generalize across diverse environments or unseen objects and instructions. Visual Language Models (VLMs) demonstrate strong scene understanding and planning capabilities but lack the ability to generate actionable policies tailored to specific robotic embodiments. To address this, Visual-Language-Action (VLA) models have emerged, yet they face challenges in long-horizon spatial reasoning and grounded task planning. In this work, we propose the Embodied Multimodal Action Model with Grounded Chain of Thought and Look-ahead Spatial Reasoning, Emma-X. Emma-X leverages our constructed hierarchical embodiment dataset based on BridgeV2, containing 60,000 robot manipulation trajectories auto-annotated with grounded task reasoning and spatial guidance. Additionally, we introduce a trajectory segmentation strategy based on gripper states and motion trajectories, which can help mitigate hallucination in grounding subtask reasoning generation. Experimental results demonstrate that Emma-X achieves superior performance over competitive baselines, particularly in real-world robotic tasks requiring spatial reasoning.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。