DynaWM：用于移动物体操纵的基于基础VLA的世界基础模型
DynaWM: A Base-VLA-Guided World Foundation Model for Moving-Object Manipulation
AI总结：
针对移动物体操纵挑战，提出DynaWM模型，用多种编码器提取特征，联合条件流匹配DiT生成动作轨迹，构建数据集评估，相比其他模型有性能提升，消融实验验证各部分作用。
摘要：
尽管视觉-语言-动作（VLA）模型受到广泛关注，但在操纵动态移动物体方面仍存在许多挑战。在大多数现有方法中，端到端正向或逆向动力学模型（即世界模型）被纳入高性能基础VLA架构，这可能会因微调不当而降低预训练良好的基础VLA模型的性能。在本文中，我们提出了DynaWM，一种基于基础VLA的世界基础模型，适用于各种用于移动物体操纵的微调与粗调基础VLA检查点。DynaWM使用基于Mamba-3的动作编码器将基础VLA产生的基础动作块编码为动作条件表示，使用V-JEPA 2.1视觉编码器从多视图观察历史中提取特征，并使用本体感觉状态编码器对机器人手臂的本体感觉状态进行编码。这些特征表示共同条件化一个流匹配DiT，以再生用于移动物体操纵的运动感知动作轨迹。为了进行系统评估，我们构建了DynaGrasp-32基准，涵盖六类移动物体操纵任务，包括速度变化、轨迹变化和多物体操纵，以及DynaGrasp-1600数据集，该数据集由32个场景、1600条示范轨迹和大约153万张图像组成。对于微调的基础VLA检查点，DynaWM分别比SmolVLA、X-VLA、π0和π0.5提高了7.19%、45.31%、1.88%和10.94%。对于粗调的基础VLA检查点，性能分别提高了35.13%、44.06%、35.69%和26.13%。消融实验表明，视觉编码将成功率提高了27.50%，而如果去除动作条件，则成功率降低45.44%。
英文摘要：
Although vision-language-action (VLA) models have received widespread attention, many challenges remain in manipulating dynamic moving objects. In most existing approaches, end-to-end forward or inverse dynamics models, i.e., world models, are incorporated into high-performance base VLA architectures, which may degrade the performance of well-pretrained base VLA models due to inappropriate fine-tuning. In this paper, we propose DynaWM, a base-VLA-guided world foundation model that adapts to a wide variety of fine-tuned and coarse-tuned base-VLA checkpoints for moving-object manipulation. DynaWM uses a Mamba-3-based action encoder to encode the base action chunk produced by the base VLA into an action-conditioning representation, a V-JEPA 2.1 vision encoder to extract features from multi-view observation history, and a proprioceptive state encoder to encode robotic-arm proprioceptive states. These feature representations jointly condition a flow-matching DiT to regenerate motion-aware action trajectories for moving-object manipulation. For systematic evaluation, we construct the DynaGrasp-32 benchmark, covering six categories of moving-object manipulation tasks, including velocity variation, trajectory variation, and multi-object manipulation, as well as the DynaGrasp-1600 dataset, which consists of 32 scenarios, 1,600 demonstration trajectories, and approximately 1.53M images. For fine-tuned base-VLA checkpoints, DynaWM achieves percentage improvements of 7.19, 45.31, 1.88, and 10.94 over SmolVLA, X-VLA, π0, and π0.5, respectively. For coarse-tuned base-VLA checkpoints, performance increases by 35.13, 44.06, 35.69, and 26.13 percentage, respectively. Ablation experiments show that visual encoding enhances success by 27.50%, while reducing success by 45.44% if action conditioning is removed.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
