FlowVLA：通过可插拔3D流学习实现世界动力学理解
FlowVLA: Enabling World Dynamics Understanding through Pluggable 3D Flow Learning
AI总结：
FlowVLA是一种仅在训练时使用时间对齐3D流的可插拔框架，通过双层级流监督提升VLA模型的机器人操作性能、零样本稳健性与数据效率，且不增加推理负担。
摘要：
近期3D重建领域的进展能够从RGB观测中提取时间一致的3D运动；然而，当前的视觉-语言-动作（VLA）模型仍主要依赖2D图像-文本先验。因此，它们无法准确表示与物理动作相关的3D世界的基本动力学。这种对静态2D视觉线索的过度依赖限制了VLA模型实现稳健机器人操作的能力。为解决这些限制，我们提出FlowVLA，一种重建引导的可插拔3D流学习框架，旨在赋予VLA模型对动作相关动力学的固有理解。我们的方法无需显式3D传感器输入或改变推理时架构，仅在训练期间利用时间对齐的3D流，使模型能够学习物理场景的动态演化，同时保留传统2D VLA骨干网络的计算效率。为连接场景级动力学与机器人操作，我们提出具有双层级流监督的模块化3D流学习框架。具体而言，点级流引导模型预测任务关键3D点的未来位移，而TCP级流则聚合这些运动以与机器人的TCP轨迹对齐。这种双层级流监督鼓励模型聚焦于语义有意义的动态区域，有效将场景级运动与精确控制信号耦合。在仿真和真实世界实验中的广泛评估表明，FlowVLA可持续提升操作性能、零样本稳健性并优化数据效率。FlowVLA仅在训练期间添加轻量级辅助头，推理时将其丢弃，因此无需额外3D传感器输入、架构修改或推理时计算。
英文摘要：
Recent advances in 3D reconstruction enable the extraction of temporally consistent 3D motion from RGB observations; however, current Vision-Language-Action (VLA) models still predominantly rely on 2D image-text priors. Therefore, they do not accurately represent the fundamental dynamics of the 3D world associated with physical actions. This over-reliance on static 2D visual cues limits the capacity of VLA models in achieving robust robot manipulation. To address these limitations, we introduce FlowVLA, a reconstruction-guided, pluggable 3D flow learning framework aimed at empowering VLA models with an inherent understanding of action-relevant dynamics. Instead of requiring explicit 3D sensor inputs or altering the inference-time architecture, our method is designed to utilize temporally aligned 3D flow only during training. This allows the model to learn the dynamic evolution of the physical scene while preserving the computational efficiency of conventional 2D VLA backbones. To bridge scene-level dynamics with robotic manipulation, we propose a modular 3D flow learning framework with dual-level flow supervision. Specifically, point-level flow guides the model to forecast the future displacements of task-critical 3D points, while TCP-level flow aggregates these motions to align with the robot's TCP trajectories. This dual-level flow supervision encourages the model to focus on semantically meaningful dynamic regions, effectively coupling scene-level motion with precise control signals. Extensive evaluations across simulation and real-world experiments demonstrate that FlowVLA consistently improves manipulation performance, zero-shot robustness, and optimizes data efficiency. FlowVLA adds only lightweight auxiliary heads during training and discards them at inference. Therefore, FlowVLA introduces no additional 3D sensor inputs, architectural modifications, or inference-time computation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
