arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.21746cs.RO
收藏

Avi:从体积推断中生成动作

Avi: Action from Volumetric Inference

University of California, Los Angeles(加州大学洛杉矶分校);University of pennsylvania(宾夕法尼亚大学)

Harris Song, Long Le

AI总结:

Avi提出一种3D视觉-语言-动作架构,利用3D点云和语言理解,通过经典几何变换生成动作,无需策略学习,实现对遮挡和视角变化的鲁棒性。

摘要:

我们提出Avi,一种新颖的3D视觉-语言-动作(VLA)架构,它将机器人动作生成重新定义为3D感知和空间推理问题,而非低级策略学习。现有VLA模型主要基于2D视觉输入,并在任务特定动作策略上进行端到端训练,而Avi利用3D点云和语言引导的场景理解,通过经典几何变换计算动作。最值得注意的是,Avi不训练先前的动作令牌,而是基于3D多模态大语言模型(MLLM)生成下一个点云,并通过经典变换显式计算动作。这种方法实现了对遮挡、相机姿态变化和视角变化具有鲁棒性的泛化行为。通过将机器人决策过程视为对3D表示的结构化推理任务,Avi弥合了高级语言指令与低级驱动之间的差距,无需不透明的策略学习。我们的初步结果凸显了3D视觉-语言推理作为可扩展、稳健机器人系统基础的潜力。详情请访问https://avi-3drobot.github.io/。

英文摘要:

We propose Avi, a novel 3D Vision-Language-Action (VLA) architecture that reframes robotic action generation as a problem of 3D perception and spatial reasoning, rather than low-level policy learning. While existing VLA models primarily operate on 2D visual inputs and are trained end-to-end on task-specific action policies, Avi leverages 3D point clouds and language-grounded scene understanding to compute actions through classical geometric transformations. Most notably, Avi does not train on previous action tokens, rather, we build upon a 3D Multi-modal Large Language Model (MLLM) to generate the next point cloud and explicitly calculate the actions through classical transformations. This approach enables generalizable behaviors that are robust to occlusions, camera pose variations, and changes in viewpoint. By treating the robotic decision-making process as a structured reasoning task over 3D representations, Avi bridges the gap between high-level language instructions and low-level actuation without requiring opaque policy learning. Our preliminary results highlight the potential of 3D vision-language reasoning as a foundation for scalable, robust robotic systems. Check it out at https://avi-3drobot.github.io/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑