ArtGS：面向关节物体交互式视觉-物理建模与操纵的3D高斯泼溅
ArtGS:3D Gaussian Splatting for Interactive Visual-Physical Modeling and Manipulation of Articulated Objects
浏览论文内容
中文总结 AI 辅助
ArtGS将3D高斯泼溅与视觉-物理建模结合，通过VLM提取关节骨骼信息并进行可微渲染优化，实现了关节物体建模与操纵的高精度和强泛化能力。
中文摘要 AI 辅助
由于复杂的运动学约束以及现有方法有限的物理推理能力，关节物体操纵仍然是机器人领域的一个关键挑战。在本文中，我们提出了ArtGS，这是一个新颖的框架，通过整合视觉-物理建模来扩展3D高斯泼溅（3DGS），以实现关节物体的理解与交互。ArtGS首先进行多视角RGB-D重建，随后利用视觉-语言模型（VLM）进行推理，以提取语义和结构信息，特别是关节骨骼。通过基于动态、可微分的3DGS渲染，ArtGS优化关节骨骼的参数，确保物理一致的运动约束并增强操纵策略。通过利用动态高斯泼溅、跨具身适应性和闭环优化，ArtGS建立了一个高效、可扩展且可泛化的关节物体建模与操纵新框架。在仿真和真实环境中进行的实验表明，ArtGS在多种关节物体上的关节估计精度和操纵成功率方面显著优于以往方法。更多图像和视频可在项目网站上获取：https://sites.google.com/view/artgs/home
英文摘要
Articulated object manipulation remains a critical challenge in robotics due to the complex kinematic constraints and the limited physical reasoning of existing methods. In this work, we introduce ArtGS, a novel framework that extends 3D Gaussian Splatting (3DGS) by integrating visual-physical modeling for articulated object understanding and interaction. ArtGS begins with multi-view RGB-D reconstruction, followed by reasoning with a vision-language model (VLM) to extract semantic and structural information, particularly the articulated bones. Through dynamic, differentiable 3DGS-based rendering, ArtGS optimizes the parameters of the articulated bones, ensuring physically consistent motion constraints and enhancing the manipulation policy. By leveraging dynamic Gaussian splatting, cross-embodiment adaptability, and closed-loop optimization, ArtGS establishes a new framework for efficient, scalable, and generalizable articulated object modeling and manipulation. Experiments conducted in both simulation and real-world environments demonstrate that ArtGS significantly outperforms previous methods in joint estimation accuracy and manipulation success rates across a variety of articulated objects. Additional images and videos are available on the project website: https://sites.google.com/view/artgs/home
发表机构
- Shanghai Jiao Tong University（上海交通大学）
- Shanghai AI Laboratory（上海人工智能实验室）
- National University of Singapore（国立新加坡大学）
- Princeton University（普林斯顿大学）
- Stanford University（斯坦福大学）
- Hefei University of Technology（合肥工业大学）
机构由 AI 辅助整理，请以论文原文为准。
