RoboPack:学习触觉感知的动力学模型用于密集装箱
RoboPack: Learning Tactile-Informed Dynamics Models for Dense Packing
AI总结:
RoboPack 框架通过循环图神经网络融合视觉与触觉数据学习动力学模型,使机器人在仅约 30 分钟真实交互数据训练后即可完成非抓取式操作和密集装箱任务,并在长时域预测和实际操作中优于现有方法。
摘要:
触觉反馈对于理解许多操作任务中刚体和可变形物体的动力学至关重要,例如非抓取式操作和密集装箱。我们提出了一种结合视觉和触觉感知的机器人操作方法,通过学习一个神经网络的、触觉感知的动力学模型来实现。我们提出的框架 RoboPack 采用循环图神经网络,从历史的视觉-触觉观测中估计物体状态,包括粒子级和物体级潜在物理信息,并进行未来状态预测。我们的触觉感知动力学模型从真实世界数据中学习,可以通过模型预测控制解决下游机器人任务。我们在配备柔性 Soft-Bubble 触觉传感器的真实机器人上演示了该方法,用于非抓取式操作和密集装箱任务,其中机器人必须从直接和间接交互中推断物体的物理属性。每个任务仅使用平均 30 分钟的真实世界交互数据进行训练,我们的模型就能进行在线自适应并做出触觉感知的预测。通过在长时域动力学预测和真实世界操作中的广泛评估,与之前基于学习的方法和基于物理的仿真系统相比,我们的方法展示了优越的有效性。
英文摘要:
Tactile feedback is critical for understanding the dynamics of both rigid and deformable objects in many manipulation tasks, such as non-prehensile manipulation and dense packing. We introduce an approach that combines visual and tactile sensing for robotic manipulation by learning a neural, tactile-informed dynamics model. Our proposed framework, RoboPack, employs a recurrent graph neural network to estimate object states, including particles and object-level latent physics information, from historical visuo-tactile observations and to perform future state predictions. Our tactile-informed dynamics model, learned from real-world data, can solve downstream robotics tasks with model-predictive control. We demonstrate our approach on a real robot equipped with a compliant Soft-Bubble tactile sensor on non-prehensile manipulation and dense packing tasks, where the robot must infer the physics properties of objects from direct and indirect interactions. Trained on only an average of 30 minutes of real-world interaction data per task, our model can perform online adaptation and make touch-informed predictions. Through extensive evaluations in both long-horizon dynamics prediction and real-world manipulation, our method demonstrates superior effectiveness compared to previous learning-based and physics-based simulation systems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。