RGBManip：通过主动物体位姿估计实现基于单目图像的机器人操作
RGBManip: Monocular Image-based Robotic Manipulation through Active Object Pose Estimation
AI总结：
本文提出RGBManip框架，利用夹爪上的单目相机进行主动多视角6D物体位姿估计，并通过强化学习平衡位姿精度与操作效率，在仿真和真实环境中取得先进性能。
摘要：
机器人操作需要对环境进行准确感知，而环境固有的复杂性和持续变化的特性使其成为一项重大挑战。在此背景下，RGB图像和点云观测是基于视觉的机器人操作中两种常用模态，但每种模态都有其局限性。由于发射—接收成像原理的限制，商用点云观测往往存在采样稀疏和输出含噪等问题。另一方面，RGB图像虽然富含纹理信息，却缺少机器人操作所必需的关键深度与三维信息。为缓解这些挑战，我们提出一种仅使用图像的机器人操作框架，该框架利用安装在机器人平行夹爪上的手眼单目相机。通过随机器人夹爪一起运动，该相机能够在操作过程中从多个视角主动感知物体。这使得估计可用于操作的物体6D位姿成为可能。虽然从更多且多样化的视角获取图像通常可以改善位姿估计，但也会增加操作时间。为解决这一权衡，我们采用强化学习策略，使操作策略与主动感知同步，在6D位姿精度和操作效率之间取得平衡。我们在仿真和真实环境中的实验结果均展示了本方法的最先进有效性。我们相信，本方法将启发面向真实世界的机器人操作的进一步研究。
英文摘要：
Robotic manipulation requires accurate perception of the environment, which poses a significant challenge due to its inherent complexity and constantly changing nature. In this context, RGB image and point-cloud observations are two commonly used modalities in visual-based robotic manipulation, but each of these modalities have their own limitations. Commercial point-cloud observations often suffer from issues like sparse sampling and noisy output due to the limits of the emission-reception imaging principle. On the other hand, RGB images, while rich in texture information, lack essential depth and 3D information crucial for robotic manipulation. To mitigate these challenges, we propose an image-only robotic manipulation framework that leverages an eye-on-hand monocular camera installed on the robot's parallel gripper. By moving with the robot gripper, this camera gains the ability to actively perceive object from multiple perspectives during the manipulation process. This enables the estimation of 6D object poses, which can be utilized for manipulation. While, obtaining images from more and diverse viewpoints typically improves pose estimation, it also increases the manipulation time. To address this trade-off, we employ a reinforcement learning policy to synchronize the manipulation strategy with active perception, achieving a balance between 6D pose accuracy and manipulation efficiency. Our experimental results in both simulated and real-world environments showcase the state-of-the-art effectiveness of our approach. %, which, to the best of our knowledge, is the first to achieve robust real-world robotic manipulation through active pose estimation. We believe that our method will inspire further research on real-world-oriented robotic manipulation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
