主动视觉或许是你所需的一切：探索双臂机器人操作中的主动视觉
Active Vision Might Be All You Need: Exploring Active Vision in Bimanual Robotic Manipulation
AI总结：
本文提出AV-ALOHA双臂遥操作系统，通过额外7自由度机器人手臂携带立体相机并由人类演示学习主动视觉策略，动态优化相机视点，实验证明其在可见性受限任务中显著优于固定相机。
摘要：
模仿学习在利用视觉反馈执行高精度操作任务方面已展现出显著潜力。然而，在模仿学习中，通常的做法是将相机固定放置，这会导致遮挡和视野受限等问题。此外，相机往往被放置在宽泛的通用位置，缺乏针对机器人任务的有效视点。在本工作中，我们研究了主动视觉（AV）在模仿学习和操作中的效用，其中，除了操作策略之外，机器人还从人类演示中学习一种主动视觉策略，以动态改变机器人的相机视点，从而获得关于其环境和给定任务的更好信息。我们引入了AV-ALOHA，一个带有主动视觉的新型双臂遥操作机器人系统，它是ALOHA 2机器人系统的扩展，集成了一个额外的7自由度机器人手臂，该手臂仅携带一台立体相机，其唯一任务是寻找最佳视点。该相机将立体视频流传输给佩戴虚拟现实（VR）头显的操作员，使操作员能够通过头部和身体运动来控制相机位姿。该系统提供了沉浸式遥操作体验，具备双臂第一人称控制能力，使操作员能够动态探索和搜索场景，并同时与环境进行交互。我们在真实世界和仿真环境中对我们的系统进行了模仿学习实验，涵盖多种强调视点规划的任务。我们的结果证明了人类引导的主动视觉在模仿学习中的有效性，表明在可见性受限的任务中，与固定相机相比有显著改进。项目网站：https://soltanilara.github.io/av-aloha/
英文摘要：
Imitation learning has demonstrated significant potential in performing high-precision manipulation tasks using visual feedback. However, it is common practice in imitation learning for cameras to be fixed in place, resulting in issues like occlusion and limited field of view. Furthermore, cameras are often placed in broad, general locations, without an effective viewpoint specific to the robot's task. In this work, we investigate the utility of active vision (AV) for imitation learning and manipulation, in which, in addition to the manipulation policy, the robot learns an AV policy from human demonstrations to dynamically change the robot's camera viewpoint to obtain better information about its environment and the given task. We introduce AV-ALOHA, a new bimanual teleoperation robot system with AV, an extension of the ALOHA 2 robot system, incorporating an additional 7-DoF robot arm that only carries a stereo camera and is solely tasked with finding the best viewpoint. This camera streams stereo video to an operator wearing a virtual reality (VR) headset, allowing the operator to control the camera pose using head and body movements. The system provides an immersive teleoperation experience, with bimanual first-person control, enabling the operator to dynamically explore and search the scene and simultaneously interact with the environment. We conduct imitation learning experiments of our system both in real-world and in simulation, across a variety of tasks that emphasize viewpoint planning. Our results demonstrate the effectiveness of human-guided AV for imitation learning, showing significant improvements over fixed cameras in tasks with limited visibility. Project website: https://soltanilara.github.io/av-aloha/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
