arXiv 2507.198172025-07-26cs.RO

Ag2x2：面向零样本双臂操作的鲁棒智能体无关视觉表示

Ag2x2: Robust Agent-Agnostic Visual Representations for Zero-Shot Bimanual Manipulation

National Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence (BIGAI)（国家关键人工智能实验室，北京通用人工智能研究院（BIGAI））；School of Psychological and Cognitive Sciences, Peking University（北京大学心理与认知科学学院）；Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）；Beijing Key Laboratory of Behavior and Mental Health, Peking University（北京行为与心理健康重点实验室，北京大学）；Yuanpei College, Peking University（北京大学元培学院）；Department of Computer Science and Technology, University of Cambridge（剑桥大学计算机科学与技术系）；Embodied Intelligence Lab, PKU-Wuhan Institute for Artificial Intelligence（具身智能实验室，北京大学-武汉人工智能研究院）

Ziyin Xiong, Yinghan Chen, Puhao Li, Yixin Zhu, Tengyu Liu, Siyuan Huang

备注：: Accepted to IROS 2025, oral presentation. Project page link: https://ziyin-xiong.github.io/ag2x2.github.io/

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对现有零样本单臂操作方法忽略双臂协调所需智能体特定信息的问题，提出具备协调感知视觉表示的Ag2x2框架，在多基准任务上表现优于基线及专家奖励训练策略，还可支撑无专家监督的模仿学习。

摘要：

双臂操作是人类日常活动的基础，但由于其协调控制的固有复杂性，至今仍是一项具有挑战性的任务。近期的研究进展通过从人类视频中提取的智能体无关视觉表示，实现了单臂操作技能的零样本学习；然而，这些方法忽略了双臂协调所需的关键智能体特定信息，例如末端执行器位置。我们提出Ag2x2，这是一个用于双臂操作的计算框架，通过具备协调感知能力的视觉表示，在保持智能体无关性的同时，联合编码物体状态和手部运动模式。大量实验表明，Ag2x2在来自Bi-DexHands和PerAct2的13项不同双臂任务中取得了73.5%的成功率，包括涉及绳索等可变形物体的挑战性场景。这一性能优于基线方法，甚至超过了使用专家设计奖励训练的策略的成功率。此外，我们证明通过Ag2x2学习到的表示可有效用于模仿学习，构建了一个无需专家监督的可扩展技能获取流程。Ag2x2在无需人类演示或人工设计奖励的情况下，仍能在各类任务中保持鲁棒性能，这标志着朝着可扩展学习复杂双臂机器人技能迈出了一步。

英文摘要：

Bimanual manipulation, fundamental to human daily activities, remains a challenging task due to its inherent complexity of coordinated control. Recent advances have enabled zero-shot learning of single-arm manipulation skills through agent-agnostic visual representations derived from human videos; however, these methods overlook crucial agent-specific information necessary for bimanual coordination, such as end-effector positions. We propose Ag2x2, a computational framework for bimanual manipulation through coordination-aware visual representations that jointly encode object states and hand motion patterns while maintaining agent-agnosticism. Extensive experiments demonstrate that Ag2x2 achieves a 73.5% success rate across 13 diverse bimanual tasks from Bi-DexHands and PerAct2, including challenging scenarios with deformable objects like ropes. This performance outperforms baseline methods and even surpasses the success rate of policies trained with expert-engineered rewards. Furthermore, we show that representations learned through Ag2x2 can be effectively leveraged for imitation learning, establishing a scalable pipeline for skill acquisition without expert supervision. By maintaining robust performance across diverse tasks without human demonstrations or engineered rewards, Ag2x2 represents a step toward scalable learning of complex bimanual robotic skills.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。