在不确定的物体动力学和几何条件下实现鲁棒操作仍是现实机器人技术的核心挑战。基于固定假设训练的策略,当物体质量、摩擦系数、尺寸或形状发生变化时,性能会下降。基于嵌入的适应方法,如快速运动适应(Rapid Motor Adaptation, RMA),通过将动作条件化于从交互历史推断出的环境动力学潜在表示来解决这一问题。本研究探究该范式是否能从单臂设置扩展到协调的双臂操作,其中更高维度的控制、耦合接触以及共享物体力引入了额外复杂性。我们在三个接触丰富的ManiSkill任务上评估双臂RMA,这些任务采用随机YCB物体属性。双臂RMA利用观测-动作历史和双腕深度图像,在双臂拾取和提升任务上保持特权策略性能,而未进行潜在条件化的匹配固定PPO策略成功率较低;所有方法在长 horizon 堆叠任务上仍存在局限。我们进一步在双Franka提升任务上展示了无需真实世界微调的直接仿真到真实迁移。这些结果共同表明,潜在条件化为将鲁棒控制扩展到协调双臂系统提供了有用结构,而长 horizon 任务需要额外的时间结构。
英文摘要:
Robust manipulation under uncertain object dynamics and geometry remains a core challenge in real-world robotics. Policies trained under fixed assumptions often degrade when object mass, friction, scale, or shape vary. Embedding-based adaptation methods such as Rapid Motor Adaptation (RMA) address this by conditioning actions on latent representations of environment dynamics inferred from interaction history. We investigate whether this paradigm scales from single arm settings to coordinated bimanual manipulation, where higher dimensional control, coupled contacts, and shared object forces introduce additional complexity. We evaluate bimanual RMA on three contact-rich ManiSkill tasks with randomized YCB object properties. Bimanual RMA, using observation-action history and dual wrist depth images, retains privileged policy performance on bimanual picking and lifting, while an otherwise matched stationary PPO policy without latent conditioning achieves low success; all methods remain limited on long-horizon stacking. We further demonstrate direct sim-to-real transfer on a dual Franka lifting task without real-world finetuning. Together, these results suggest that latent conditioning provides useful structure for scaling robust control to coordinated bimanual systems, while longer horizon tasks require additional temporal structure.
CUBic: Coordinated Unified Bimanual Perception and Control Framework
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University(北京未来区块链与隐私计算高级创新中心,人工智能学院,北京航空航天大学);Westlake University(西湖大学);Zhejiang University(浙江大学);Peking University(北京大学)
Department of AI Convergence, Gwangju Institute of Science and Technology (GIST)(人工智能融合系,全州科学技术学院(GIST));Department of AI Machinery, Korea Institute of Machinery & Materials (KIMM)(人工智能机械系,韩国机械材料研究院(KIMM))
ALPHA-$α$ and Bi-ACT Are All You Need: Importance of Position and Force Information/Control for Imitation Learning of Unimanual and Bimanual Robotic Manipulation with Low-Cost System
Osaka University(大阪大学);Graduate School of Information Science and Technology of Osaka University(大阪大学信息科学与技术研究生院);School of Engineering Science, Osaka University(大阪大学工程科学学院)
Morphologically Equivariant Flow Matching for Bimanual Mobile Manipulation
PEARL Lab, Dept. of Computer Science, TU Darmstadt, Germany(德累斯顿技术大学计算机科学系PEARL实验室);Istituto Italiano di Tecnologia(意大利技术研究院);Robotics Institute Germany(德国机器人研究所)
Ag2x2: Robust Agent-Agnostic Visual Representations for Zero-Shot Bimanual Manipulation
National Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence (BIGAI)(国家关键人工智能实验室,北京通用人工智能研究院(BIGAI));School of Psychological and Cognitive Sciences, Peking University(北京大学心理与认知科学学院);Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院);Beijing Key Laboratory of Behavior and Mental Health, Peking University(北京行为与心理健康重点实验室,北京大学);Yuanpei College, Peking University(北京大学元培学院);Department of Computer Science and Technology, University of Cambridge(剑桥大学计算机科学与技术系);Embodied Intelligence Lab, PKU-Wuhan Institute for Artificial Intelligence(具身智能实验室,北京大学-武汉人工智能研究院)
RoTri-Diff: A Spatial Robot-Object Triadic Interaction-Guided Diffusion Model for Bimanual Manipulation
School of Computer Science, Nanjing University(南京大学计算机科学学院);School of Intelligence Science and Technology, Nanjing University(南京大学智能科学与技术学院);School of Computing, National University of Singapore(新加坡国立大学计算机学院);Department of Computer Science and Engineering, The Hong Kong University of Science and Technology(香港科技大学计算机科学与工程系)
PA-BiCoop: A Primary-Auxiliary Cooperative Framework for General Bimanual Manipulation
SGIT AI Lab, State Grid Corporation of China(国家电网公司SGIT人工智能实验室);The Hong Kong University of Science and Technology, Guangzhou(香港科技大学(广州));Baidu Research, Beijing, China(百度研究院);Zhejiang University of Technology, Hangzhou(浙江工业大学)
Task-Aware Bimanual Affordance Prediction via VLM-Guided Semantic-Geometric Reasoning
Department of Computer Science, Technical University of Darmstadt(达姆施塔特工业大学计算机科学系);German Research Center for AI (DFKI)(德国人工智能研究中心);Centre for Cognitive Science, Technical University of Darmstadt(达姆施塔特工业大学认知科学中心);Hessian Center for Artificial Intelligence (Hessian.AI), Darmstadt(黑森州人工智能中心);Robotics Institute Germany (RIG)(德国机器人研究所)
Rethinking Bimanual Robotic Manipulation: Learning with Decoupled Interaction Framework
School of Computer Science and Engineering, Sun Yat-sen University, China(中山大学计算机科学与工程学院);Peng Cheng Laboratory, China(鹏城实验室);Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education, China(教育部人工智能与先进计算重点实验室);Guangdong Province Key Laboratory of Information Security Technology, China(广东省信息安全技术重点实验室);Imperial-X Initiative and Department of Bioengineering, Imperial College London, U.K.(帝国理工学院生物工程系)
BiDexGrasp: Coordinated Bimanual Dexterous Grasps across Object Geometries and Sizes
School of Computer Science and Engineering, Sun Yat-sen University(中山大学计算机科学与工程学院);School of Computer Science, Peking University(北京大学计算机学院);Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院)
HumanoidGen: Data Generation for Bimanual Dexterous Manipulation via LLM Reasoning
Fudan University(复旦大学);Institute of Artificial Intelligence (TeleAI), China Telecom(人工智能研究所(TeleAI),中国电信);University of Science and Technology of China(中国科学技术大学);East China University of Science and Technology(东华大学)
An Optimization-Augmented Control Framework for Single and Coordinated Multi-Arm Robotic Manipulation
Department of Electrical and Electronics Engineering, Middle East Technical University(电子与电气工程系,中东部技术大学);Department of Computer Engineering, Bilkent University(计算机工程系,比尔肯特大学)
Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning
Institute for AI, Peking University(北京大学人工智能研究院);Center on Frontiers of Computing Studies, Peking University(北京大学前沿计算研究中心);University College London(伦敦大学学院);Carnegie Mellon University(卡内基梅隆大学);Beijing Institute for General Artificial Intelligence(北京通用人工智能研究院)
Adapt as You Say: Online Interactive Bimanual Skill Adaptation via Human Language Feedback
Collaborative and Versatile Robots (CLOVER) Laboratory, T-Stone Robotics Institute, The Chinese University of Hong Kong(香港中文大学T-Stone机器人研究所协作与多功能机器人(CLOVER)实验室);Department of Mechanical and Aerospace Engineering, University of California, Los Angeles(加州大学洛杉矶分校机械与航空航天工程系);Department of Advanced Robotics, Istituto Italiano di Tecnologia(意大利技术研究院先进机器人系)
A Neurorobotic Embodiment for Exploring the Dynamical Interactions of a Spiking Cerebellar Model and a Robot Arm During Vision-based Manipulation Tasks
The Hong Kong Polytechnic University(香港理工大学);Technical University of Denmark(丹麦技术大学)
Towards a Generalizable Bimanual Foundation Policy via Flow-based Video Prediction
Institute of Artificial intelligence (TeleAI), China Telecom(人工智能研究院(TeleAI),中国电信);Northwestern Polytechnical University(西北工业大学);Hong Kong University of Science and Technology(香港科技大学)
Self-organized adaptation of a simple neural circuit enables complex robot behaviour
Bernstein Center for Computational Neuroscience(伯恩斯坦计算神经科学中心);Institute for Solid State Physics, ISFH / University of Hannover(汉诺威大学ISFH固态物理研究所);Max Planck Institute for Dynamics & Self-Organization(马克斯·普朗克动力学与自组织研究所);University of Göttingen(哥廷根大学)