Although there has been rapid progress in endowing robots with the ability to solve complex manipulation tasks, generating control policies for bimanual robots to solve tasks involving two hands is still challenging because of the difficulties in effective temporal and spatial coordination. With emergent abilities in terms of step-by-step reasoning and in-context learning, Large Language Models (LLMs) have demonstrated promising potential in a variety of robotic tasks. However, the nature of language communication via a single sequence of discrete symbols makes LLM-based coordination in continuous space a particular challenge for bimanual tasks. To tackle this challenge, we present LAnguage-model-based Bimanual ORchestration (LABOR), an agent utilizing an LLM to analyze task configurations and devise coordination control policies for addressing long-horizon bimanual tasks. We evaluate our method through simulated experiments involving two classes of long-horizon tasks using the NICOL humanoid robot. Our results demonstrate that our method outperforms the baseline in terms of success rate. Additionally, we thoroughly analyze failure cases, offering insights into LLM-based approaches in bimanual robotic control and revealing future research trends. The project website can be found at http://labor-agent.github.io.
HumanoidGen: Data Generation for Bimanual Dexterous Manipulation via LLM Reasoning
Fudan University(复旦大学);Institute of Artificial Intelligence (TeleAI), China Telecom(人工智能研究所(TeleAI),中国电信);University of Science and Technology of China(中国科学技术大学);East China University of Science and Technology(东华大学)
Vision Language Action (VLA) Models for Unmanned Aerial Robotics and Bimanual Manipulation: A Review
Chef Robotics;RovifyLab;IT Application Research Center, Jeonbuk Regional Branch;Department of Electrical, Computer and Software Engineering(电气与计算机软件工程系)
UniBiDex: A Unified Teleoperation Framework for Robotic Bimanual Dexterous Manipulation
Department of Computer Science, The University of Hong Kong(香港大学计算机科学系);Dongguan Key Laboratory of Intelligent Equipment and Smart Industry, School of Advanced Engineering, The Great Bay University(东莞智能装备与智能制造重点实验室,高级工程学院,大湾区大学);Department of Mechanical Engineering, The Hong Kong Polytechnic University(香港理工大学机械工程系);Guangdong Academy of Sciences(广东省科学院)
CUBic: Coordinated Unified Bimanual Perception and Control Framework
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University(北京未来区块链与隐私计算高级创新中心,人工智能学院,北京航空航天大学);Westlake University(西湖大学);Zhejiang University(浙江大学);Peking University(北京大学)
Sim-and-Human Co-training for Data-Efficient and Scene-Generalizable Bimanual Manipulation
University of Electronic Science and Technology of China(电子科学与技术大学);Southwest Jiaotong University(西南交通大学);Tongji University(同济大学);Shanghai Innovation Institute(上海创新研究院)
RoTri-Diff: A Spatial Robot-Object Triadic Interaction-Guided Diffusion Model for Bimanual Manipulation
School of Computer Science, Nanjing University(南京大学计算机科学学院);School of Intelligence Science and Technology, Nanjing University(南京大学智能科学与技术学院);School of Computing, National University of Singapore(新加坡国立大学计算机学院);Department of Computer Science and Engineering, The Hong Kong University of Science and Technology(香港科技大学计算机科学与工程系)
Integrating Ergonomics and Manipulability for Upper Limb Postural Optimization in Bimanual Human-Robot Collaboration
Department of Mechanical and Automation Engineering, T-Stone Robotics Institute, The Chinese University of Hong Kong(机械与自动化工程系、T-Stone机器人研究所、香港中文大学);Bristol Robotics Laboratory, UWE Bristol(布里斯托尔机器人实验室、UWE布里斯托尔);HKCLR, Hong Kong(香港物流机器人中心、香港)
Information-Theoretic Detection of Bimanual Interactions for Dual-Arm Robot Plan Generation
Human-Robot Interfaces and Interaction (HRII) Laboratory, Istituto Italiano di Tecnologia(人机交互(HRII)实验室,意大利技术研究院);Dept. of Informatics, Bioengineering, Robotics, and Systems Engineering, University of Genoa(信息学、生物工程、机器人学和系统工程系,热那亚大学)
UltraDexGrasp: Learning Universal Dexterous Grasping for Bimanual Robots with Synthetic Data
Shanghai AI Laboratory(上海人工智能实验室);The Chinese University of Hong Kong(香港中文大学);Zhejiang University(浙江大学);The University of Hong Kong(香港大学);Peking University(北京大学)
Morphologically Equivariant Flow Matching for Bimanual Mobile Manipulation
PEARL Lab, Dept. of Computer Science, TU Darmstadt, Germany(德累斯顿技术大学计算机科学系PEARL实验室);Istituto Italiano di Tecnologia(意大利技术研究院);Robotics Institute Germany(德国机器人研究所)
TOCALib: Optimal control library with interpolation for bimanual manipulation and obstacles avoidance
FRC CSC RAS(俄罗斯科学院联邦研究中心计算机科学与控制研究所);MIPT(莫斯科物理技术学院);Robotics Center, Sberbank of Russia(俄罗斯联邦储蓄银行机器人中心);Skolkovo Institute of Science and Technology(斯科尔科沃科学技术学院);AIRI(人工智能研究院);Moscow Institute of Physics and Technology(莫斯科物理技术学院)