Vision-based teleoperation offers the possibility to endow robots with human-level intelligence to physically interact with the environment, while only requiring low-cost camera sensors. However, current vision-based teleoperation systems are designed and engineered towards a particular robot model and deploy environment, which scales poorly as the pool of the robot models expands and the variety of the operating environment increases. In this paper, we propose AnyTeleop, a unified and general teleoperation system to support multiple different arms, hands, realities, and camera configurations within a single system. Although being designed to provide great flexibility to the choice of simulators and real hardware, our system can still achieve great performance. For real-world experiments, AnyTeleop can outperform a previous system that was designed for a specific robot hardware with a higher success rate, using the same robot. For teleoperation in simulation, AnyTeleop leads to better imitation learning performance, compared with a previous system that is particularly designed for that simulator. Project page: https://yzqin.github.io/anyteleop/.
TeleOpBench: A Simulator-Centric Benchmark for Dual-Arm Dexterous Teleoperation
Shanghai AI Laboratory(上海人工智能实验室);Zhejiang University(浙江大学);The Chinese University of Hong Kong(香港中文大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The University of Hong Kong(香港大学);Feeling AI(FeelAI);Tsinghua Shenzhen International Graduate School(清华大学深圳国际研究生院)
CDF-Glove: A Cable-Driven Force Feedback Glove for Dexterous Teleoperation
Center for Artificial Intelligence and Robotics, Shenzhen International Graduate School, Tsinghua University(人工智能与机器人中心,深圳国际研究生院,清华大学);PKU-Psibot Joint Lab(北京大学-PsiBot联合实验室);Department of Advanced Manufacturing, Shenzhen International Graduate School, Tsinghua University(先进制造系,深圳国际研究生院,清华大学)
Human-Exoskeleton Kinematic Calibration to Improve Hand Tracking for Dexterous Teleoperation
Walker Department of Mechanical Engineering, The University of Texas at Austin(德克萨斯大学机械工程系);Sony Group Corporation, Tokyo, Japan(索尼集团公司,日本东京);Meta Reality Labs Research, Redmond, WA, USA(Meta现实实验室研究)
DexCast: Learning Real-World Dexterous Control from Human Videos
Tsinghua University(清华大学);TsingTec(清华科技园(注:TsingTec为清华大学相关科技产业机构,此处保留英文原名));Fudan University(复旦大学);Shenzhen University(深圳大学);Nanyang Technological University(南洋理工大学);Georgia Institute of Technology(佐治亚理工学院)
TypeTele: Releasing Dexterity in Teleoperation by Dexterous Manipulation Types
School of Computer Science and Engineering, Sun Yat-sen University, China(中山大学计算机科学与工程学院);Stanford University, USA(斯坦福大学);Imperial College London, UK(伦敦帝国理工学院)
DexTele: A Dual-Arm Dexterous Teleoperation System Based on Motion Retargeting and Adaptive Force Control
School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen(人工智能学院,香港中文大学(深圳));School of Computing, University of Portsmouth(计算学院,朴茨茅斯大学)
UniBiDex: A Unified Teleoperation Framework for Robotic Bimanual Dexterous Manipulation
Department of Computer Science, The University of Hong Kong(香港大学计算机科学系);Dongguan Key Laboratory of Intelligent Equipment and Smart Industry, School of Advanced Engineering, The Great Bay University(东莞智能装备与智能制造重点实验室,高级工程学院,大湾区大学);Department of Mechanical Engineering, The Hong Kong Polytechnic University(香港理工大学机械工程系);Guangdong Academy of Sciences(广东省科学院)
DexTwist: Dexterous Hand Retargeting for Twist Motion via Mixed Reality-based Teleoperation
Autonomous Systems Lab, Institute of Computer Technology, TU Wien(自主系统实验室,计算机技术学院,维也纳技术大学);Institute of Robotics and Mechatronics (DLR), German Aerospace Center(机器人与机电系统研究所(DLR),德国航空航天中心)
TACROSS: An Efficient and Low-Cost Scalable Human Touch System Across Heterogeneous Tactile Sensors for Dexterous Robot Learning
The University of Hong Kong(香港大学);INFIFORCE;The Chinese University of Hong Kong (Shenzhen)(香港中文大学(深圳));Wuhan Textile University(武汉纺织大学);Ocean University of China(中国海洋大学);Wuhan University(武汉大学)
Towards Human-Like Manipulation through RL-Augmented Teleoperation and Mixture-of-Dexterous-Experts VLA
Shanghai Jiao Tong University(上海交通大学);Sharpa;Shanghai AI Lab(上海人工智能实验室);Zhongguancun Academy(中关村学院);National University of Singapore(新加坡国立大学);Shanghai Innovation Institute(上海创新研究院)