Studies in robot teleoperation have been centered around action specifications -- from continuous joint control to discrete end-effector pose control. However, these robot-centric interfaces often require skilled operators with extensive robotics expertise. To make teleoperation accessible to non-expert users, we propose the framework "Scene Editing as Teleoperation" (SEaT), where the key idea is to transform the traditional "robot-centric" interface into a "scene-centric" interface -- instead of controlling the robot, users focus on specifying the task's goal by manipulating digital twins of the real-world objects. As a result, a user can perform teleoperation without any expert knowledge of the robot hardware. To achieve this goal, we utilize a category-agnostic scene-completion algorithm that translates the real-world workspace (with unknown objects) into a manipulable virtual scene representation and an action-snapping algorithm that refines the user input before generating the robot's action plan. To train the algorithms, we procedurally generated a large-scale, diverse kit-assembly dataset that contains object-kit pairs that mimic real-world object-kitting tasks. Our experiments in simulation and on a real-world system demonstrate that our framework improves both the efficiency and success rate for 6DoF kit-assembly tasks. A user study demonstrates that SEaT framework participants achieve a higher task success rate and report a lower subjective workload compared to an alternative robot-centric interface. Video can be found at https://www.youtube.com/watch?v=-NdR3mkPbQQ .
Investigating the Benefits of Nonlinear Action Maps in Data-Driven Teleoperation
University of Alberta(阿尔伯塔大学);Universite de Montreal(蒙特利尔大学);University of Innsbruck(因斯布鲁克大学);Alberta Machine Intelligence Institute (Amii)(阿尔伯塔机器智能研究所(Amii))
Intelligent Mode-switching Framework for Teleoperation
James Watt School of Engineering, University of Glasgow(格拉斯哥大学詹姆斯·瓦特工程学院);School of Electrical and Information Engineering, The University of Sydney(悉尼大学电气与信息工程学院)
Multi-DoF Time Domain Passivity Approach Based Drift Compensation for Telemanipulation
Institute of Robotics and Mechatronics of the German Aerospace Center (DLR)(德国航空航天中心(DLR)机器人与机电一体化研究所);Department of Electrical Engineering, COPPE, Federal University of Rio de Janeiro (UFRJ)(里约热内卢联邦大学(UFRJ)COPPE电气工程系)
CDF-Glove: A Cable-Driven Force Feedback Glove for Dexterous Teleoperation
Center for Artificial Intelligence and Robotics, Shenzhen International Graduate School, Tsinghua University(人工智能与机器人中心,深圳国际研究生院,清华大学);PKU-Psibot Joint Lab(北京大学-PsiBot联合实验室);Department of Advanced Manufacturing, Shenzhen International Graduate School, Tsinghua University(先进制造系,深圳国际研究生院,清华大学)
TeleOpBench: A Simulator-Centric Benchmark for Dual-Arm Dexterous Teleoperation
Shanghai AI Laboratory(上海人工智能实验室);Zhejiang University(浙江大学);The Chinese University of Hong Kong(香港中文大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The University of Hong Kong(香港大学);Feeling AI(FeelAI);Tsinghua Shenzhen International Graduate School(清华大学深圳国际研究生院)
WetRobo: A Reproducible Robot Kit for Coding Agents in Biological Laboratories
The University of Tokyo(东京大学);RIKEN Pioneering Research Institute(日本理化学研究所先驱研究研究所);New York University(纽约大学);Google DeepMind(谷歌DeepMind);National Institute for Materials Science(日本国立材料科学研究所);RIKEN Center for Advanced Intelligence Project(日本理化学研究所先进智能项目中心)
A study on a Real-Time VR-Based Teleoperation Framework for Manipulator in Dynamic Environment
Department of Robotics, Hanyang University(韩世大学机器人系);Department of Smart Construction Engineering, Hanyang University(韩世大学智能建造工程系);Department of Interdisciplinary Robot Engineering Systems, Hanyang University(韩世大学跨学科机器人工程系统系);School of Smart Convergence Engineering, Hanyang University, Ansan(韩世大学智能融合工程学院,安山)
Stanford University(斯坦福大学);University of Montpellier(蒙彼利埃大学);CNRS(法国国家科学研究中心);Laboratory of Informatics, Robotics and Microelectronics of Montpellier (LIRMM)(蒙彼利埃信息学、机器人学与微电子学实验室(LIRMM))
Bounded haptic teleoperation of a quadruped robot's foot posture for sensing and manipulation
Institute of Perception, Action and Behaviour, School of Informatics, The University of Edinburgh(爱丁堡大学信息学院感知、行动与行为研究所);Oxford Robotics Institute at the University of Oxford(牛津大学牛津机器人研究所);School of Computer Science/Department of Electrical and Computer Engineering at McGill University(麦吉尔大学计算机科学学院/电气与计算机工程系)
Human-as-Humanoid: Enabling Zero-Shot Humanoid Learning from Ego-Exo Human Videos with Human-Aligned Embodiments
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));DeepCybo;ZGCA;ZGCI;Harbin Institute of Technology(哈尔滨工业大学);Huazhong University of Science and Technology(华中科技大学);Beihang University(北京航空航天大学)