Human video datasets used for cotraining robot manipulation policies largely consist of curated demonstrations where motions are orchestrated to resemble robot behavior and 3D hand poses are captured with specialized hardware. A more plentiful source of data is everyday Internet video, but it is an open question what factors enable transfer from such videos to robots. We investigate this using a new dataset of 532 human videos with 28 hours of high-quality triangulated hand labels and natural motions. We find that hand pose quality affects transfer, but even with accurate hands, the inherent motion gap hinders transfer unless the vision and policy networks specialize to each embodiment. Our cotraining recipe yields consistent improvements, with an absolute success rate gain of $29.7\%$ in the low-robot-data regime across six manipulation tasks.
MotionTrans: Human VR Data Enable Motion-Level Learning for Robotic Manipulation Policies
Institute for Interdisciplinary Information Sciences（交叉信息学院）；Shanghai Qi Zhi Institute（上海启智研究所）；State Key Laboratory of Multimedia Information Processing School of Computer Science（多媒体信息处理国家重点实验室计算机科学学院）；Shanghai Jiao Tong University（上海交通大学）；Wuhan University（武汉大学）
Risk-Aware Reinforcement Learning for Mobile Manipulation
Oxford Robotics Institute, Department of Engineering Science, University of Oxford（牛津大学机器人研究所、工程科学系、牛津大学）；Dyson Institute of Engineering & Technology（戴森工程与技术研究所）；Bristol Robotics Laboratory, School of Engineering, University of West England（布里斯托尔机器人实验室、工程学院、西英格兰大学）
MolmoSpaces: A Large-Scale Open Ecosystem for Robot Navigation and Manipulation
Allen Institute for AI（艾伦人工智能研究所）；University of Washington（华盛顿大学）；University of California, Los Angeles（加州大学洛杉矶分校）；University of California, Berkeley（加州大学伯克利分校）
The effects of traffic congestion are widespread and are an impedance to everyday life. Piecewise constant driving policies have shown promise in helping mitigate traffic congestion in simulation environments. However, no works currently test these policies in situations involving real human users. Thus, we propose to evaluate these policies through the use of a shared control framework in a collaborative experiment with the human driver and the driving policy aiming to co-operatively mitigate congestion. We intend to use the CARLA simulator alongside the Flow framework to conduct user studies…
Disentangling perception and reasoning for improving data efficiency in learning cloth manipulation without demonstrations
Advanced Robotics, Istituto Italiano di Tecnologia (IIT)（先进机器人技术研究所（IIT））；epartment of Mechanical and Automation Engineering, T-Stone Robotics Institute, The Chinese University of Hong Kong（机械与自动化工程系，腾讯机器人研究院，香港中文大学）；University of Genova（热那亚大学）
Placing by Touching: An empirical study on the importance of tactile sensing for precise object placing
Institut de Robòtica i Informàtica Industrial, CSIC-UPC（西班牙国家研究委员会-加泰罗尼亚理工大学工业机器人与信息技术研究所）；Bielefeld University（比勒费尔德大学）；PAL Robotics（PAL机器人公司）；Technical University Darmstadt（达姆施塔特工业大学）；German Research Center for AI (DFKI)（德国人工智能研究中心）；Centre for Cognitive Science（认知科学中心）
Robust Instant Policy: Leveraging Student's t-Regression Model for Robust In-context Imitation Learning of Robot Manipulation
Industrial Cyber-Physical Systems Research Center at the National Institute of Advanced Industrial Science and Technology (AIST), Japan（日本国家先进工业科学技术研究院工业闭环系统研究中心）
Autonomously navigating a robot in everyday crowded spaces requires solving complex perception and planning challenges. When using only monocular image sensor data as input, classical two-dimensional planning approaches cannot be used. While images present a significant challenge when it comes to perception and planning, they also allow capturing potentially important details, such as complex geometry, body movement, and other visual cues. In order to successfully solve the navigation task from only images, algorithms must be able to model the scene and its dynamics using only this channel of …
When an autonomous robot learns how to execute actions, it is of interest to know if and when the execution policy can be generalised to variations of the learning scenarios. This can inform the robot about the necessity of additional learning, as using incomplete or unsuitable policies can lead to execution failures. Generalisation is particularly relevant when a robot has to deal with a large variety of objects and in different contexts. In this paper, we propose and analyse a strategy for generalising parameterised execution models of manipulation actions over different objects based on an …
Towards a General Humanoid Loco-Manipulation Model via Egocentric Whole-Body Human Data Pretraining
Alibaba Group（阿里巴巴集团）；Sichuan University（四川大学）；Shanghai Innovation Institute（上海创新研究院）；Beihang University（北京航空航天大学）；Hong Kong University of Science and Technology（香港科技大学）
FastUMI: A Scalable and Hardware-Independent Universal Manipulation Interface with Dataset
Shanghai AI Lab（上海人工智能实验室）；Shanghai Jiao Tong University（上海交通大学）；University of Bristol（布里斯托大学）；Fudan University（复旦大学）；The University of Hong Kong（香港大学）；Xi’an Jiaotong-Liverpool University（西交利物浦大学）；Institute of AI, China Telecom Corp Ltd（中国电信股份有限公司人工智能研究院）
