Current recipes for robot learning require massive amounts of robot demonstrations to obtain robust and adaptive policies. In this work, we introduce Steertuning, a system that enables robots to learn new tasks in new environments from 5 human demonstrations in under 5
Steertuning consists of two parts. (1) We pretrain a 400M-parameter diffusion policy on our egocentric human dataset, which consists of 100 hours of densely annotated random interactions. This pretraining makes Steertuning robust to background differences and enables spatial and object generalization when deployed on new tasks. (2) Given 5 demonstrations of a new task, we finetune our model for only 2 minutes (2000 training updates) and steer the policy's actions during reverse sampling using the demonstration whose observation is closest to that of the deployment environment. Together with a carefully designed data processing pipeline for embodiment transfer, this enables Steertuning to achieve a 71% success rate across 12 tasks in 12 new environments, ranging from lab settings to kitchens in new homes. The entire process, from data collection for a new task to robot policy deployment, takes less than 5 minutes. Our full dataset, model weights, codebase, and robot videos will be made available at our website: https://steertuning.github.io.
LAPT-Scale: Scaling Embodied Policies with Latent Action Pre-Training on Human Videos
Joy Future Academy, JD（京东探索研究院）；School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）计算机科学与技术学院）；Tsinghua SIGS, Tsinghua University（清华大学深圳国际研究生院）；Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）人工智能学院）
LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies
Tsinghua University（清华大学）；Jilin University（吉林大学）；Nankai University（南开大学）；Peking University（北京大学）；Harbin Institute of Technology（哈尔滨工业大学）；Zhongguancun Academy（中关村学院）；Striding.AI；Infinigence AI
Carnegie Mellon University（卡内基梅隆大学）；Facebook AI Research（Facebook人工智能研究）
Numerous past works have tackled the problem of task-driven navigation. But, how to effectively explore a new environment to enable a variety of down-stream tasks has received much less attention. In this work, we study how agents can autonomously explore realistic and complex 3D environments without the context of task-rewards. We propose a learning-based approach and investigate different policy architectures, reward functions, and training paradigms. We find that the use of policies with spatial memory that are bootstrapped with imitation learning and finally finetuned with coverage rewards…
Learning complex manipulation tasks in realistic, obstructed environments is a challenging problem due to hard exploration in the presence of obstacles and high-dimensional visual observations. Prior work tackles the exploration problem by integrating motion planning and reinforcement learning. However, the motion planner augmented policy requires access to state information, which is often not available in the real-world settings. To this end, we propose to distill a state-based motion planner augmented policy to a visual control policy via (1) visual behavioral cloning to remove the motion p…
We present a novel sensor-based learning navigation algorithm to compute a collision-free trajectory for a robot in dense and dynamic environments with moving obstacles or targets. Our approach uses deep reinforcement learning-based expert policy that is trained using a sim2real paradigm. In order to increase the reliability and handle the failure cases of the expert policy, we combine with a policy extraction technique to transform the resulting policy into a decision tree format. The resulting decision tree has properties which we use to analyze and modify the policy and improve performance …
The current dominant paradigm in sensorimotor control, whether imitation or reinforcement learning, is to train policies directly in raw action spaces such as torque, joint angle, or end-effector position. This forces the agent to make decisions individually at each timestep in training, and hence, limits the scalability to continuous, high-dimensional, and long-horizon tasks. In contrast, research in classical robotics has, for a long time, exploited dynamical systems as a policy representation to learn robot behaviors via demonstrations. These techniques, however, lack the flexibility and ge…
Model-free deep reinforcement learning has been shown to exhibit good performance in domains ranging from video games to simulated robotic manipulation and locomotion. However, model-free methods are known to perform poorly when the interaction time with the environment is limited, as is the case for most real-world robotic tasks. In this paper, we study how maximum entropy policies trained using soft Q-learning can be applied to real-world robotic manipulation. The application of this method to real-world manipulation is facilitated by two important features of soft Q-learning. First, soft Q-…
Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）；Google DeepMind（谷歌DeepMind）
We present a method for fast training of vision based control policies on real robots. The key idea behind our method is to perform multi-task Reinforcement Learning with auxiliary tasks that differ not only in the reward to be optimized but also in the state-space in which they operate. In particular, we allow auxiliary task policies to utilize task features that are available only at training-time. This allows for fast learning of auxiliary policies, which subsequently generate good data for training the main, vision-based control policies. This method can be seen as an extension of the Sche…
AUTOLAB at the University of California, Berkeley（加利福尼亚大学伯克利分校AUTOLAB）；Toyota Research Institute（丰田研究院）
Robotic manipulation of deformable 1D objects such as ropes, cables, and hoses is challenging due to the lack of high-fidelity analytic models and large configuration spaces. Furthermore, learning end-to-end manipulation policies directly from images and physical interaction requires significant time on a robot and can fail to generalize across tasks. We address these challenges using interpretable deep visual representations for rope, extending recent work on dense object descriptors for robot manipulation. This facilitates the design of interpretable and transferable geometric policies built…
ARDuP: Active Region Video Diffusion for Universal Policies
University of Maryland, College Park（马里兰大学帕克分校）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；California Institute of Technology（加州理工学院）；NVIDIA Corporation（英伟达公司）
Cosmos-H-Surgical: Learning Surgical Robot Policies from Videos via World Modeling
NVIDIA；The Chinese University of Hong Kong（香港中文大学）；Sung Kyun Kwan University（全州大学）；Wenzhou Medical University（温州医学院）；National University of Singapore（新加坡国立大学）；Ruijin Hospital（瑞金医院）
SC3-Eval: Evaluating Robot Foundation Models via Self-Consistent Video Generation
University of Toronto（多伦多大学）；Vector Institute（向量研究所）；NVIDIA（英伟达）；Physical Intelligence；Stanford University（斯坦福大学）；UC Berkeley（加州大学伯克利分校）；Allen Institute for AI（艾伦人工智能研究所）
