This paper presents a novel method for learning reward functions for robotic motions by harnessing the power of a CLIP-based model. Traditional reward function design often hinges on manual feature engineering, which can struggle to generalize across an array of tasks. Our approach circumvents this challenge by capitalizing on CLIP's capability to process both state features and image inputs effectively. Given a pair of consecutive observations, our model excels in identifying the motion executed between them. We showcase results spanning various robotic activities, such as directing a gripper to a designated target and adjusting the position of a cube. Through experimental evaluations, we underline the proficiency of our method in precisely deducing motion and its promise to enhance reinforcement learning training in the realm of robotics.
Princeton University（普林斯顿大学）；University of Pennsylvania（宾夕法尼亚大学）
Reinforcement learning (RL) is a general and well-known method that a robot can use to learn an optimal control policy to solve a particular task. We would like to build a versatile robot that can learn multiple tasks, but using RL for each of them would be prohibitively expensive in terms of both time and wear-and-tear on the robot. To remedy this problem, we use the Policy Gradient Efficient Lifelong Learning Algorithm (PG-ELLA), an online multi-task RL algorithm that enables the robot to efficiently learn multiple consecutive tasks by sharing knowledge between these tasks to accelerate lear…
Robotic-CLIP: Fine-tuning CLIP on Action Data for Robotic Applications
FPT Software AI Center（FPT软件人工智能中心）；Automation & Control Institute (ACIN), TU Wien（维也纳工业大学自动化与控制研究所）；Center for Vision, Automation & Control, AIT Austrian Institute of Technology (GmbH)（奥地利技术研究所（GmbH）视觉、自动化与控制中心）；The University of Tokyo（东京大学）；University of Liverpool（利物浦大学）；National University of Singapore（新加坡国立大学）；University of Arkansas（阿肯色大学）
We propose a general and model-free approach for Reinforcement Learning (RL) on real robotics with sparse rewards. We build upon the Deep Deterministic Policy Gradient (DDPG) algorithm to use demonstrations. Both demonstrations and actual interactions are used to fill a replay buffer and the sampling ratio between demonstrations and transitions is automatically tuned via a prioritized replay mechanism. Typically, carefully engineered shaping rewards are required to enable the agents to efficiently explore on high dimensional control problems such as robotics. They are also required for model-b…
Sun Yat-sen University（中山大学）；Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；Dark Matter AI Inc.（暗物质人工智能公司）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
Although deep reinforcement learning (RL) has been successfully applied to a variety of robotic control tasks, it's still challenging to apply it to real-world tasks, due to the poor sample efficiency. Attempting to overcome this shortcoming, several works focus on reusing the collected trajectory data during the training by decomposing them into a set of policy-irrelevant discrete transitions. However, their improvements are somewhat marginal since i) the amount of the transitions is usually small, and ii) the value assignment only happens in the joint states. To address these issues, this pa…
Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Model
Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）；University of Chinese Academy of Sciences（中国科学院大学）；The Chinese University of Hong Kong（香港中文大学）
Point Cloud Segmentation for Autonomous Clip Positioning in Laparoscopic Cholecystectomy on a Phantom
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；HIDSS4Health - Helmholtz Information and Data Science School for Health（亥姆霍兹信息与数据科学健康学校）；Friedrich-Alexander-University, Erlangen-Nuremberg（弗里德里希-亚历山大大学埃尔朗根-纽伦堡）；University Hospital Carl Gustav Carus and Centre for Tactile Internet with Human-in-the-loop (CeTI), Dresden University of Technology（卡尔·古斯塔夫·卡鲁斯大学医院及德累斯顿工业大学触觉互联网人机共融卓越中心）
Google（谷歌）；UC Berkeley（加州大学伯克利分校）；University of Warsaw（华沙大学）
Goal-conditioned policies for robotic navigation can be trained on large, unannotated datasets, providing for good generalization to real-world settings. However, particularly in vision-based settings where specifying goals requires an image, this makes for an unnatural interface. Language provides a more convenient modality for communication with robots, but contemporary methods typically require expensive supervision, in the form of trajectories annotated with language descriptions. We present a system, LM-Nav, for robotic navigation that enjoys the benefits of training on unannotated large …
Shanghai Jiao Tong University（上海交通大学）；University of Victoria（维多利亚大学）
The interaction topology is critical for efficient cooperation of mobile robotic networks (MRNs). We focus on the local topology inference problem of MRNs under formation control, where an inference robot with limited observation range can manoeuvre among the formation robots. This problem faces new challenges brought by the highly coupled influence of unobservable formation robots, inaccessible formation inputs, and unknown interaction range. The novel idea here is to advocate a range-shrink strategy to perfectly avoid the influence of unobservable robots while filtering the input. To that en…
AtomBridge: Agentic VLA Inference Plugin for Long-Horizon Tasks in Scientific Experiments
School of Computer Science and Engineering & School of Software Engineering & School of Artificial Intelligence, Southeast University, Nanjing, China（计算机科学与工程学院、软件工程学院、人工智能学院，东南大学，南京，中国）；Pattern Learning and Mining Lab, Southeast University, Nanjing, China（模式学习与挖掘实验室，东南大学，南京，中国）
3D Feature Distillation with Object-Centric Priors
Department of Artificial Intelligence University of Groningen, the Neteherlands（格罗宁根大学人工智能系）；School of Informatics University of Edinburgh, United Kingdom（爱丁堡大学信息学院）；Department of Computer Science University College London, United Kingdom（伦敦大学学院计算机科学系）
Technical University of Vienna（维也纳技术大学）；AIT Austrian Institute of Technology（奥地利技术研究院）
Motion and dynamic environments, especially under challenging lighting conditions, are still an open issue for robust robotic applications. In this paper, we propose an end-to-end pipeline for real-time, low latency, 6 degrees-of-freedom pose estimation of fiducial markers. Instead of achieving a pose estimation through a conventional frame-based approach, we employ the high-speed abilities of event-based sensors to directly refine the spatial transformation, using consecutive events. Furthermore, we introduce a novel two-way verification process for detecting tracking errors by backtracking t…
Adapt2Reward: Adapting Video-Language Models to Generalizable Robotic Rewards via Failure Prompts
School of Software Technology, Zhejiang University（浙江大学软件学院）；School of Computer Sciene and Technology, Hangzhou Dianzi University（杭州电子科技大学计算机学院）；China Mobile (Zhejiang) Research & Innovation Institute（中国移动（浙江）研究院）；State Key Lab of CAD&CG, Zhejiang University（浙江大学CAD&CG国家重点实验室）；The Hong Kong Polytechnic University（香港理工大学）；Zhiyuan Research Institute（智源研究院）；School of Computer Sciene and Technology, Xidian University（西安电子科技大学计算机学院）
NUS Graduate School for Integrative Sciences and Engineering (NGS), National University of Singapore（新加坡国立大学综合科学与工程研究生院）；Dept. of Biomedical Engineering, National University of Singapore（新加坡国立大学生物医学工程系）；Dept. of Instrumentation and Control Engineering, NIT Trichy（印度国立理工学院蒂鲁奇拉帕利分校仪器与控制工程系）；Dept. of Otolaryngology, Singapore General Hospital（新加坡中央医院耳鼻喉科）
Representation learning of the task-oriented attention while tracking instrument holds vast potential in image-guided robotic surgery. Incorporating cognitive ability to automate the camera control enables the surgeon to concentrate more on dealing with surgical instruments. The objective is to reduce the operation time and facilitate the surgery for both surgeons and patients. We propose an end-to-end trainable Spatio-Temporal Multi-Task Learning (ST-MTL) model with a shared encoder and spatio-temporal decoders for the real-time surgical instrument segmentation and task-oriented saliency dete…
Technical University of Munich（慕尼黑工业大学）；Shanghai Jiao Tong University（上海交通大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Robotic ultrasound (US) imaging aims at overcoming some of the limitations of free-hand US examinations, e.g. difficulty in guaranteeing intra- and inter-operator repeatability. However, due to anatomical and physiological variations between patients and relative movement of anatomical substructures, it is challenging to robustly generate optimal trajectories to examine the anatomies of interest, in particular, when they comprise articulated joints. To address this challenge, this paper proposes a vision-based approach allowing autonomous robotic US limb scanning. To this end, an atlas MRI tem…
Hierarchical Pose Estimation and Mapping with Multi-Scale Neural Feature Fields
University of Bonn（波恩大学）；Autonomous Intelligent Systems（自主智能系统研究所）；Computer Science Institute VI（计算机科学第六研究所）；Center for Robotics（机器人中心）；Lamarr Institute for Machine Learning and Artificial Intelligence（拉马尔机器学习与人工智能研究所）
