Providing explanations of chosen robotic actions can help to increase the transparency of robotic planning and improve users' trust. Social sciences suggest that the best explanations are contrastive, explaining not just why one action is taken, but why one action is taken instead of another. We formalize the notion of contrastive explanations for robotic planning policies based on Markov decision processes, drawing on insights from the social sciences. We present methods for the automated generation of contrastive explanations with three key factors: selectiveness, constrictiveness, and responsibility. The results of a user study with 100 participants on the Amazon Mechanical Turk platform show that our generated contrastive explanations can help to increase users' understanding and trust of robotic planning policies while reducing users' cognitive burden.
As safety violations can lead to severe consequences in real-world robotic applications, the increasing deployment of Reinforcement Learning (RL) in robotic domains has propelled the study of safe exploration for reinforcement learning (safe RL). In this work, we propose a risk preventive training method for safe RL, which learns a statistical contrastive classifier to predict the probability of a state-action pair leading to unsafe states. Based on the predicted risk probabilities, we can collect risk preventive trajectories and reshape the reward function with risk penalties to induce safe R…
King’s College London（伦敦国王学院）；Ulm University（乌尔姆大学）
The development of robotics and AI agents has enabled their wider usage in human surroundings. AI agents are more trusted to make increasingly important decisions with potentially critical outcomes. It is essential to consider the ethical consequences of the decisions made by these systems. In this paper, we present how contrastive explanations can be used for comparing the ethics of plans. We build upon an existing ethical framework to allow users to make suggestions to plans and receive contrastive explanations.
Electrical and Computer Engineering, The University of Auckland（奥克兰大学电气与计算机工程学院）；School of Computer Science, University of Birmingham（伯明翰大学计算机学院）
A robot's ability to provide descriptions of its decisions and beliefs promotes effective collaboration with humans. Providing such transparency is particularly challenging in integrated robot systems that include knowledge-based reasoning methods and data-driven learning algorithms. Towards addressing this challenge, our architecture couples the complementary strengths of non-monotonic logical reasoning, deep learning, and decision-tree induction. During reasoning and learning, the architecture enables a robot to provide on-demand relational descriptions of its decisions, beliefs, and the out…
New York University（纽约大学）；University of California, Berkeley（加州大学伯克利分校）
Recent advances in unsupervised representation learning significantly improved the sample efficiency of training Reinforcement Learning policies in simulated environments. However, similar gains have not yet been seen for real-robot reinforcement learning. In this work, we focus on enabling data-efficient real-robot learning from pixels. We present Contrastive Pre-training and Data Augmentation for Efficient Robotic Learning (CoDER), a method that utilizes data augmentation and unsupervised learning to achieve sample-efficient training of real-robot arm policies from sparse rewards. While cont…
Modeling the world can benefit robot learning by providing a rich training signal for shaping an agent's latent state space. However, learning world models in unconstrained environments over high-dimensional observation spaces such as images is challenging. One source of difficulty is the presence of irrelevant but hard-to-model background distractions, and unimportant visual details of task-relevant entities. We address this issue by learning a recurrent latent dynamics model which contrastively predicts the next observation. This simple model leads to surprisingly robust robotic control even…
Carnegie Mellon University（卡内基梅隆大学）；University of Notre Dame（圣母大学）
Liquid state estimation is important for robotics tasks such as pouring; however, estimating the state of transparent liquids is a challenging problem. We propose a novel segmentation pipeline that can segment transparent liquids such as water from a static, RGB image without requiring any manual annotations or heating of the liquid for training. Instead, we use a generative model that is capable of translating images of colored liquids into synthetically generated transparent liquid images, trained only on an unpaired dataset of colored and transparent liquid images. Segmentation labels of co…
ClearDepth: Enhanced Stereo Perception of Transparent Objects for Robotic Manipulation
TAMS (Technical Aspects of Multimodal Systems), Department of Informatics, University of Hamburg（汉堡大学信息学院）；Agile Robots SE（敏捷机器人公司）；Technical University of Munich（慕尼黑技术大学）；Mohamed Bin Zayed University of Artificial Intelligence (MBUZAI)（Mohamed Bin Zayed人工智能大学）
National Key Laboratory for Parallel and Distributed Processing, National University of Defense Technology（国防科技大学并行与分布处理国家重点实验室）
Many robotic tasks require heavy computation, which can easily exceed the robot's onboard computer capability. A promising solution to address this challenge is outsourcing the computation to the cloud. However, exploiting the potential of cloud resources in robotic software is difficult, because it involves complex code modification and extensive (re)configuration procedures. Moreover, quality of service (QoS) such as timeliness, which is critical to robot's behavior, have to be considered. In this paper, we propose a transparent and QoS-aware software framework called Cloudroid for cloud rob…
Trans2Occ: Voxel Occupancy Estimation and Grasp for Transparent Objects from Simulation to Reality
Shanghai AI Laboratory（上海人工智能实验室）；SUSTech（南方科技大学）；CUHK（香港中文大学）；Harbin Institute of Technology（哈尔滨工业大学）；University of Oxford（牛津大学）；Beihang University（北京航空航天大学）；Nanyang Technological University（南洋理工大学）；University of Science and Technology of China（中国科学技术大学）
CLAR: Learning 3D Representations for Robotic Manipulation by Fusing Masked Reconstruction with Multi-Level Contrastive Alignment
SKL-MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所SKL-MAIS）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Carnegie Mellon University（卡内基梅隆大学）；Galbot；CFCS, School of Computer Science, Peking University（北京大学计算机科学与技术学院CFCS）
Contrastive Imitation Learning for Language-guided Multi-Task Robotic Manipulation
AI Thrust, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）AI Thrust）；The Hong Kong University of Science and Technology（香港科技大学）
Transparent objects are prevalent across many environments of interest for dexterous robotic manipulation. Such transparent material leads to considerable uncertainty for robot perception and manipulation, and remains an open challenge for robotics. This problem is exacerbated when multiple transparent objects cluster into piles of clutter. In household environments, for example, it is common to encounter piles of glassware in kitchens, dining rooms, and reception areas, which are essentially invisible to modern robots. We present the GlassLoc algorithm for grasp pose detection of transparent …
ConLA: Contrastive Latent Action Learning from Human Videos for Robotic Manipulation
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；State Key Laboratory of Mobile Network and Mobile Multimedia Technology, Shenzhen（移动网络与移动多媒体技术国家重点实验室（深圳））；ZTE Corporation（中兴通讯）；Shanghai Jiao Tong University（上海交通大学）；Central South University（中南大学）
A Contrastive Fewshot RGBD Traversability Segmentation Framework for Indoor Robotic Navigation
Department of Computer Science and Engineering, University of Texas at Arlington（计算机科学与工程系，德克萨斯大学阿灵顿分校）；Uber（优步）；Cognitive Robotics Lab, Department of Electrical Engineering and Computer Science, University of Arkansas（认知机器人实验室，电气工程与计算机科学系，阿肯色大学）
Robotic-CLIP: Fine-tuning CLIP on Action Data for Robotic Applications
FPT Software AI Center（FPT软件人工智能中心）；Automation & Control Institute (ACIN), TU Wien（维也纳工业大学自动化与控制研究所）；Center for Vision, Automation & Control, AIT Austrian Institute of Technology (GmbH)（奥地利技术研究所（GmbH）视觉、自动化与控制中心）；The University of Tokyo（东京大学）；University of Liverpool（利物浦大学）；National University of Singapore（新加坡国立大学）；University of Arkansas（阿肯色大学）
We propose a self-supervised approach for learning representations of objects from monocular videos and demonstrate it is particularly useful in situated settings such as robotics. The main contributions of this paper are: 1) a self-supervising objective trained with contrastive learning that can discover and disentangle object attributes from video without using any labels; 2) we leverage object self-supervision for online adaptation: the longer our online model looks at objects in a video, the lower the object identification error, while the offline baseline remains with a large fixed error;…
Surgical-VQLA++: Adversarial Contrastive Learning for Calibrated Robust Visual Question-Localized Answering in Robotic Surgery
The Chinese University of Hong Kong（香港中文大学）；Shenzhen Research Institute, The Chinese University of Hong Kong（香港中文大学深圳研究院）；University College London（伦敦大学学院）；National University of Singapore（新加坡国立大学）
Robots Pre-train Robots: Manipulation-Centric Robotic Representation from Large-Scale Robot Datasets
University of California, San Diego（加利福尼亚大学圣迭戈分校）；Tongji University（同济大学）；Shanghai Jiao Tong University（上海交通大学）；University of Maryland, College Park（马里兰大学帕克分校）；Tsinghua University（清华大学）
SR3D: Unleashing Single-view 3D Reconstruction for Transparent and Specular Object Grasping
Beijing University of Posts and Telecommunications（北京邮电大学）；Center on Frontiers of Computing Studies, Peking University（北京大学前沿计算研究中心）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
University of Naples Federico II（那不勒斯费德里科二世大学）；Istituto Italiano di Tecnologia（意大利技术研究院）
The high request for autonomous and flexible HRI implies the necessity of deploying Machine Learning (ML) mechanisms in the robot control. Indeed, the use of ML techniques, such as Reinforcement Learning (RL), makes the robot behaviour, during the learning process, not transparent to the observing user. In this work, we proposed an emotional model to improve the transparency in RL tasks for human-robot collaborative scenarios. The architecture we propose supports the RL algorithm with an emotional model able to both receive human feedback and exhibit emotional responses based on the learning p…
Politecnico di Torino（都灵理工大学）；PIC4SeR (PoliTo Interdepartmental Centre for Service Robotics)（PIC4SeR（都灵理工大学服务机器人跨部门中心））
The development of precision agriculture has gradually introduced automation in the agricultural process to support and rationalize all the activities related to field management. In particular, service robotics plays a predominant role in this evolution by deploying autonomous agents able to navigate in fields while executing different tasks without the need for human intervention, such as monitoring, spraying and harvesting. In this context, global path planning is the first necessary step for every robotic mission and ensures that the navigation is performed efficiently and with complete fi…
Safety Aware Task Planning via Large Language Models in Robotics
University of Minnesota – Twin Cities（明尼苏达大学双城分校）；Institute for Robotics and Intelligent Machines (IRIM), Georgia Institute of Technology（佐治亚理工学院机器人与智能机器研究所）；Pontificia Universidad Católica de Chile（智利天主教大学）
High-Fidelity One-Step Generative Visuomotor Policy via Recursive Correction, Frequency Consistency, and Contrastive Flow Matching
School of Safety Science and Engineering, Anhui University of Science and Technology（安徽理工大学安全科学与工程学院）；State Key Laboratory of Digital Intelligent Technology for Unmanned Coal Mining, Anhui University of Science and Technology（安徽理工大学煤矿智能开采技术与装备国家重点实验室）；School of Artificial Intelligence, Anhui University of Science and Technology（安徽理工大学人工智能学院）
Google Brain（谷歌大脑）；University of Southern California（南加州大学）
We propose a self-supervised approach for learning representations and robotic behaviors entirely from unlabeled videos recorded from multiple viewpoints, and study how this representation can be used in two robotic imitation settings: imitating object interactions from videos of humans, and imitating human poses. Imitation of human behavior requires a viewpoint-invariant representation that captures the relationships between end-effectors (hands or robot grippers) and the environment, object attributes, and body pose. We train our representations using a metric learning loss, where multiple s…
AISPO: Enhancing Depth Reliability for Robotic Manipulation of Non-Lambertian Objects via Affine-Invariant Shape Prior
The Hong Kong University of Science & Technology（香港科技大学）；The University of Hong Kong（香港大学）；Southern University of Science & Technology（南方科技大学）；Zhejiang University（浙江大学）；LimX Dynamics
PRTS: A Primitive Reasoning and Tasking System via Contrastive Representations
Institute of Artificial Intelligence (TeleAI), China Telecom（人工智能研究院（TeleAI），中国电信）；Tsinghua University（清华大学）；Shanghai Jiao Tong University（上海交通大学）；Fudan University（复旦大学）
Norwegian University of Science and Technology（挪威科技大学）；University of Melbourne（墨尔本大学）
Although many machine learning methods, especially from the field of deep learning, have been instrumental in addressing challenges within robotic applications, we cannot take full advantage of such methods before these can provide performance and safety guarantees. The lack of trust that impedes the use of these methods mainly stems from a lack of human understanding of what exactly machine learning models have learned, and how robust their behaviour is. This is the problem the field of explainable artificial intelligence aims to solve. Based on insights from the social sciences, we know that…
University of Torontoland（多伦托兰大学）；Googol ShallowMind（古戈尔浅智公司）；University of Edenborrow（伊登伯罗大学）
Learning state representations enables robotic planning directly from raw observations such as images. Most methods learn state representations by utilizing losses based on the reconstruction of the raw observations from a lower-dimensional latent space. The similarity between observations in the space of images is often assumed and used as a proxy for estimating similarity between the underlying states of the system. However, observations commonly contain task-irrelevant factors of variation which are nonetheless important for reconstruction, such as varying lighting and different camera view…
CLTP: Contrastive Language-Tactile Pre-training for 3D Contact Geometry Understanding
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Beihang University（北航）；Beijing University of Posts and Telecommunications（北京邮电大学）；Samsung Research China（三星中国研究院）
The Hong Kong Polytechnic University（香港理工大学）；University of York（约克大学）；Shanghai Jiatong University（上海交通大学）
Most object manipulation strategies for robots are based on the assumption that the object is rigid (i.e., with fixed geometry) and the goal's details have been fully specified (e.g., the exact target pose). However, there are many tasks that involve spatial relations in human environments where these conditions may be hard to satisfy, e.g., bending and placing a cable inside an unknown container. To develop advanced robotic manipulation capabilities in unstructured environments that avoid these assumptions, we propose a novel long-horizon framework that exploits contrastive planning in findin…
The Chinese University of Hong Kong（香港中文大学）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
Deep visuomotor policy learning, which aims to map raw visual observation to action, achieves promising results in control tasks such as robotic manipulation and autonomous driving. However, it requires a huge number of online interactions with the training environment, which limits its real-world application. Compared to the popular unsupervised feature learning for visual recognition, feature pretraining for visuomotor control tasks is much less explored. In this work, we aim to pretrain policy representations for driving tasks by watching hours-long uncurated YouTube videos. Specifically, w…
