Robotic systems that rely primarily on self-supervised learning have the potential to decrease the amount of human annotation and engineering effort required to learn control strategies. In the same way that prior robotic systems have leveraged self-supervised techniques from computer vision (CV) and natural language processing (NLP), our work builds on prior work showing that the reinforcement learning (RL) itself can be cast as a self-supervised problem: learning to reach any goal without human-specified rewards or labels. Despite the seeming appeal, little (if any) prior work has demonstrated how self-supervised RL methods can be practically deployed on robotic systems. By first studying a challenging simulated version of this task, we discover design decisions about architectures and hyperparameters that increase the success rate by $2 \times$. These findings lay the groundwork for our main result: we demonstrate that a self-supervised RL algorithm based on contrastive learning can solve real-world, image-based robotic manipulation tasks, with tasks being specified by a single goal image provided after training.
Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RL
Tsinghua University（清华大学）；University of Cambridge（剑桥大学）；Eindhoven University of Technology（埃因霍温理工大学）；King’s College London（伦敦国王学院）；Tencent Robotics X（腾讯机器人实验室）
Compositional Transduction with Latent Analogies for Offline Goal-Conditioned Reinforcement Learning
Department of Electrical and Computer Engineering and ASRI, Seoul National University（电气与计算机工程系和首尔国立大学ASRI）；Independent researcher（独立研究者）；Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）
Department of Computer Science, Purdue University, USA（普渡大学计算机科学系）；College of Engineering, Purdue University, USA（普渡大学工程学院）；DEVCOM Army Research Laboratory, USA（美国国防部 DEVCOM 军事研究实验室）
针对离线目标条件强化学习中值函数估计困难的问题，提出一种通过空间测度聚合约束（而非逐点微分约束）来诱导距离类值几何的方法，称为Mollified Value Learning（MVL），在导航和高维机器人操作任务中提升了目标达成性能。
QHyer: Q-conditioned Hybrid Attention-mamba Transformer for Offline Goal-conditioned RL
National Key Laboratory of Human-Machine Hybrid Augmented Intelligence（人机混合增强智能国家重点实验室）；Institute of Artificial Intelligence（人工智能研究院）；Robotics, Xi'an Jiaotong University（机器人技术，西安交通大学）；Computer Science, University College London（计算机科学，伦敦大学学院）；School of Engineering, Westlake University（工程学院，西湖大学）
Option-aware Temporally Abstracted Value for Offline Goal-Conditioned Reinforcement Learning
Department of Electrical and Computer Engineering (ECE), Seoul National University（电子与计算机工程系，首尔国立大学）；Interdisciplinary Program in Artificial Intelligence (IPAI), Seoul National University（人工智能跨学科项目，首尔国立大学）；ASRI / INMC, Seoul National University（ASRI/INMC，首尔国立大学）
PRTS: A Primitive Reasoning and Tasking System via Contrastive Representations
Institute of Artificial Intelligence (TeleAI), China Telecom（人工智能研究院（TeleAI），中国电信）；Tsinghua University（清华大学）；Shanghai Jiao Tong University（上海交通大学）；Fudan University（复旦大学）
