Debiased Offline Representation Learning for Fast Online Adaptation in Non-stationary Dynamics
National Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室);School of Artificial Intelligence, Nanjing University(南京大学人工智能学院);Polixir Technologies(Polixir科技公司)
Xinyu Zhang, Wenjie Qiu, Yi-Chen Li, Lei Yuan, Chengxing Jia, Zongzhang Zhang, Yang Yu
Developing policies that can adjust to non-stationary environments is essential for real-world reinforcement learning applications. However, learning such adaptable policies in offline settings, with only a limited set of pre-collected trajectories, presents significant challenges. A key difficulty arises because the limited offline data makes it hard for the context encoder to differentiate between changes in the environment dynamics and shifts in the behavior policy, often leading to context misassociations. To address this issue, we introduce a novel approach called Debiased Offline Representation for fast online Adaptation (DORA). DORA incorporates an information bottleneck principle that maximizes mutual information between the dynamics encoding and the environmental data, while minimizing mutual information between the dynamics encoding and the actions of the behavior policy. We present a practical implementation of DORA, leveraging tractable bounds of the information bottleneck principle. Our experimental evaluation across six benchmark MuJoCo tasks with variable parameters demonstrates that DORA not only achieves a more precise dynamics encoding but also significantly outperforms existing baselines in terms of performance.
Prototypical Distillation and Debiased Tuning for Black-box Unsupervised Domain Adaptation
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);University of Science and Technology of China(中国科学技术大学);School of Intelligence Science and Technology, University of Science and Technology Beijing(北京科技大学智能科学与技术学院)
Sim2O: Efficient Offline-to-Online MARL via Joint Action Composition
Department of Computer Science and Engineering, Washington University in St. Louis(圣路易斯华盛顿大学计算机科学与工程系);The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China(中国科学院自动化研究所复杂系统认知与决策智能重点实验室)
Policy-Driven World Model Adaptation for Robust Offline Model-based Reinforcement Learning
The University of Hong Kong, Hong Kong SAR(香港大学);INFIFORCE Intelligent Technology Co., Ltd., Hangzhou, China(INFIFORCE智能技术有限公司);Tsinghua University, Beijing, China(清华大学);Carnegie Mellon University, Pittsburgh, PA 15213, USA(卡内基梅隆大学)
Ensemble Successor Representations for Task Generalization in Offline-to-Online Reinforcement Learning
Harbin Institute of Technology(哈尔滨工业大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Northwestern Polytechnical University(西北工业大学);Shenzhen Research Institute of Northwestern Polytechnical University(西北工业大学深圳研究院)
RoMA: Robust Model Adaptation for Offline Model-based Optimization
Korea Advanced Institute of Science and Technology (KAIST)(韩国科学技术院);Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)(穆罕默德·本·扎耶德人工智能大学);BioMap(百图生科)
Towards Robust Offline-to-Online Reinforcement Learning via Uncertainty and Smoothness
Northwestern Polytechnical University(西北工业大学);Harbin Institute of Technology(哈尔滨工业大学);The Hong Kong University of Science and Technology(香港科技大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Shenzhen Research Institute of Northwestern Polytechnical University(西北工业大学深圳研究院)
Adaptive prediction theory combining offline and online learning
State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China(数学科学国家重点实验室,数学与系统科学研究院,中国科学院,北京);School of Mathematical Science, University of Chinese Academy of Sciences, Beijing 100049, China(数学科学学院,中国科学院大学,北京)
Semantic Caching for Low-Cost LLM Serving: From Offline Learning to Online Adaptation
University of Washington(华盛顿大学);Carnegie Mellon University(卡内基梅隆大学);The Chinese University of Hong Kong(香港中文大学);City University of Hong Kong(香港城市大学);Microsoft Research(微软研究院)
Provably Efficient Third-Person Imitation from Offline Observation
Courant Institute of Mathematical Sciences, New York University(纽约大学库朗数学科学研究所);Center for Data Science, New York University(纽约大学数据科学中心);Institute for Advanced Study, Princeton(普林斯顿高等研究院)
Tailoring deep learning for real-time brain-computer interfaces: From offline models to calibration-free online decoding
Institute of Signal Processing and System Theory, University of Stuttgart(信号处理与系统理论研究所,斯图加特大学);Clinical Neurotechnology Laboratory, Department of Psychiatry and Neurosciences, Charité Campus Mitte (CCM), Charité - Universitätsmedizin Berlin(临床神经技术实验室,精神病学与神经科学系,Charité-柏林大学医学院)