本文提出基于 FiLM 条件化的 RND 反探索方法，解决离线强化学习中分布外动作惩罚问题，在 D4RL 上性能媲美集成方法且大幅优于无集成方法。
摘要：
尽管 Random Network Distillation（RND）在多个领域取得了成功，但研究表明其作为不确定性估计器不足以惩罚离线强化学习中的分布外动作。本文重新审视这些结果，发现若对 RND 先验采用朴素的条件化选择，actor 将无法有效最小化反探索奖励，而判别性并非问题所在。我们证明，基于 Feature-wise Linear Modulation（FiLM）的条件化可避免这一局限，从而得到一种简单高效的、无需集成的 Soft Actor-Critic 算法。我们在 D4RL 基准上进行评估，结果表明其性能可与基于集成的方法相当，并大幅优于无需集成的方法。
英文摘要：
Despite the success of Random Network Distillation (RND) in various domains, it was shown as not discriminative enough to be used as an uncertainty estimator for penalizing out-of-distribution actions in offline reinforcement learning. In this paper, we revisit these results and show that, with a naive choice of conditioning for the RND prior, it becomes infeasible for the actor to effectively minimize the anti-exploration bonus and discriminativity is not an issue. We show that this limitation can be avoided with conditioning based on Feature-wise Linear Modulation (FiLM), resulting in a simple and efficient ensemble-free algorithm based on Soft Actor-Critic. We evaluate it on the D4RL benchmark, showing that it is capable of achieving performance comparable to ensemble-based methods and outperforming ensemble-free approaches by a wide margin.
EDGE: Experience-Distillation for Guided Exploration in Agentic Reinforcement Learning
School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences（中国科学院大学先进交叉科学学院）；Wuhan AI Research（武汉人工智能研究院）
Teacher-Guided Policy Optimization for On-Policy Reasoning Distillation under Large Policy Divergence
School of Computer Science and Engineering, Northeastern University, China（东北大学计算机科学与工程学院）；Tsinghua University（清华大学）；Meituan（美团）；Meta AI；NiuTrans Research, Shenyang, China（新译研究院，沈阳，中国）
Offline Exploration-Aware Fine-Tuning for Long-Chain Mathematical Reasoning
NLP Lab, School of Computer Science and Engineering, Northeastern University, Shenyang, China（东北大学计算机科学与工程学院自然语言处理实验室，中国沈阳）；Pattern Recognition Center, WeChat AI, Tencent Inc, China（腾讯公司微信人工智能部门模式识别中心，中国）
Double Distillation Network for Multi-Agent Reinforcement Learning
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；School of Automation Engineering, University of Electronic Science and Technology of China（电子科技大学自动化工程学院）
Efficient Anti-exploration via VQVAE and Fuzzy Clustering in Offline Reinforcement Learning
State Key Laboratory of Multimodal Artificial Intelligence Systems（多模态人工智能系统国家重点实验室）；State Key Laboratory of Management and Control for Complex Systems（复杂系统管理与控制国家重点实验室）；Chinese Academy of Sciences（中国科学院）；WAYTOUS Inc.（WAYTOUS公司）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东人工智能与数字经济实验室（深圳））；Institute of Artificial Intelligence and Robotics（人工智能与机器人研究院）；Xi’an Jiaotong University（西安交通大学）；School of Artificial Intelligence（人工智能学院）；Key Laboratory of Intelligent Sensing System and Security (Hubei University)（智能感知系统与安全重点实验室（湖北大学））；Ministry of Education（教育部）；Department of Electrical and Computer Engineering（电气与计算机工程系）
From Proprietary to Open-Source: Bridging the Distribution Gap via Multi-Agent Protocol Distillation in Agentic Search
Beijing Institute of Technology（北京理工大学）；East China Normal University（华东师范大学）；University of Science and Technology of China（中国科学技术大学）；Tsinghua University（清华大学）；University of Chinese Academy of Sciences（中国科学院大学）
Random Network Distillation Based Deep Reinforcement Learning for AGV Path Planning
School of Electronic and Information Engineering, Tongji University（同济大学电子与信息工程学院）；Virtual Vehicle Research GmbH（虚拟车辆研究有限公司）；Graz University of Technology（格拉茨工业大学）
EvoCoT: Overcoming the Exploration Bottleneck in Reinforcement Learning
School of Computer Science, Peking University（北京大学计算机科学学院）；College of AI, Tsinghua University（清华大学人工智能学院）；Imperial College London（伦敦帝国理工学院）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）
