The human brain uses selective attention to filter perceptual input so that only the components that are useful for behaviour are processed using its limited computational resources. We focus on one particular form of visual attention known as feature-based attention, which is concerned with identifying features of the visual input that are important for the current task regardless of their spatial location. Visual feature-based attention has been proposed to improve the efficiency of Reinforcement Learning (RL) by reducing the dimensionality of state representations and guiding learning towards relevant features. Despite achieving human level performance in complex perceptual-motor tasks, Deep RL algorithms have been consistently criticised for their poor efficiency and lack of flexibility. Visual feature-based attention therefore represents one option for addressing these criticisms. Nevertheless, it is still an open question how the brain is able to learn which features to attend to during RL. To help answer this question we propose a novel algorithm, termed Selective Particle Attention (SPA), which imbues a Deep RL agent with the ability to perform selective feature-based attention. SPA learns which combinations of features to attend to based on their bottom-up saliency and how accurately they predict future reward. We evaluate SPA on a multiple choice task and a 2D video game that both involve raw pixel input and dynamic changes to the task structure. We show various benefits of SPA over approaches that naively attend to either all or random subsets of features. Our results demonstrate (1) how visual feature-based attention in Deep RL models can improve their learning efficiency and ability to deal with sudden changes in task structure and (2) that particle filters may represent a viable computational account of how visual feature-based attention occurs in the brain.
What If Consensus Lies? Selective-Complementary Reinforcement Learning at Test Time
School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Nanjing University（南京大学）
Selective experience replay compression using coresets for lifelong deep reinforcement learning in medical imaging
Rice University（莱斯大学）；McGovern Medical School, UTHealth Houston（德克萨斯大学健康科学中心休斯顿麦戈文医学院）；The Johns Hopkins University School of Medicine（约翰斯·霍普金斯大学医学院）；University of Maryland School of Medicine（马里兰大学医学院）
John A. Paulson School of Engineering and Applied Sciences, Harvard University（哈佛大学约翰·A·保尔森工程与应用科学学院）；Computer Science Department, Stanford University（斯坦福大学计算机科学系）；The Taub Faculty of Computer Science, Technion - Israel Institute of Technology（以色列理工学院陶布计算机科学学院）
Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts
Carnegie Mellon University（卡内基梅隆大学）；Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Meta AI
Beyond Penalization: Diffusion-based Out-of-Distribution Detection and Selective Regularization in Offline Reinforcement Learning
School of Computer Science and Technology, Tongji University, Shanghai, China（同济大学计算机科学与技术学院，上海，中国）；MOE Key Lab of Embedded System and Service Computing, Tongji University, Shanghai, China（教育部嵌入式系统与服务计算重点实验室，同济大学，上海，中国）；Shanghai Innovation Institute（上海创新研究院）
Deep Reinforcement Learning Models Predict Visual Responses in the Brain: A Preliminary Result
Vidyasirimedhi Institute of Science and Technology（威达亚西里梅迪科技学院）；Imperial College London（帝国理工学院）；King Mongkut’s University of Technology Thonburi（国王科技大学）
SkillScaffold: Scaffolding Agentic Reinforcement Learning with Policy-Adaptive Skill Credit Assignment and Selective Injection
Zhejiang University（浙江大学）；Kuaishou Technology（快手科技）；University of the Chinese Academy of Sciences（中国科学院大学）；Zhejiang University of Science and Technology（浙江科技学院）
IVR-R1: Refining Trajectories through Iterative Visual-Grounded Reasoning in Reinforcement Learning
Hangzhou International Innovation Institute, Beihang University（北京航空航天大学杭州国际创新研究院）；School of Artificial Intelligence, Beihang University（北京航空航天大学人工智能学院）；Kuaishou Technology（快手科技）；Shenzhen Institute of Advanced Integration Technology, Shenzhen（深圳先进集成技术研究院）
Look where you look! Saliency-guided Q-networks for generalization in visual Reinforcement Learning
IRT Saint-Exupéry（圣埃克苏佩里工业技术研究院）；ISAE-SUPAERO（法国高等航空航天学院）；IMT, INSA Toulouse（图卢兹国立应用科学学院数学与信息学研究所）；ANITI（图卢兹人工智能与自然智能研究所）；IMT, Université Paul Sabatier（保罗·萨巴蒂埃大学数学与信息学研究所）；Université de Toulouse（图卢兹大学）
深度强化学习策略尽管在模拟视觉控制任务中具有出色的效率，但在输入训练图像中对干扰进行概括的能力令人失望。图像统计数据或分散注意力的背景元素的变化是防止这种控制策略的通用性和现实世界适用性的陷阱。我们阐述的直觉，一个好的视觉政策应该能够识别哪些像素是重要的，它的决定，并保持这种识别跨图像的重要信息源。这意味着具有小泛化差距的策略的训练应该集中在这些重要的像素上，而忽略其他像素。这导致了显着性引导Q网络（SGQN）的引入，这是一种用于视觉强化学习的通用方法，与任何价值函数学习方法兼容。SGQN极大地提高了Soft Actor-Critic代理的泛化能力，并在Deepmind Control Generalization基准测试中优于现有的最先进方法，在训练效率、泛化差距和策略可解释性方面提供了新的参考。
Decision Mamba: Reinforcement Learning via Hybrid Selective Sequence Modeling
School of Artificial Intelligence, Jilin University（吉林大学人工智能学院）；Institute of High Performance Computing, A*STAR（新加坡高性能计算研究院（新加坡科技研究局））；Lehigh University（理海大学）
我们提出了DrQ-v2，一个无模型强化学习（RL）算法的视觉连续控制。DrQ-v2建立在DrQ的基础上，DrQ是一种非策略的行动者-批评者方法，使用数据增强直接从像素中学习。我们介绍了几项改进，这些改进在DeepMind Control Suite上产生了最先进的结果。值得注意的是，DrQ-v2能够直接从像素观察中解决复杂的人形运动任务，这是以前无模型RL无法实现的。DrQ-v2概念简单，易于实现，与以前的工作相比，它提供了更好的计算占用空间，大多数任务只需8小时即可在单个GPU上进行训练。最后，我们公开发布DrQ-v2的实现，为RL从业者提供强大且计算效率高的基线。
A Visual Communication Map for Multi-Agent Deep Reinforcement Learning
Institute for Intelligent Systems Research and Innovation, Deakin University（迪肯大学智能系统研究与创新研究所）；School of Information Technology, Deakin University（迪肯大学信息技术学院）
