arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2607.00796cs.LG
收藏

面向视觉强化学习泛化的任务相关表示解耦

Task-Relevant Representation Decoupling for Visual Reinforcement Learning Generalization

Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, School of Computer Science & Technology, Beijing Jiaotong University(北京交通大学计算机科学与技术学院交通数据挖掘与具身智能北京市重点实验室);CSSC Intelligent Innovation Research Institute(中国船舶集团智能创新研究院);Zhejiang University(浙江大学)

Jinwen Wang, Youfang Lin, Xiaobo Hu, Qian Xu, Shuo Wang, Zhuo Chen, Kai Lv

AI总结:

提出自监督任务相关表示解耦算法T2RD,通过表示一致性、交叉重建和交叉动态预测分离任务相关与无关特征,在控制任务中实现SOTA泛化性能。

摘要:

视觉强化学习在解决控制任务方面取得了显著成功。然而,将学习到的策略泛化到新环境仍然是一个主要挑战,因为智能体往往过度适应训练环境中的任务无关特征。为了解决这个问题,我们引入了将观测解耦为任务相关和任务无关表示的概念。基于这一思想,我们提出了一种用于VRL的自监督任务相关表示解耦(T2RD)算法。该算法由三个部分组成:任务相关表示一致性、交叉重建和交叉动态预测。前两个部分实现了内容和风格特征的解耦,但得到的内容表示不一定与任务相关。为了进一步从内容表示中提炼任务相关特征,我们设计了引入动态预测的第三个部分。T2RD在DeepMind控制套件和机器人操作任务中实现了最先进的泛化性能和样本效率。

英文摘要:

Visual Reinforcement Learning (VRL) has achieved considerable success in solving control tasks. However, generalizing learned policies to new environments remains a major challenge, as agents often overfit to task-irrelevant features in the training environment. To solve this problem, we introduce the concept of decoupling observations into task-relevant and task-irrelevant representations. Building on this idea, we propose a self-supervised Task-Relevant Representation Decoupling (T2RD) algorithm for VRL. This algorithm consists of three components: task-relevant representation consistency, cross-reconstruction, and cross-dynamic prediction. The first two components achieve the decoupling of content and style features, but the resulting content representations are not necessarily task-relevant. To further refine task-relevant features from content representations, we design the third component that introduces dynamic prediction. T2RD achieves State-Of-The-Art (SOTA) generalization performance and sample efficiency in the DeepMind Control Suite and Robotic Manipulation tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑