arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2304.13892cs.LGcs.AI
收藏

从像素中发现以对象为中心的广义价值函数

Discovering Object-Centric Generalized Value Functions From Pixels

École de technologie supérieure(高等技术学院);Mila-Quebec AI Institute(米拉-魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);McGill University(麦吉尔大学);CIFAR AI Chair(加拿大高等研究院人工智能讲席);DeepMind(深度思维)

Somjit Nath, Gopeshh Raaj Subbaraj, Khimya Khetarpal, Samira Ebrahimi Kahou

AI总结:

本文提出一种从像素中发现以对象为中心的特征并将其转化为时间连贯的“问题”函数以利用广义价值函数进行控制的方法,在平稳和非平稳环境中表现优异,且学习到的表征具有可解释性并促进快速适应。

摘要:

深度强化学习在从高维输入中提取有用表征方面取得了显著进展,尽管这依赖于手工设计的辅助任务和伪奖励。以面向控制和快速适应的以对象为中心的方式自动学习此类表征,仍然是一个开放的研究问题。本文提出一种方法,试图从对象中发现有意义的特征,将其转化为时间连贯的“问题”函数,并利用随后学习到的广义价值函数进行控制。我们将该方法与最先进的技术及其他消融实验进行比较,展示了在平稳和非平稳环境下的竞争性能。最后,我们还研究了所发现的广义价值函数,并通过定性分析表明,学习到的表征不仅具有可解释性,而且以对象为中心,这些对象在不同任务间的变化中保持不变,从而促进快速适应。

英文摘要:

Deep Reinforcement Learning has shown significant progress in extracting useful representations from high-dimensional inputs albeit using hand-crafted auxiliary tasks and pseudo rewards. Automatically learning such representations in an object-centric manner geared towards control and fast adaptation remains an open research problem. In this paper, we introduce a method that tries to discover meaningful features from objects, translating them to temporally coherent "question" functions and leveraging the subsequent learned general value functions for control. We compare our approach with state-of-the-art techniques alongside other ablations and show competitive performance in both stationary and non-stationary settings. Finally, we also investigate the discovered general value functions and through qualitative analysis show that the learned representations are not only interpretable but also, centered around objects that are invariant to changes across tasks facilitating fast adaptation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑