This paper presents a comprehensive overview of autotelic Reinforcement Learning (RL), emphasizing the role of intrinsic motivations in the open-ended formation of skill repertoires. We delineate the distinctions between knowledge-based and competence-based intrinsic motivations, illustrating how these concepts inform the development of autonomous agents capable of generating and pursuing self-defined goals. The typology of Intrinsically Motivated Goal Exploration Processes (IMGEPs) is explored, with a focus on the implications for multi-goal RL and developmental robotics. The autotelic learning problem is framed within a reward-free Markov Decision Process (MDP), WHERE agents must autonomously represent, generate, and master their own goals. We address the unique challenges in evaluating such agents, proposing various metrics for measuring exploration, generalization, and robustness in complex environments. This work aims to advance the understanding of autotelic RL agents and their potential for enhancing skill acquisition in a diverse and dynamic setting.
A Definition of Open-Ended Learning Problems for Goal-Conditioned Agents
Sorbonne Université（索邦大学）；CNRS（法国国家科学研究中心）；Institut des Systèmes Intelligents et de Robotique（智能系统与机器人研究所）；INRIA Bordeaux Sud-Ouest（法国国家信息与自动化研究所波尔多南部-西部分部）；Institute of Cognitive Sciences and Technologies（认知科学与技术研究所）；CNR（意大利国家研究委员会）；CITIC（信息与通信技术中心）；Universidade da Coruña（拉科鲁尼亚大学）；Massachusetts Institute of Technology（麻省理工学院）
Autonomous Open-Ended Learning of Tasks with Non-Stationary Interdependencies
Universidade da Coruña（拉科鲁尼亚大学）；Consiglio Nazionale delle Ricerche（意大利国家研究委员会）；Istituto di Scienze e Tecnologie della Cognizione（认知科学与技术研究所）；CITIC research center（CITIC研究中心）
The Conditions of Physical Embodiment Enable Generalization and Care
Institute for Advanced Consciousness Studies（先进意识研究所）；Monash Centre for Consciousness and Contemplative Studies（莫纳什意识与冥想研究中心）；University of Oxford（牛津大学）；Topos Institute（拓斯研究所）；Allen Discovery Center（艾伦发现中心）；Okinawa Institute of Science and Technology（冲绳科学技术研究所）；Max Planck Institute for Human Cognitive and Brain Sciences（马克斯·普朗克人类认知与脑科学研究所）；University of Hertfordshire（赫特福德郡大学）；Brain and Creativity Institute（大脑与创造力研究所）
Laboratory of Computational Embodied Neuroscience, Istituto di Scienze e Tecnologie della Cognizione, Consiglio Nazionale delle Ricerche（国家研究委员会认知科学与技术研究所计算具身神经科学实验室）
A Motivational Architecture for Open-Ended Learning Challenges in Robots
Integrated Group for Engineering Research (GII) CITIC research center Universidade da Coruña（整合工程研究组（GII）CITIC研究中心乌拉圭大学）；Istituto di Scienze e Tecnologie della Cognizione (ISTC) Consiglio Nazionale delle Ricerche (CNR)（认知科学与技术研究所（ISTC）国家研究委员会（CNR））
Centre for the Governance of AI（治理人工智能中心）；Harvard Law School（哈佛法学院）；University of Oxford（牛津大学）；University of Cambridge（剑桥大学）；Australian National University（澳大利亚国立大学）；Johns Hopkins University（约翰霍普金斯大学）
