Developmental machine learning studies how artificial agents can model the way children learn open-ended repertoires of skills. Such agents need to create and represent goals, select which ones to pursue and learn to achieve them. Recent approaches have considered goal spaces that were either fixed and hand-defined or learned using generative models of states. This limited agents to sample goals within the distribution of known effects. We argue that the ability to imagine out-of-distribution goals is key to enable creative discoveries and open-ended learning. Children do so by leveraging the compositionality of language as a tool to imagine descriptions of outcomes they never experienced before, targeting them as goals during play. We introduce IMAGINE, an intrinsically motivated deep reinforcement learning architecture that models this ability. Such imaginative agents, like children, benefit from the guidance of a social peer who provides language descriptions. To take advantage of goal imagination, agents must be able to leverage these descriptions to interpret their imagined out-of-distribution goals. This generalization is made possible by modularity: a decomposition between learned goal-achievement reward function and policy relying on deep sets, gated attention and object-centered representations. We introduce the Playground environment and study how this form of goal imagination improves generalization and exploration over agents lacking this capacity. In addition, we identify the properties of goal imagination that enable these results and study the impacts of modularity and social interactions.
Psychological curiosity plays a significant role in human intelligence to enhance learning through exploration and information acquisition. In the Artificial Intelligence (AI) community, artificial curiosity provides a natural intrinsic motivation for efficient learning as inspired by human cognitive development; meanwhile, it can bridge the existing gap between AI research and practical application scenarios, such as overfitting, poor generalization, limited training samples, high computational cost, etc. As a result, curiosity-driven learning (CDL) has become increasingly popular, where agen…
The accumulation of adaptations in an open-ended manner during lifetime learning is a holy grail in reinforcement learning, intrinsic motivation, artificial curiosity, and developmental robotics. We present a specification for a cognitive architecture that is capable of specifying an unlimited range of behaviors. We then give examples of how it can stochastically explore an interesting space of adjacent possible behaviors. There are two main novelties; the first is a proper definition of the fitness of self-generated games such that interesting games are expected to evolve. The second is a mod…
Institute for Creative Technologies, USC（南加州大学创意技术研究所）；Cal State University Northridge（加州州立大学北岭分校）；Univ. of Southern California（南加州大学）；Hughes Research Laboratories (HRL)（休斯研究实验室）
Simulations, along with other similar applications like virtual worlds and video games, require computational models of intelligence that generate realistic and credible behavior for the participating synthetic characters. Cognitive architectures, which are models of the fixed structure underlying intelligent behavior in both natural and artificial systems, provide a conceptually valid common basis, as evidenced by the current efforts towards a standard model of the mind, to generate human-like intelligent behavior for these synthetic characters. Sigma is a cognitive architecture and system th…
Universität Hamburg（汉堡大学）；Universität Tübingen（图宾根大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
Cognitive Psychology and related disciplines have identified several critical mechanisms that enable intelligent biological agents to learn to solve complex problems. There exists pressing evidence that the cognitive mechanisms that enable problem-solving skills in these species build on hierarchical mental representations. Among the most promising computational approaches to provide comparable learning-based problem-solving abilities for artificial agents and robots is hierarchical reinforcement learning. However, so far the existing computational approaches have not been able to equip artifi…
What are the functions of curiosity? What are the mechanisms of curiosity-driven learning? We approach these questions about the living using concepts and tools from machine learning and developmental robotics. We argue that curiosity-driven learning enables organisms to make discoveries to solve complex problems with rare or deceptive rewards. By fostering exploration and discovery of a diversity of behavioural skills, and ignoring these rewards, curiosity can be efficient to bootstrap learning when there is no information, or deceptive information, about local improvement towards these probl…
In this work, we study the problem of Embodied Referring Expression Grounding, where an agent needs to navigate in a previously unseen environment and localize a remote object described by a concise high-level natural language instruction. When facing such a situation, a human tends to imagine what the destination may look like and to explore the environment based on prior knowledge of the environmental layout, such as the fact that a bathroom is more likely to be found near a bedroom than a kitchen. We have designed an autonomous agent called Layout-aware Dreamer (LAD), including two novel mo…
Quriosity: Analyzing Human Questioning Behavior and Causal Inquiry through Curiosity-Driven Queries
EPFL（苏黎世联邦理工学院）；ETH Zürich（苏黎世联邦理工学院）；University of Washington（华盛顿大学）；University of Toronto（多伦多大学）；Vector Institute（向量研究所）；University of Michigan（密歇根大学）；MPI for Intelligent Systems, Tübingen, Germany（图宾根智能系统研究所）
The impact of continually evolving digital technologies and the proliferation of communications and content has now been widely acknowledged to be central to understanding our world. What is less acknowledged is that this is based on the successful arousing of curiosity both at the collective and individual levels. Advertisers, communication professionals and news editors are in constant competition to capture attention of the digital population perennially shifty and distracted. This paper, tries to understand how curiosity works in the digital world by attempting the first ever work done on …
GGBond: Growing Graph-Based AI-Agent Society for Socially-Aware Recommender Simulation
Faculty of Science and Technology, Beijing Normal-Hong Kong Baptist University（科技学院，北京师范大学-香港 Baptist大学）；Research Centers for Mathematics, Advanced Institute of Natural Sciences, Beijing Normal University（数学研究中心，北京师范大学自然科学院）
