Psychological curiosity plays a significant role in human intelligence to enhance learning through exploration and information acquisition. In the Artificial Intelligence (AI) community, artificial curiosity provides a natural intrinsic motivation for efficient learning as inspired by human cognitive development; meanwhile, it can bridge the existing gap between AI research and practical application scenarios, such as overfitting, poor generalization, limited training samples, high computational cost, etc. As a result, curiosity-driven learning (CDL) has become increasingly popular, where agents are self-motivated to learn novel knowledge. In this paper, we first present a comprehensive review on the psychological study of curiosity and summarize a unified framework for quantifying curiosity as well as its arousal mechanism. Based on the psychological principle, we further survey the literature of existing CDL methods in the fields of Reinforcement Learning, Recommendation, and Classification, where both advantages and disadvantages as well as future work are discussed. As a result, this work provides fruitful insights for future CDL research and yield possible directions for further improvement.
Hong Kong University of Science and Technology（香港科技大学）
The design of recommendations strategies in the adaptive learning system focuses on utilizing currently available information to provide individual-specific learning instructions for learners. As a critical motivate for human behaviors, curiosity is essentially the drive to explore knowledge and seek information. In a psychologically inspired view, we aim to incorporate the element of curiosity for guiding learners to study spontaneously. In this paper, a curiosity-driven recommendation policy is proposed under the reinforcement learning framework, allowing for a both efficient and enjoyable p…
Perelman School of Medicine, University of Pennsylvania（宾夕法尼亚大学佩雷尔曼医学院）；School of Engineering and Applied Sciences, University of Pennsylvania（宾夕法尼亚大学工程与应用科学学院）；American University（美利坚大学）；College of Arts and Sciences, University of Pennsylvania（宾夕法尼亚大学文理学院）；Santa Fe Institute（圣塔菲研究所）
Throughout life, we might seek a calling, companions, skills, entertainment, truth, self-knowledge, beauty, and edification. The practice of curiosity can be viewed as an extended and open-ended search for valuable information with hidden identity and location in a complex space of interconnected information. Despite its importance, curiosity has been challenging to computationally model because the practice of curiosity often flourishes without specific goals, external reward, or immediate feedback. Here, we show how network science, statistical physics, and philosophy can be integrated into …
University of Alberta（阿尔伯塔大学）；Alberta Machine Intelligence Institute (Amii)（阿尔伯塔机器智能研究所）；Sony AI（索尼人工智能）；DeepMind
Curiosity for machine agents has been a focus of lively research activity. The study of human and animal curiosity, particularly specific curiosity, has unearthed several properties that would offer important benefits for machine learners, but that have not yet been well-explored in machine intelligence. In this work, we conduct a comprehensive, multidisciplinary survey of the field of animal and machine curiosity. As a principal contribution of this work, we use this survey as a foundation to introduce and define what we consider to be five of the most important properties of specific curiosi…
University of Alberta（阿尔伯塔大学）；Alberta Machine Intelligence Institute（阿尔伯塔机器智能研究所）
Curiosity for machine agents has been a focus of intense research. The study of human and animal curiosity, particularly specific curiosity, has unearthed several properties that would offer important benefits for machine learners, but that have not yet been well-explored in machine intelligence. In this work, we introduce three of the most immediate of these properties -- directedness, cessation when satisfied, and voluntary exposure -- and show how they may be implemented together in a proof-of-concept reinforcement learning agent; further, we demonstrate how the properties manifest in the b…
Collective behavior is widespread across the animal kingdom. To date, however, the developmental and mechanistic foundations of collective behavior have not been formally established. What learning mechanisms drive the development of collective behavior in newborn animals? Here, we used deep reinforcement learning and curiosity-driven learning -- two learning mechanisms deeply rooted in psychological and neuroscientific research -- to build newborn artificial agents that develop collective behavior. Like newborn animals, our agents learn collective behavior from raw sensory inputs in naturalis…
The world of competitive Esports and video gaming has seen and continues to experience steady growth in popularity and complexity. Correspondingly, more research on the topic is being published, ranging from social network analyses to the benchmarking of advanced artificial intelligence systems in playing against humans. In this paper, we present ongoing work on an intelligent agent recommendation engine that suggests actions to players in order to maximise success and enjoyment, both in the space of in-game choices, as well as decisions made around play session timing in the broader context. …
Curiosity-Diffuser: Curiosity Guide Diffusion Models for Reliability
Research Center for Intelligent Robotics, School of Astronautics, Northwestern Polytechnical University（西北工业大学航天学院智能机器人研究中心）；National Key Laboratory of Aerospace Flight Dynamics, Northwestern Polytechnical University（西北工业大学航天飞行动力学国家重点实验室）
What are the functions of curiosity? What are the mechanisms of curiosity-driven learning? We approach these questions about the living using concepts and tools from machine learning and developmental robotics. We argue that curiosity-driven learning enables organisms to make discoveries to solve complex problems with rare or deceptive rewards. By fostering exploration and discovery of a diversity of behavioural skills, and ignoring these rewards, curiosity can be efficient to bootstrap learning when there is no information, or deceptive information, about local improvement towards these probl…
Exploring Artificial Intelligence Tutor Teammate Adaptability to Harness Discovery Curiosity and Promote Learning in the Context of Interactive Molecular Dynamics
Nowadays, model-free reinforcement learning algorithms have achieved remarkable performance on many decision making and control tasks, but high sample complexity and low sample efficiency still hinder the wide use of model-free reinforcement learning algorithms. In this paper, we argue that if we intend to design an intelligent agent that learns fast and transfers well, the agent must be able to reflect key elements of intelligence, like intuition, Memory, PredictionandCuriosity. We propose an agent framework that integrates off-policy reinforcement learning with world model learning, so as to…
City University of Hong Kong（香港城市大学）；New York University（纽约大学）；La Trobe University（乐卓博大学）；National Institute of Technology Calicut（卡利卡特国家理工学院）；SP Jain School of Global Management（SP Jain全球管理学院）；Singapore University of Social Sciences（新加坡社会科学大学）
We discuss the objectives of any endeavor in creating artificial intelligence, AI, and provide a possible alternative. Intelligence might be an unintended consequence of curiosity left to roam free, best exemplified by a frolicking infant. This suggests that our attempts at AI could have been misguided. What we actually need to strive for can be termed artificial curiosity, AC, and intelligence happens as a consequence of those efforts. For this unintentional yet welcome aftereffect to set in a foundational list of guiding principles needs to be present. We start with the intuition for this li…
Rutgers University–Newark（罗格斯大学纽瓦克分校）；Institute for Advanced Study（普林斯顿高等研究院）
The goal of explainable Artificial Intelligence (XAI) is to generate human-interpretable explanations, but there are no computationally precise theories of how humans interpret AI generated explanations. The lack of theory means that validation of XAI must be done empirically, on a case-by-case basis, which prevents systematic theory-building in XAI. We propose a psychological theory of how humans draw conclusions from saliency maps, the most common form of XAI explanation, which for the first time allows for precise prediction of explainee inference conditioned on explanation. Our theory posi…
Autonomous lifelong development and learning is a fundamental capability of humans, differentiating them from current deep learning systems. However, other branches of artificial intelligence have designed crucial ingredients towards autonomous learning: curiosity and intrinsic motivation, social learning and natural interaction with peers, and embodiment. These mechanisms guide exploration and autonomous choice of goals, and integrating them with deep learning opens stimulating perspectives. Deep learning (DL) approaches made great advances in artificial intelligence, but are still far away f…
Ortus is a simple virtual organism that also serves as an initial framework for investigating and developing biologically-based artificial intelligence. Born from a goal to create complex virtual intelligence and an initial attempt to model C. elegans, Ortus implements a number of mechanisms observed in organic nervous systems, and attempts to fill in unknowns based upon plausible biological implementations and psychological observations. Implemented mechanisms include excitatory and inhibitory chemical synapses, bidirectional gap junctions, and Hebbian learning with its Stentian extension. We…
Donders Institute for Brain, Cognition, and Behaviour（唐德斯大脑、认知与行为研究所）；Radboud University（拉德堡德大学）；Joint Research Centre, European Commission（欧盟委员会联合研究中心）；Ghent University（根特大学）；imec（微电子研究中心）；Instituto Superio Técnico（高等技术学院）；University of Lisbon（里斯本大学）
Understanding and modelling children's cognitive processes and their behaviour in the context of their interaction with robots and social artificial intelligence systems is a fundamental prerequisite for meaningful and effective robot interventions. However, children's development involve complex faculties such as exploration, creativity and curiosity which are challenging to model. Also, often children express themselves in a playful way which is different from a typical adult behaviour. Different children also have different needs, and it remains a challenge in the current state of the art t…
Position: Stop Evaluating AI with Human Tests, Develop Principled, AI-specific Tests instead
Max Planck Institute for Intelligent Systems, Tübingen（马克斯·普朗克智能系统研究所，图宾根）；ETH Zurich, Zurich（苏黎世联邦理工学院，苏黎世）；University of Tübingen, Methods Center, Tübingen（图宾根大学，方法中心，图宾根）；Massachusetts Institute of Technology, Cambridge（麻省理工学院，剑桥）
George Washington University（乔治华盛顿大学）；Vienna University of Economics and Business（维也纳经济大学）；University of Vienna（维也纳大学）
By defining the current limits (and thereby the frontiers), many boundaries are shaping, and will continue to shape, the future of Artificial Intelligence (AI). We push on these boundaries in order to make further progress into what were yesterday's frontiers. They are both pliable and resilient - always creating new boundaries of what AI can (or should) achieve. Among these are technical boundaries (such as processing capacity), psychological boundaries (such as human trust in AI systems), ethical boundaries (such as with AI weapons), and conceptual boundaries (such as the AI people can imagi…
Self-Emotion-Mediated Exploration in Artificial Intelligence Mirrors: Findings from Cognitive Psychology
University of Coimbra（科英布拉大学）；Institute for Biomedical Imaging and Translational Research（生物医学影像与转化研究 institutes）；Institute of Systems and Robotics（系统与机器人研究所）
Nanyang Technological University（南洋理工大学）；Alibaba-NTU Singapore Joint Research Institute（阿里巴巴-新加坡南洋理工大学联合研究院）；The University of British Columbia（不列颠哥伦比亚大学）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；Hong Kong University of Science and Technology（香港科技大学）
As artificial intelligence (AI) systems become increasingly ubiquitous, the topic of AI governance for ethical decision-making by AI has captured public imagination. Within the AI research community, this topic remains less familiar to many researchers. In this paper, we complement existing surveys, which largely focused on the psychological, social and legal discussions of the topic, with an analysis of recent advances in technical solutions for AI governance. By reviewing publications in leading AI conferences including AAAI, AAMAS, ECAI and IJCAI, we propose a taxonomy which divides the fie…
In this paper I argue that the search for explainable models and interpretable decisions in AI must be reformulated in terms of the broader project of offering a pragmatic and naturalistic account of understanding in AI. Intuitively, the purpose of providing an explanation of a model or a decision is to make it understandable to its stakeholders. But without a previous grasp of what it means to say that an agent understands a model or a decision, the explanatory strategies will lack a well-defined goal. Aside from providing a clearer objective for XAI, focusing on understanding also allows us …
How to advance general game playing artificial intelligence by player modelling
BrainWork Research Centre（脑力工作研究中心）；Finnish Institute of Occupational Health（芬兰职业健康研究所）；University of Helsinki（赫尔辛基大学）
由于被称为“深度学习”的各种技术，一般的玩游戏人工智能最近已经看到了重要的进步。然而，这些进步隐藏了同样重要的局限性，即它们依赖于：大量数据集;偶然构建的问题;以及缺乏任何人类水平的复杂性，包括其他人类对手。另一方面，深度学习系统确实击败了人类冠军，例如在围棋中，并不能很好地概括。深度学习的强大同时也暴露了它的弱点。鉴于深度学习主要是对成熟方法的巧妙重新配置，超越最先进的技术要求前瞻性的有远见的解决方案，而不仅仅是更多的相同。我提出的论点，一般的游戏玩人工智能将需要一个广义的球员模型。这是因为游戏本质上是人类的人工制品，因此，作为一类问题，包含需要人类风格解决问题方法的案例。我将这一论点与最先进的一般游戏代理的表现联系起来。然后，我描述了一个概念，一个正式的范畴理论基础，一个广义的球员模型。这种正式的模型方法集成了我现有的“Behavlets”方法，用于心理衍生的玩家建模：Cowley，B.，Charles，D.（2016年）。Behavlets：A Method for Practical Player Modelling Using Psychology-Based Player Traits and Domain Specific Features. User Modeling and User-Adapted Interaction，26（2），257-306.
A Generic Review of Integrating Artificial Intelligence in Cognitive Behavioral Therapy
School of Software Engineering, Beijing University of Technology（北京工业大学软件学院）；School of Nursing, Wuhan University（武汉大学护理学院）；Sorbonne Université（索邦大学）；ICM（巴黎脑研究所）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化研究所）；Inserm（法国国家健康与医学研究院）；AP-HP（巴黎公共医疗集团）；Hôpital de la Pitié-Salpêtrière（皮蒂-萨尔皮特里耶医院）
Princeton AI Lab（普林斯顿人工智能实验室）；Princeton University（普林斯顿大学）；Institute for Human-Centered AI（以人为中心的人工智能研究所）；Helmholtz Munich（海德堡-慕尼黑研究所）；Ludwig-Maximilians-University（路德维希-马克西米利安大学）
In the monograph "Strong artificial intelligence. On the Approaches to Superintelligence" contains an overview of general artificial intelligence (AGI). As an anthropomorphic research area, it includes Brain Principles Programming (BPP) -- the formalization of universal mechanisms (principles) of the brain work with information, which are implemented at all levels of the organization of nervous tissue. This monograph contains a formalization of these principles in terms of category theory. However, this formalization is not enough to develop algorithms for working with information. In this pap…
EHESS（高等社会科学研究院）；ENS（高等师范学校）；PSL Research University（巴黎文理研究大学）；CNRS（法国国家科学研究中心）；INRIA（法国国家信息与自动化研究所）
During their first years of life, infants learn the language(s) of their environment at an amazing speed despite large cross cultural variations in amount and complexity of the available language input. Understanding this simple fact still escapes current cognitive and linguistic theories. Recently, spectacular progress in the engineering science, notably, machine learning and wearable technology, offer the promise of revolutionizing the study of cognitive development. Machine learning offers powerful learning algorithms that can achieve human-like performance on many linguistic tasks. Wearabl…
University of Augsburg（奥格斯堡大学）；Imperial College London（帝国理工学院）
Charisma is considered as one's ability to attract and potentially also influence others. Clearly, there can be considerable interest from an artificial intelligence's (AI) perspective to provide it with such skill. Beyond, a plethora of use cases opens up for computational measurement of human charisma, such as for tutoring humans in the acquisition of charisma, mediating human-to-human conversation, or identifying charismatic individuals in big social data. A number of models exist that base charisma on various dimensions, often following the idea that charisma is given if someone could and …
