In many complex dynamical systems, artificial or natural, one can observe self-organization of patterns emerging from local rules. Cellular automata, like the Game of Life (GOL), have been widely used as abstract models enabling the study of various aspects of self-organization and morphogenesis, such as the emergence of spatially localized patterns. However, findings of self-organized patterns in such models have so far relied on manual tuning of parameters and initial states, and on the human eye to identify interesting patterns. In this paper, we formulate the problem of automated discovery of diverse self-organized patterns in such high-dimensional complex dynamical systems, as well as a framework for experimentation and evaluation. Using a continuous GOL as a testbed, we show that recent intrinsically-motivated machine learning algorithms (POP-IMGEPs), initially developed for learning of inverse models in robotics, can be transposed and used in this novel application area. These algorithms combine intrinsically-motivated goal exploration and unsupervised learning of goal space representations. Goal space representations describe the interesting features of patterns for which diverse variations should be discovered. In particular, we compare various approaches to define and learn goal space representations from the perspective of discovering diverse spatially localized patterns. Moreover, we introduce an extension of a state-of-the-art POP-IMGEP algorithm which incrementally learns a goal representation using a deep auto-encoder, and the use of CPPN primitives for generating initialization parameters. We show that it is more efficient than several baselines and equally efficient as a system pre-trained on a hand-made database of patterns identified by human experts.
Time Series Motif Discovery (TSMD) is defined as searching for patterns that are previously unknown and appear with a given frequency in time series. Another problem strongly related with TSMD is Word Segmentation. This problem has received much attention from the community that studies early language acquisition in babies and toddlers. The development of biologically plausible models for word segmentation could greatly advance this field. Therefore, in this article, we propose the Variable Input Length Map (VILMAP) for Motif Discovery and Word Segmentation. The model is based on the Self-Orga…
Center for Data Science, New York University（纽约大学数据科学中心）；Department of Psychology, New York University（纽约大学心理学系）；Department of Computer Science, University of Sheffield（谢菲尔德大学计算机科学系）；Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology（麻省理工学院脑与认知科学系）；Center for Brains Minds and Machines（大脑心智与机器中心）
Both scientists and children make important structural discoveries, yet their computational underpinnings are not well understood. Structure discovery has previously been formalized as probabilistic inference about the right structural form --- where form could be a tree, ring, chain, grid, etc. [Kemp & Tenenbaum (2008). The discovery of structural form. PNAS, 105(3), 10687-10692]. While this approach can learn intuitive organizations, including a tree for animals and a ring for the color circle, it assumes a strong inductive bias that considers only these particular forms, and each form is ex…
National Research Institute in Computer Science and Control (INRIA)（法国国家信息与自动化研究所（INRIA））
In data analysis new forms of complex data have to be considered like for example (symbolic data, functional data, web data, trees, SQL query and multimedia data, ...). In this context classical data analysis for knowledge discovery based on calculating the center of gravity can not be used because input are not $\mathbb{R}^p$ vectors. In this paper, we present an application on real world symbolic data using the self-organizing map. To this end, we propose an extension of the self-organizing map that can handle symbolic data.
Self-organization of complex morphological patterns from local interactions is a fascinating phenomenon in many natural and artificial systems. In the artificial world, typical examples of such morphogenetic systems are cellular automata. Yet, their mechanisms are often very hard to grasp and so far scientific discoveries of novel patterns have primarily been relying on manual tuning and ad hoc exploratory search. The problem of automated diversity-driven discovery in these systems was recently introduced [26, 62], highlighting that two key ingredients are autonomous exploration and unsupervis…
Intrinsically motivated spontaneous exploration is a key enabler of autonomous developmental learning in human children. It enables the discovery of skill repertoires through autotelic learning, i.e. the self-generation, self-selection, self-ordering and self-experimentation of learning goals. We present an algorithmic approach called Intrinsically Motivated Goal Exploration Processes (IMGEP) to enable similar properties of autonomous learning in machines. The IMGEP architecture relies on several principles: 1) self-generation of goals, generalized as parameterized fitness functions; 2) select…
Nova Information Management School (NOVA IMS)（诺瓦信息管理学院（NOVA IMS））；Universidade Nova de Lisboa（新里斯本大学）
Motif discovery is a fundamental step in data mining tasks for time-series data such as clustering, classification and anomaly detection. Even though many papers have addressed the problem of how to find motifs in time-series by proposing new motif discovery algorithms, not much work has been done on the exploration of the motifs extracted by these algorithms. In this paper, we argue that visually exploring time-series motifs computed by motif discovery algorithms can be useful to understand and debug results. To explore the output of motif discovery algorithms, we propose the use of an adapte…
Sparse, self-organizing ensembles of local kernels detect rare statistical anomalies
NSF AI Institute for Artificial Intelligence and Fundamental Interactions（NSF人工智能与基础相互作用研究院）；Laboratory for Nuclear Science（核科学实验室）；School of Engineering and Applied Sciences（工程与应用科学学院）；Kempner Institute for the Study of Natural and Artificial Intelligence（自然与人工智能研究学院）
Hebbian Physics Networks: A Self-Organizing Computational Architecture Based on Local Physical Laws
Department of Mechanical Engineering, The University of Tokyo, Japan（东京大学机械工程系）；Graduate School of Engineering, Nagoya Institute of Technology, Japan（名古屋技术大学工学研究科）；International Research Center for Neurointelligence, The University of Tokyo, Japan（东京大学神经智能国际研究中心）
SelfAI: A self-directed framework for long-horizon scientific discovery
University of Electronic Science and Technology of China（电子科技大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Xian Jiaotong University（西安交通大学）；Yale University（耶鲁大学）；Washington University in St. Louis（圣路易斯华盛顿大学）；Monash University（莫纳什大学）
Structured Scaling of AI Discovery Across Diverse Scientific Domains
Wizard Intelligence Learning Lab, Stanford University（智能巫师学习实验室，斯坦福大学）；Peking University（北京大学）；Tsinghua University（清华大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Perimeter Institute for Theoretical Physics（圆周理论物理研究所）；Harvard University（哈佛大学）；Ecole Polytechnique Federale de Lausanne（洛桑联邦理工学院）；Princeton University（普林斯顿大学）；University of Waterloo（滑铁卢大学）
Enhancing generalizability of model discovery across parameter space with multi-experiment equation learning (ME-EQL)
Departments of Mathematics and Biology, Duke University（数学与生物学系，杜克大学）；Department of Mathematics and Statistics, The College of New Jersey（数学与统计学系，新泽西学院）；Department of Mathematics, Center for Research in Scientific Computation, North Carolina State University（数学系，科学计算研究中心，北卡罗来纳州立大学）；Department of Applied Mathematics, University of California, Merced（应用数学系，加州大学默塞德分校）；Department of Mathematics, Purdue University（数学系，普渡大学）
The automatic and efficient discovery of skills, without supervision, for long-living autonomous agents, remains a challenge of Artificial Intelligence. Intrinsically Motivated Goal Exploration Processes give learning agents a human-inspired mechanism to sequentially select goals to achieve. This approach gives a new perspective on the lifelong learning problem, with promising results on both simulated and real-world experiments. Until recently, those algorithms were restricted to domains with experimenter-knowledge, since the Goal Space used by the agents was built on engineered feature extra…
Beijing National Laboratory for Molecular Sciences (BNLMS)（北京分子科学国家实验室）
Most algorithms for the multi-armed bandit problem in reinforcement learning aimed to maximize the expected reward, which are thus useful in searching the optimized candidate with the highest reward (function value) for diverse applications (e.g., AlphaGo). However, in some typical application scenaios such as drug discovery, the aim is to search a diverse set of candidates with high reward. Here we propose a reversible upper confidence bound (rUCB) algorithm for such a purpose, and demonstrate its application in virtual screening upon intrinsically disordered proteins (IDPs). It is shown that…
Hyperbolic Multimodal Representation Learning for Biological Taxonomies
Simon Fraser University（西蒙弗雷泽大学）；University of Waterloo（滑铁卢大学）；Vector Institute（向量研究所）；University of Guelph（圭尔夫大学）；Alberta Machine Intelligence Institute (Amii)（阿尔伯塔机器智能研究所）；Aalborg University（奥胡斯大学）
