Machine learning potentials (MLPs) trained on accurate quantum chemical data can retain the high accuracy, while inflicting little computational demands. On the downside, they need to be trained for each individual system. In recent years, a vast number of MLPs has been trained from scratch because learning additional data typically requires to train again on all data to not forget previously acquired knowledge. Additionally, most common structural descriptors of MLPs cannot represent efficiently a large number of different chemical elements. In this work, we tackle these problems by introducing element-embracing atom-centered symmetry functions (eeACSFs) which combine structural properties and element information from the periodic table. These eeACSFs are a key for our development of a lifelong machine learning potential (lMLP). Uncertainty quantification can be exploited to transgress a fixed, pre-trained MLP to arrive at a continuously adapting lMLP, because a predefined level of accuracy can be ensured. To extend the applicability of an lMLP to new systems, we apply continual learning strategies to enable autonomous and on-the-fly training on a continuous stream of new data. For the training of deep neural networks, we propose the continual resilient (CoRe) optimizer and incremental learning strategies relying on rehearsal of data, regularization of parameters, and the architecture of the model.
When New Generators Arrive: Lifelong Machine-Generated Text Attribution via Ridge Feature Transfer
Wuhan University(武汉大学);Ant Group(蚂蚁集团);The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));Institute of Deep Perception Technology, JITRI(感知技术研究院,JITRI)
An Auto-ML Framework Based on GBDT for Lifelong Learning
Beijing University of Posts and Telecommunications(北京邮电大学);Central South University(中南大学);Inspur Electronic Information Industry Co.,Ltd(浪潮电子信息产业股份有限公司)
An Empirical Investigation of the Role of Pre-training in Lifelong Learning
Carnegie Mellon University(卡内基梅隆大学);Mila - Quebec AI Institute(米拉-魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);École Polytechnique de Montréal(蒙特利尔综合理工学院)
Neuromorphic Cybersecurity with Semi-supervised Lifelong Learning
School of EECS(电子工程与科学学院);The Pennsylvania State University(宾夕法尼亚州立大学);Dept. of EECS(电子工程与科学系);University of Michigan(密歇根大学);Quantum Ventura, Inc.(Quantum Ventura 公司);University of Texas at Arlington Research Institute(德克萨斯大学阿灵顿研究 institute)
AutoML @ NeurIPS 2018 challenge: Design and Results
Instituto Nacional de Astrofísica, Óptica, y Electrónica(墨西哥国家天体物理、光学与电子研究所);ChaLearn;Paradigm(第四范式);UPSud/INRIA Université Paris-Saclay(巴黎萨克雷大学巴黎第十一大学/法国国家信息与自动化研究所);Acadia University(阿卡迪亚大学);Microsoft Research(微软研究院);Hong Kong University of Science and Technology(香港科技大学)
Quantum continual learning of quantum data realizing knowledge backward transfer
South China Agricultural University(华南农业大学);Sun Yat-sen University(中山大学);Guilin University of Electronic Technology(桂林电子科技大学);Tsinghua University(清华大学)