How more data can hurt: Instability and regularization in next-generation reservoir computing
Santa Fe Institute（圣达菲研究所）；Department of Electrical and Computer Engineering（电气与计算机工程系）；Clarkson University（克拉克逊大学）；Clarkson Center for Complex Systems Science（复杂系统科学中心）；Department of Physics（物理系）；Toronto Metropolitan University（多伦多 Metropolitan 大学）；Department of Automation（自动化系）
Yuanzhao Zhang, Edmilson Roque dos Santos, Huixin Zhang, Sean P. Cornelius
It has been found recently that more data can, counter-intuitively, hurt the performance of deep neural networks. Here, we show that a more extreme version of the phenomenon occurs in data-driven models of dynamical systems. To elucidate the underlying mechanism, we focus on next-generation reservoir computing (NGRC) -- a popular framework for learning dynamics from data. We find that, despite learning a better representation of the flow map with more training data, NGRC can adopt an ill-conditioned ``integrator'' and lose stability. We link this data-induced instability to the auxiliary dimensions created by the delayed states in NGRC. Based on these findings, we propose simple strategies to mitigate the instability, either by increasing regularization strength in tandem with data size, or by carefully introducing noise during training. Our results highlight the importance of proper regularization in data-driven modeling of dynamical systems.
On the emergence of numerical instabilities in Next Generation Reservoir Computing
Department of Electrical and Computer Engineering, Clarkson University, Potsdam, 13699, New York, U.S.A.（电气与计算机工程系，克林顿大学）；Clarkson Center for Complex Systems Science, Clarkson University, Potsdam, 13699, New York, U.S.A.（复杂系统科学中心，克林顿大学）
Sparse Federated Echo State Networks for Time-Series Classification via Smooth-L1 Regularization
Faculty of Computer Science and Business Information Systems（计算机科学与商业信息系统学院）；Technical University of Applied Sciences Würzburg-Schweinfurt（维尔茨堡-施韦因富特应用技术大学）
Learning with Imbalanced Noisy Data by Preventing Bias in Sample Selection
School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；Terminus Group（特步集团）；School of Electronic and Information Engineering, Tongji University（同济大学电子与信息工程学院）
Data-Augmented Deep Learning for Downhole Depth Sensing and Validation
State Key Laboratory of Electronic Thin Films and Integrated Devices, University of Electronic Science and Technology of China（电子薄膜与集成器件国家重点实验室，电子科学与技术大学）；Southwest Branch of China National Petroleum Corporation Logging Co., Ltd.（中国石油天然气集团有限公司西南分公司）；School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电子与电气工程学院）
