We consider the problem of estimating a linear time-invariant (LTI) dynamical system from a single trajectory via streaming algorithms, which is encountered in several applications including reinforcement learning (RL) and time-series analysis. While the LTI system estimation problem is well-studied in the {\em offline} setting, the practically important streaming/online setting has received little attention. Standard streaming methods like stochastic gradient descent (SGD) are unlikely to work since streaming points can be highly correlated. In this work, we propose a novel streaming algorithm, SGD with Reverse Experience Replay ($\mathsf{SGD}-\mathsf{RER}$), that is inspired by the experience replay (ER) technique popular in the RL literature. $\mathsf{SGD}-\mathsf{RER}$ divides data into small buffers and runs SGD backwards on the data stored in the individual buffers. We show that this algorithm exactly deconstructs the dependency structure and obtains information theoretically optimal guarantees for both parameter error and prediction error. Thus, we provide the first -- to the best of our knowledge -- optimal SGD-style algorithm for the classical problem of linear system identification with a first order oracle. Furthermore, $\mathsf{SGD}-\mathsf{RER}$ can be applied to more general settings like sparse LTI identification with known sparsity pattern, and non-linear dynamical systems. Our work demonstrates that the knowledge of data dependency structure can aid us in designing statistically and computationally efficient algorithms which can "decorrelate" streaming samples.
Continual Learning with Diffusion-based Generative Replay for Industrial Streaming Data
Shien-Ming Wu School of Intelligent Engineering, South China University of Technology（华南理工大学吴贤铭智能工程学院）；Pazhou Lab（琶洲实验室）；China Academy of Information and Communications Technology（中国信息通信研究院）
A Unified Replay-based Continuous Learning Framework for Spatio-Temporal Prediction on Streaming Data
Aalborg University（奥尔堡大学）；East China Normal University（华东师范大学）；University of Electronic Science and Technology of China（电子科技大学）；Huawei Cloud Database Innovation Lab（华为云数据库创新实验室）
