Scaling-up Distributed Processing of Data Streams for Machine Learning
Target Corporation（塔吉特公司）；University of Michigan（密歇根大学）；Rutgers University–New Brunswick（罗格斯大学新伯朗士威分校）
Matthew Nokleby, Haroon Raja, Waheed U. Bajwa
备注：
45 pages, 9 figures; preprint of a journal paper published in Proceedings of the IEEE (Special Issue on Optimization for Data-driven Learning and Control)
发表信息：
Proc. of the IEEE, vol. 108, no. 11, pp. 1984-2012, Nov. 2020
Emerging applications of machine learning in numerous areas involve continuous gathering of and learning from streams of data. Real-time incorporation of streaming data into the learned models is essential for improved inference in these applications. Further, these applications often involve data that are either inherently gathered at geographically distributed entities or that are intentionally distributed across multiple machines for memory, computational, and/or privacy reasons. Training of models in this distributed, streaming setting requires solving stochastic optimization problems in a collaborative manner over communication links between the physical entities. When the streaming data rate is high compared to the processing capabilities of compute nodes and/or the rate of the communications links, this poses a challenging question: how can one best leverage the incoming data for distributed training under constraints on computing capabilities and/or communications rate? A large body of research has emerged in recent decades to tackle this and related problems. This paper reviews recently developed methods that focus on large-scale distributed stochastic optimization in the compute- and bandwidth-limited regime, with an emphasis on convergence analysis that explicitly accounts for the mismatch between computation, communication and streaming rates. In particular, it focuses on methods that solve: (i) distributed stochastic convex problems, and (ii) distributed principal component analysis, which is a nonconvex problem with geometric structure that permits global convergence. For such methods, the paper discusses recent advances in terms of distributed algorithmic designs when faced with high-rate streaming data. Further, it reviews guarantees underlying these methods, which show there exist regimes in which systems can learn from distributed, streaming data at order-optimal rates.
Distributed Learning in the Non-Convex World: From Batch to Streaming Data, and Beyond
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；University of Minnesota（明尼苏达大学）；The Chinese University of Hong Kong（香港中文大学）；IBM Thomas J. Watson Research Center（IBM托马斯·J·沃森研究中心）
FiLark: a streaming-first software framework for end-to-end exploration, annotation, and algorithm integration in distributed acoustic sensing
organization= State Key Laboratory of Ocean Sensing \& Ocean College, Zhejiang University , city= Zhoushan , postcode= 316021 , country= China；organization= College of Information Science；Electronic Engineering, Zhejiang University , city= Hangzhou , country= China；organization= College of Computer Science；Technology, Zhejiang University , city= Hangzhou , country= China
Cross Layer Optimization and Distributed Reinforcement Learning for Wireless 360° Video Streaming
King Fahd University of Petroleum and Minerals (KFUPM)（法赫德国王石油与矿产大学）；Telefónica Research（西班牙电信研究院）；University of the Basque Country (UPV/EHU)（巴斯克大学）；University of Oulu（奥卢大学）；Purdue University（普渡大学）
