The vast quantity of information brought by big data as well as the evolving computer hardware encourages success stories in the machine learning community. In the meanwhile, it poses challenges for the Gaussian process (GP) regression, a well-known non-parametric and interpretable Bayesian model, which suffers from cubic complexity to data size. To improve the scalability while retaining desirable prediction quality, a variety of scalable GPs have been presented. But they have not yet been comprehensively reviewed and analyzed in order to be well understood by both academia and industry. The review of scalable GPs in the GP community is timely and important due to the explosion of data size. To this end, this paper is devoted to the review on state-of-the-art scalable GPs involving two main categories: global approximations which distillate the entire data and local approximations which divide the data for subspace learning. Particularly, for global approximations, we mainly focus on sparse approximations comprising prior approximations which modify the prior but perform exact inference, posterior approximations which retain exact prior but perform approximate inference, and structured sparse approximations which exploit specific structures in kernel matrix; for local approximations, we highlight the mixture/product of experts that conducts model averaging from multiple local experts to boost predictions. To present a complete review, recent advances for improving the scalability and capability of scalable GPs are reviewed. Finally, the extensions and open issues regarding the implementation of scalable GPs in various scenarios are reviewed and discussed to inspire novel ideas for future research avenues.
Ensemble Kalman Filtering Meets Gaussian Process SSM for Non-Mean-Field and Online Inference
School of Science & Engineering, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）理工学院）；Future Network of Intelligence Institute（未来智能网络研究院）；Shenzhen Research Institute of Big Data（深圳大数据研究院）；National University of Singapore（新加坡国立大学）
Local Gaussian Processes for Efficient Fine-Grained Traffic Speed Prediction
School of Information Systems, Singapore Management University（新加坡管理大学信息系统学院）；Smeal College of Business, Pennsylvania State University（宾夕法尼亚州立大学斯米尔商学院）
Recent Advances in Data-Driven Wireless Communication Using Gaussian Processes: A Comprehensive Survey
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；University of Science and Technology of China（中国科学技术大学）；Alibaba Group（阿里巴巴集团）；Shenzhen Research Institute of Big Data（深圳大数据研究院）；China United Network Communications Corporation（中国联合网络通信集团有限公司）
Parallel Gaussian Process Regression for Big Data: Low-Rank Representation Meets Markov Approximation
National University of Singapore（新加坡国立大学）；Singapore-MIT Alliance for Research and Technology（新加坡-麻省理工学院研究与技术联盟）；Massachusetts Institute of Technology（麻省理工学院）
Big Data Meet Cyber-Physical Systems: A Panoramic Survey
Texas A&M University at Qatar（德克萨斯农工大学卡塔尔分校）；Virginia Tech（弗吉尼亚理工大学）；Universidad de Chile（智利大学）；Nanjing University of Science & Technology（南京理工大学）；InterDigital Communications, LLC.（InterDigital通信有限责任公司）
