Distributed computing systems often consist of hundreds of nodes, executing tasks with different resource requirements. Efficient resource provisioning and task scheduling in such systems are non-trivial and require close monitoring and accurate forecasting of the state of the system, specifically resource utilisation at its constituent machines. Two challenges present themselves towards these objectives. First, collecting monitoring data entails substantial communication overhead. This overhead can be prohibitively high, especially in networks where bandwidth is limited. Second, forecasting models to predict resource utilisation should be accurate and need to exhibit high inference speed. Mission critical scheduling and resource allocation algorithms use these predictions and rely on their immediate availability. To address the first challenge, we present a communication-efficient data collection mechanism. Resource utilisation data is collected at the individual machines in the system and transmitted to a central controller in batches. Each batch is processed by an adaptive data-reduction algorithm based on Fourier transforms and truncation in the frequency domain. We show that the proposed mechanism leads to a significant reduction in communication overhead while incurring only minimal error and adhering to accuracy guarantees. To address the second challenge, we propose a deep learning architecture using complex Gated Recurrent Units to forecast resource utilisation. This architecture is directly integrated with the above data collection mechanism to improve inference speed of our forecasting model. Using two real-world datasets, we demonstrate the effectiveness of our approach, both in terms of forecasting accuracy and inference speed. Our approach resolves challenges encountered in resource provisioning frameworks and can be applied to other forecasting problems.
Seismic monitoring of carbon storage sequestration is a challenging problem involving both fluid-flow physics and wave physics. Additionally, monitoring usually requires the solvers for these physics to be coupled and differentiable to effectively invert for the subsurface properties of interest. To drastically reduce the computational cost, we introduce a learned coupled inversion framework based on the wave modeling operator, rock property conversion and a proxy fluid-flow simulator. We show that we can accurately use a Fourier neural operator as a proxy for the fluid-flow simulator for a fr…
FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspective
Beijing Institute of Technology（北京理工大学）；Tongji University（同济大学）；University of Oxford（牛津大学）；University of Macau（澳门大学）；HeFei University of Technology（合肥工业大学）；Macquarie University（麦考瑞大学）
Beijing Institute of Technology（北京理工大学）；University of Technology Sydney（悉尼科技大学）；Tongji University（同济大学）；HeFei University of Technology（合肥工业大学）
The key problem in multivariate time series (MTS) analysis and forecasting aims to disclose the underlying couplings between variables that drive the co-movements. Considerable recent successful MTS methods are built with graph neural networks (GNNs) due to their essential capacity for relational modeling. However, previous work often used a static graph structure of time-series variables for modeling MTS failing to capture their ever-changing correlations over time. To this end, a fully-connected supra-graph connecting any two variables at any two timestamps is adaptively learned to capture t…
NVIDIA Corporation（英伟达公司）；University of Michigan（密歇根大学）；Rice University（莱斯大学）；California Institute of Technology（加州理工学院）；Purdue University（普渡大学）
FourCastNet, short for Fourier Forecasting Neural Network, is a global data-driven weather forecasting model that provides accurate short to medium-range global predictions at $0.25^{\circ}$ resolution. FourCastNet accurately forecasts high-resolution, fast-timescale variables such as the surface wind speed, precipitation, and atmospheric water vapor. It has important implications for planning wind energy resources, predicting extreme weather events such as tropical cyclones, extra-tropical cyclones, and atmospheric rivers. FourCastNet matches the forecasting accuracy of the ECMWF Integrated F…
Fourier Basis Mapping: A Time-Frequency Learning Framework for Time Series Forecasting
School of Computing, Macquarie University（麦考瑞大学计算机学院）；Department of Automation, Shanghai Jiao Tong University（上海交通大学自动化系）；department of Electrical and Electronic Engineering, Hong Kong Polytechnic University（香港理工大学电子与电气工程系）
Quantum Fourier Transform Based Kernel for Solar Irrandiance Forecasting
Ecole Polytechnique Fédérale de Lausanne (EPFL)（瑞士联邦理工学院）；Mechanical Engineering Department, Universidad de Chile（智利大学机械工程系）；Departamento de Ingeniería Mecánica y Metalúrgica, Escuela de Ingeniería, Pontificia Universidad Católica de Chile（智利天主教大学机械工程与冶金系）；Garrick Institute for the Risk Sciences, University of California（加州大学风险科学研究所）
Division of Engineering and Applied Science, California Institute of Technology, Pasadena, CA 91125, USA（美国加州理工学院工程与应用科学系）
We apply Fourier neural operators (FNOs), a state-of-the-art operator learning technique, to forecast the temporal evolution of experimentally measured velocity fields. FNOs are a recently developed machine learning method capable of approximating solution operators to systems of partial differential equations through data alone. The learned FNO solution operator can be evaluated in milliseconds, potentially enabling faster-than-real-time modeling for predictive flow control in physical systems. Here we use FNOs to predict how physical fluid flows evolve in time, training with particle image v…
MoFE: A Novel Mixture-of-Experts Framework with Fourier Neural Operators for Cryptocurrency Forecasting
School of Art and Science University of Rochester（罗切斯特大学艺术与科学学院）；AGI Lab Beijing Institute of Mathematical Sciences and Applications（北京数学科学与应用研究院AGI实验室）
Multivariate Long-term Time Series Forecasting with Fourier Neural Filter
Department of Statistics & Data Science, UCLA（统计与数据科学系，加州大学洛杉矶分校）；School of Psychological and Cognitive Sciences, Peking University（心理与认知科学学院，北京大学）；Institute for Artificial Intelligence, Peking University（人工智能研究院，北京大学）；Beijing Key Laboratory of Behavior and Mental Health, Peking University（北京行为与心理健康重点实验室，北京大学）
Extreme weather amplified by climate change is causing increasingly devastating impacts across the globe. The current use of physics-based numerical weather prediction (NWP) limits accuracy due to high computational cost and strict time-to-solution limits. We report that a data-driven deep learning Earth system emulator, FourCastNet, can predict global weather and generate medium-range forecasts five orders-of-magnitude faster than NWP while approaching state-of-the-art accuracy. FourCast-Net is optimized and scales efficiently on three supercomputing systems: Selene, Perlmutter, and JUWELS Bo…
UC San Diego（加州大学圣地亚哥分校）；AWS AI Labs（亚马逊云科技人工智能实验室）；AWS（亚马逊云科技）
Transformer-based models have gained large popularity and demonstrated promising results in long-term time-series forecasting in recent years. In addition to learning attention in time domain, recent works also explore learning attention in frequency domains (e.g., Fourier domain, wavelet domain), given that seasonal patterns can be better captured in these domains. In this work, we seek to understand the relationships between attention models in different time and frequency domains. Theoretically, we show that attention models in different domains are equivalent under linear conditions (i.e.,…
Seoul National University（首尔大学）；Korea Aerospace University（韩国航空大学）；Korea Advanced Institute of Science and Technology（韩国科学技术院）
Multiple seasonal patterns play a key role in time series forecasting, especially for business time series where seasonal effects are often dramatic. Previous approaches including Fourier decomposition, exponential smoothing, and seasonal autoregressive integrated moving average (SARIMA) models do not reflect the distinct characteristics of each period in seasonal patterns. We propose a mixed hierarchical seasonality (MHS) model. Intermediate parameters for each seasonal period are first estimated, and a mixture of intermediate parameters is taken. This results in a model that automatically le…
Washington University in St. Louis（圣路易斯华盛顿大学）；NVIDIA（英伟达）
Data-driven models, such as FourCastNet (FCN), have shown exemplary performance in high-resolution global weather forecasting. This performance, however, is based on supervision on mesh-gridded weather data without the utilization of raw climate observational data, the gold standard ground truth. In this work we develop a methodology to correct, remap, and fine-tune gridded uniform forecasts of FCN so it can be directly compared against observational ground truth, which is sparse and non-uniform in space and time. This is akin to bias correction and post-processing of numerical weather predict…
AWEMixer: Adaptive Wavelet-Enhanced Mixer Network for Long-Term Time Series Forecasting
School of Computer Science and Technology, Xi’an Jiaotong University（西安交通大学计算机科学与技术学院）；Department of Research and Development, Shandong New Beiyang Information Technology Co., Ltd.（山东新北洋信息技术有限公司）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术学院）
AR-KAN: Autoregressive-Weight-Enhanced Kolmogorov-Arnold Network for Time Series Forecasting
School of Information Science and Engineering, Southeast University（东南大学信息科学与工程学院）；School of Economics and Management, Southeast University（东南大学经济管理学院）
Streamlining Ocean Dynamics Modeling with Fourier Neural Operators: A Multiobjective Hyperparameter and Architecture Optimization Approach
Argonne National Laboratory（阿贡国家实验室）；Florida State University（佛罗里达州立大学）；Université Paris-Saclay（巴黎萨克雷大学）；Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）；Oak Ridge National Laboratory（橡树岭国家实验室）
Time-series forecasting is an important task in both academic and industry, which can be applied to solve many real forecasting problems like stock, water-supply, and sales predictions. In this paper, we study the case of retailers' sales forecasting on Tmall|the world's leading online B2C platform. By analyzing the data, we have two main observations, i.e., sales seasonality after we group different groups of retails and a Tweedie distribution after we transform the sales (target to forecast). Based on our observations, we design two mechanisms for sales forecasting, i.e., seasonality extract…
Huge Ensembles Part II: Properties of a Huge Ensemble of Hindcasts Generated with Spherical Fourier Neural Operators
Lawrence Berkeley National Laboratory (LBNL)（劳伦斯伯克利国家实验室）；University of California, Berkeley（加州大学伯克利分校）；NVIDIA Corporation（英伟达公司）；National Energy Research Scientific Computing Center (NERSC)（国家能源研究科学计算中心）；Indiana University（印第安纳大学）；University of California, Irvine（加州大学欧文分校）
University of Washington（华盛顿大学）；Pacific Northwest National Laboratory（太平洋西北国家实验室）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Time series forecasting remains a central challenge problem in almost all scientific disciplines. We introduce a novel load forecasting method in which observed dynamics are modeled as a forced linear system using Dynamic Mode Decomposition (DMD) in time delay coordinates. Central to this approach is the insight that grid load, like many observables on complex real-world systems, has an "almost-periodic" character, i.e., a continuous Fourier spectrum punctuated by dominant peaks, which capture regular (e.g., daily or weekly) recurrences in the dynamics. The forecasting method presented takes a…
Huge Ensembles Part I: Design of Ensemble Weather Forecasts using Spherical Fourier Neural Operators
Lawrence Berkeley National Laboratory (LBNL)（劳伦斯伯克利国家实验室）；University of California, Berkeley（加州大学伯克利分校）；NVIDIA Corporation（英伟达公司）；Indiana University（印第安纳大学）；National Energy Research Scientific Computing Center (NERSC)（国家能源研究科学计算中心）；University of California, Irvine（加州大学欧文分校）
Deep Frequency Derivative Learning for Non-stationary Time Series Forecasting
University of Oxford（牛津大学）；Beijing Institute of Technology（北京理工大学）；Jilin University（吉林大学）；Chinese Academy of Science（中国科学院）；Tongji University（同济大学）；Hefei University of Technology（合肥工业大学）
Recent studies have shown that deep learning models such as RNNs and Transformers have brought significant performance gains for long-term forecasting of time series because they effectively utilize historical information. We found, however, that there is still great room for improvement in how to preserve historical information in neural networks while avoiding overfitting to noise presented in the history. Addressing this allows better utilization of the capabilities of deep learning models. To this end, we design a \textbf{F}requency \textbf{i}mproved \textbf{L}egendre \textbf{M}emory model…
Although Transformer-based methods have significantly improved state-of-the-art results for long-term series forecasting, they are not only computationally expensive but more importantly, are unable to capture the global view of time series (e.g. overall trend). To address these problems, we propose to combine Transformer with the seasonal-trend decomposition method, in which the decomposition method captures the global profile of time series while Transformers capture more detailed structures. To further enhance the performance of Transformer for long-term prediction, we exploit the fact that…
StFT: Spatio-temporal Fourier Transformer for Long-term Dynamics Prediction
School of Computing University of Utah（计算学院犹他大学）；AMCR Division Lawrence Berkeley National Laboratory（伯克利国家实验室AMCR部门）；Applied Mathematics and Computational Research Division Lawrence Berkeley National Laboratory（伯克利国家实验室应用数学与计算研究部门）
OLMA: One Loss for More Accurate Time Series Forecasting
Beijing University of Posts and Telecommunications（北京邮电大学）；Beijing Key Laboratory of Network System and Network Culture（北京网络系统与网络文化重点实验室）；Beijing Academy of Blockchain and Edge Computing（北京区块链与边缘计算研究院）
Diffusion-corrected Autoregressive Fourier Neural Operator for Droplet Evolution Prediction
New Jersey Institute of Technology（新泽西理工学院）；University of Georgia（佐治亚大学）；University at Buffalo（纽约州立大学布法罗分校）；Hanyang University（汉阳大学）；Northeastern University（东北大学）
