We study the problem of efficiently scaling ensemble-based deep neural networks for multi-step time series (TS) forecasting on a large set of time series. Current state-of-the-art deep ensemble models have high memory and computational requirements, hampering their use to forecast millions of TS in practical scenarios. We propose N-BEATS(P), a global parallel variant of the N-BEATS model designed to allow simultaneous training of multiple univariate TS forecasting models. Our model addresses the practical limitations of related models, reducing the training time by half and memory requirement by a factor of 5, while keeping the same level of accuracy in all TS forecasting settings. We have performed multiple experiments detailing the various ways to train our model and have obtained results that demonstrate its capacity to generalize in various forecasting conditions and setups.
In this work, we propose an ensemble forecasting approach based on randomized neural networks. Improved randomized learning streamlines the fitting abilities of individual learners by generating network parameters in accordance with the data and target function features. A pattern-based representation of time series makes the proposed approach suitable for forecasting time series with multiple seasonality. We propose six strategies for controlling the diversity of ensemble members. Case studies conducted on four real-world forecasting problems verified the effectiveness and superior performanc…
Pontifical Catholic University of Parana（巴拉那天主教大学）；Federal Technological University of Parana（巴拉那联邦理工大学）；Federal University of Parana（巴拉那联邦大学）
Accurate forecasting is important for decision-makers. Recently, the Amazon rainforest is reaching record levels of the number of fires, a situation that concerns both climate and public health problems. Obtaining the desired forecasting accuracy becomes difficult and challenging. In this paper were developed a novel heterogeneous decomposition-ensemble model by using Seasonal and Trend decomposition based on Loess in combination with algorithms for short-term load forecasting multi-month-ahead, to explore temporal patterns of Amazon rainforest fires in Brazil. The results demonstrate the prop…
Monash University（莫纳什大学）；University of Melbourne（墨尔本大学）；Uber Technologies Inc（优步技术公司）
With large quantities of data typically available nowadays, forecasting models that are trained across sets of time series, known as Global Forecasting Models (GFM), are regularly outperforming traditional univariate forecasting models that work on isolated series. As GFMs usually share the same set of parameters across all time series, they often have the problem of not being localised enough to a particular series, especially in situations where datasets are heterogeneous. We study how ensembling techniques can be used with generic GFMs and univariate models to solve this issue. Our work sys…
Kaunas University of Technology（考纳斯理工大学）；Rivile（Rivile私人有限责任公司）
Amounts of historical data collected increase and business intelligence applicability with automatic forecasting of time series are in high demand. While no single time series modeling method is universal to all types of dynamics, forecasting using an ensemble of several methods is often seen as a compromise. Instead of fixing ensemble diversity and size, we propose to predict these aspects adaptively using meta-learning. Meta-learning here considers two separate random forest regression models, built on 390 time-series features, to rank 22 univariate forecasting methods and recommend ensemble…
The contribution of this work is twofold: (1) We introduce a collection of ensemble methods for time series forecasting to combine predictions from base models. We demonstrate insights on the power of ensemble learning for forecasting, showing experiment results on about 16000 openly available datasets, from M4, M5, M3 competitions, as well as FRED (Federal Reserve Economic Data) datasets. Whereas experiments show that ensembles provide a benefit on forecasting results, there is no clear winning ensemble strategy (plus hyperparameter configuration). Thus, in addition, (2), we propose a meta-le…
Częstochowa University of Technology（琴斯托霍瓦工业大学）；Facebook（脸书）
Time series forecasting is a challenging problem particularly when a time series expresses multiple seasonality, nonlinear trend and varying variance. In this work, to forecast complex time series, we propose ensemble learning which is based on randomized neural networks, and boosted in three ways. These comprise ensemble learning based on residuals, corrected targets and opposed response. The latter two methods are employed to ensure similar forecasting tasks are solved by all ensemble members, which justifies the use of exactly the same base models at all stages of ensembling. Unification of…
This work presents ensemble forecasting of monthly electricity demand using pattern similarity-based forecasting methods (PSFMs). PSFMs applied in this study include $k$-nearest neighbor model, fuzzy neighborhood model, kernel regression model, and general regression neural network. An integral part of PSFMs is a time series representation using patterns of time series sequences. Pattern representation ensures the input and output data unification through filtering a trend and equalizing variance. Two types of ensembles are created: heterogeneous and homogeneous. The former consists of differe…
Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；School of Mathematical Sciences, University of Chinese Academy of Sciences（中国科学院大学数学科学学院）；School of Management, Xi'an Jiaotong University（西安交通大学管理学院）
A new variational mode decomposition (VMD) based deep learning approach is proposed in this paper for time series forecasting problem. Firstly, VMD is adopted to decompose the original time series into several sub-signals. Then, a convolutional neural network (CNN) is applied to learn the reconstruction patterns on the decomposed sub-signals to obtain several reconstructed sub-signals. Finally, a long short term memory (LSTM) network is employed to forecast the time series with the decomposed sub-signals and the reconstructed sub-signals as inputs. The proposed VMD-CNN-LSTM approach is origina…
A Synergistic Approach: Dynamics-AI Ensemble in Tropical Cyclone Forecasting
Institute of Atmospheric Physics, Chinese Academy of Sciences（中国科学院大气物理研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Shanghai Academy of Artificial Intelligence for Science（上海科学智能研究院）；Fudan University（复旦大学）；Henan Meteorological Observatory（河南省气象台）；Tianjin Meteorological Information Center（天津市气象信息中心）
UiT The Arctic University of Norway（挪威北极大学）；Kongsberg Satellite Services（康斯伯格卫星服务公司）；NORCE Norwegian Research Centre（挪威NORCE研究中心）
This paper presents a novel probabilistic forecasting method called ensemble conformalized quantile regression (EnCQR). EnCQR constructs distribution-free and approximately marginally valid prediction intervals (PIs), which are suitable for nonstationary and heteroscedastic time series data. EnCQR can be applied on top of a generic forecasting model, including deep learning architectures. EnCQR exploits a bootstrap ensemble estimator, which enables the use of conformal predictors for time series by removing the requirement of data exchangeability. The ensemble learners are implemented as gener…
University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）
Accurately forecasting Arctic sea ice from subseasonal to seasonal scales has been a major scientific effort with fundamental challenges at play. In addition to physics-based earth system models, researchers have been applying multiple statistical and machine learning models for sea ice forecasting. Looking at the potential of data-driven sea ice forecasting, we propose an attention-based Long Short Term Memory (LSTM) ensemble method to predict monthly sea ice extent up to 1 month ahead. Using daily and monthly satellite retrieved sea ice data from NSIDC and atmospheric and oceanic variables f…
Enhancing the robustness and accuracy of time series forecasting models is an active area of research. Recently, Artificial Neural Networks (ANNs) have found extensive applications in many practical forecasting problems. However, the standard backpropagation ANN training algorithm has some critical issues, e.g. it has a slow convergence rate and often converges to a local minimum, the complex pattern of error surfaces, lack of proper training parameters selection methods, etc. To overcome these drawbacks, various improved training methods have been developed in literature; but, still none of t…
National Technical University of Athens（雅典国家技术大学）；Air Force Support Command, Hellenic Air Force（希腊空军支援司令部）；University of Patras（帕特雷大学）
Daily streamflow forecasting through data-driven approaches is traditionally performed using a single machine learning algorithm. Existing applications are mostly restricted to examination of few case studies, not allowing accurate assessment of the predictive performance of the algorithms involved. Here we propose super learning (a type of ensemble learning) by combining 10 machine learning algorithms. We apply the proposed algorithm in one-step ahead forecasting mode. For the application, we exploit a big dataset consisting of 10-year long time series of daily streamflow, precipitation and t…
University of Macau（澳门大学）；Victoria University（维多利亚大学）
Although transfer learning is proven to be effective in computer vision and natural language processing applications, it is rarely investigated in forecasting financial time series. Majority of existing works on transfer learning are based on single-source transfer learning due to the availability of open-access large-scale datasets. However, in financial domain, the lengths of individual time series are relatively short and single-source transfer learning models are less effective. Therefore, in this paper, we investigate multi-source deep transfer learning for financial time series. We propo…
Seasonal influenza infects between 10 and 50 million people in the United States every year, overburdening hospitals during weeks of peak incidence. Named by the CDC as an important tool to fight the damaging effects of these epidemics, accurate forecasts of influenza and influenza-like illness (ILI) forewarn public health officials about when, and where, seasonal influenza outbreaks will hit hardest. Multi-model ensemble forecasts---weighted combinations of component models---have shown positive results in forecasting. Ensemble forecasts of influenza outbreaks have been static, training on al…
Time Series Forecasting is at the core of many practical applications such as sales forecasting for business, rainfall forecasting for agriculture and many others. Though this problem has been extensively studied for years, it is still considered a challenging problem due to complex and evolving nature of time series data. Typical methods proposed for time series forecasting modeled linear or non-linear dependencies between data observations. However it is a generally accepted notion that no one method is universally effective for all kinds of time series data. Attempts have been made to use d…
University of North Carolina at Charlotte（北卡罗来纳大学夏洛特分校）
To mitigate the uncertainty of variable renewable resources, two off-the-shelf machine learning tools are deployed to forecast the solar power output of a solar photovoltaic system. The support vector machines generate the forecasts and the random forest acts as an ensemble learning method to combine the forecasts. The common ensemble technique in wind and solar power forecasting is the blending of meteorological data from several sources. In this study though, the present and the past solar power forecasts from several models, as well as the associated meteorological data, are incorporated in…
Norwegian University of Science and Technology（挪威科技大学）
This paper presents an ensemble forecasting method that shows strong results on the M4 Competition dataset by decreasing feature and model selection assumptions, termed DONUT (DO Not UTilize human beliefs). Our assumption reductions, primarily consisting of auto-generated features and a more diverse model pool for the ensemble, significantly outperform the statistical, feature-based ensemble method FFORMA by Montero-Manso et al. (2020). We also investigate feature extraction with a Long Short-term Memory Network (LSTM) Autoencoder and find that such features contain crucial information not cap…
Ericsson Global Services（爱立信全球服务）；Indian Institute of Technology - Madras（印度理工学院马德拉斯分校）；Indian Institute of Petroleum and Energy（印度石油与能源学院）；Honeywell Connected Enterprise（霍尼韦尔互联企业）
Ensemble models are powerful model building tools that are developed with a focus to improve the accuracy of model predictions. They find applications in time series forecasting in varied scenarios including but not limited to process industries, health care, and economics where a single model might not provide optimal performance. It is known that if models selected for data modelling are distinct (linear/non-linear, static/dynamic) and independent (minimally correlated models), the accuracy of the predictions is improved. Various approaches suggested in the literature to weigh the ensemble m…
Adaptive Rainfall Forecasting from Multiple Geographical Models Using Matrix Profile and Ensemble Learning
Ha Noi University of Science and Technology（河内科学技术大学）；Center of Environmental Intelligence（环境智能中心）；VinUniversity（文大学）；University of Southern California（南加州大学）
With COVID-19 affecting every country globally and changing everyday life, the ability to forecast the spread of the disease is more important than any previous epidemic. The conventional methods of disease-spread modeling, compartmental models, are based on the assumption of spatiotemporal homogeneity of the spread of the virus, which may cause forecasting to underperform, especially at high spatial resolutions. In this paper we approach the forecasting task with an alternative technique - spatiotemporal machine learning. We present COVID-LSTM, a data-driven model based on a Long Short-term M…
University of Washington（华盛顿大学）；Pacific Northwest National Laboratory（太平洋西北国家实验室）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Time series forecasting remains a central challenge problem in almost all scientific disciplines. We introduce a novel load forecasting method in which observed dynamics are modeled as a forced linear system using Dynamic Mode Decomposition (DMD) in time delay coordinates. Central to this approach is the insight that grid load, like many observables on complex real-world systems, has an "almost-periodic" character, i.e., a continuous Fourier spectrum punctuated by dominant peaks, which capture regular (e.g., daily or weekly) recurrences in the dynamics. The forecasting method presented takes a…
Time series forecasting is difficult. It is difficult even for recurrent neural networks with their inherent ability to learn sequentiality. This article presents a recurrent neural network based time series forecasting framework covering feature engineering, feature importances, point and interval predictions, and forecast evaluation. The description of the method is followed by an empirical study using both LSTM and GRU networks.
Many applications require the ability to judge uncertainty of time-series forecasts. Uncertainty is often specified as point-wise error bars around a mean or median forecast. Due to temporal dependencies, such a method obscures some information. We would ideally have a way to query the posterior probability of the entire time-series given the predictive variables, or at a minimum, be able to draw samples from this distribution. We use a Bayesian dictionary learning algorithm to statistically generate an ensemble of forecasts. We show that the algorithm performs as well as a physics-based ensem…
QGAPHEnsemble : Combining Hybrid QLSTM Network Ensemble via Adaptive Weighting for Short Term Weather Forecasting
Georgia Institute of Technology（佐治亚理工学院）；Indian Institute of Engineering Science and Technology, Shibpur（印度工程科学技术学院（希布普尔））；IIT Roorkee（鲁基印度理工学院）；National Institute of Technology, Rourkela（罗尔克拉国家理工学院）；National Institute of Technology Karnataka（卡纳塔克邦国家理工学院）；Jadavpur University（贾达沃普尔大学）
FuXi-ENS: A machine learning model for medium-range ensemble weather forecasting
Artificial Intelligence Innovation and Incubation Institute, Fudan University（复旦大学人工智能创新与孵化研究院）；Shanghai Academy of Artificial Intelligence for Science（上海科学人工智能研究院）；National Meteorological Information Center, China Meteorological Administration（中国气象局国家气象信息中心）
Recent studies concerning the point electricity price forecasting have shown evidence that the hourly German Intraday Continuous Market is weak-form efficient. Therefore, we take a novel, advanced approach to the problem. A probabilistic forecasting of the hourly intraday electricity prices is performed by simulating trajectories in every trading window to receive a realistic ensemble to allow for more efficient intraday trading and redispatch. A generalized additive model is fitted to the price differences with the assumption that they follow a zero-inflated distribution, precisely a mixture …
This article presents a new method for forecasting Value at Risk. Convolutional neural networks can do time series forecasting, since they can learn local patterns in time. A simple modification enables them to forecast not the mean, but arbitrary quantiles of the distribution, and thus allows them to be applied to VaR-forecasting. The proposed model can learn from the price history of different assets, and it seems to produce fairly accurate forecasts.
