Forecast Network-Wide Traffic States for Multiple Steps Ahead: A Deep Learning Approach Considering Dynamic Non-Local Spatial Correlation and Non-Stationary Temporal Dependency
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University（武汉大学测绘遥感信息工程国家重点实验室）
Xinglei Wang, Xuefeng Guan, Jun Cao, Na Zhang, Huayi Wu
备注：
29 pages, 16 figures, 3 tables
发表信息：
Transportation research part C: emerging technologies. 119 (2020): 102763
Obtaining accurate information about future traffic flows of all links in a traffic network is of great importance for traffic management and control applications. This research studies two particular problems in traffic forecasting: (1) capture the dynamic and non-local spatial correlation between traffic links and (2) model the dynamics of temporal dependency for accurate multiple steps ahead predictions. To address these issues, we propose a deep learning framework named Spatial-Temporal Sequence to Sequence model (STSeq2Seq). This model builds on sequence to sequence (seq2seq) architecture to capture temporal feature and relies on graph convolution for aggregating spatial information. Moreover, STSeq2Seq defines and constructs pattern-aware adjacency matrices (PAMs) based on pair-wise similarity of the recent traffic patterns on traffic links and integrate it into graph convolution operation. It also deploys a novel seq2sesq architecture which couples a convolutional encoder and a recurrent decoder with attention mechanism for dynamic modeling of long-range dependence between different time steps. We conduct extensive experiments using two publicly-available large-scale traffic datasets and compare STSeq2Seq with other baseline models. The numerical results demonstrate that the proposed model achieves state-of-the-art forecasting performance in terms of various error measures. The ablation study verifies the effectiveness of PAMs in capturing dynamic non-local spatial correlation and the superiority of proposed seq2seq architecture in modeling non-stationary temporal dependency for multiple steps ahead prediction. Furthermore, qualitative analysis is conducted on PAMs as well as the attention weights for model interpretation.
Recurrent and convolutional neural networks are the most common architectures used for time series forecasting in deep learning literature. These networks use parameter sharing by repeating a set of fixed architectures with fixed parameters over time or space. The result is that the overall architecture is time-invariant (shift-invariant in the spatial domain) or stationary. We argue that time-invariance can reduce the capacity to perform multi-step-ahead forecasting, where modelling the dynamics at a range of scales and resolutions is required. We propose ForecastNet which uses a deep feed-fo…
Université Libre de Bruxelles（布鲁塞尔自由大学）；Cairo University（开罗大学）
Multi-step ahead forecasting is still an open challenge in time series forecasting. Several approaches that deal with this complex problem have been proposed in the literature but an extensive comparison on a large number of tasks is still missing. This paper aims to fill this gap by reviewing existing strategies for multi-step ahead forecasting and comparing them in theoretical and practical terms. To attain such an objective, we performed a large scale comparison of these different strategies using a large experimental benchmark (namely the 111 series from the NN5 forecasting competition). I…
House of Energy Markets and Finance（能源市场与金融研究院）；University of Duisburg-Essen（杜伊斯堡-埃森大学）
We present a winning method of the IEEE DataPort Competition on Day-Ahead Electricity Demand Forecasting: Post-COVID Paradigm. The day-ahead load forecasting approach is based on online forecast combination of multiple point prediction models. It contains four steps: i) data cleaning and preprocessing, ii) a holiday adjustment procedure, iii) training of individual forecasting models, iv) forecast combination by smoothed Bernstein Online Aggregation (BOA). The approach is flexible and can quickly adopt to new energy system situations as they occurred during and after COVID-19 shutdowns. The po…
Modeling price risks is crucial for economic decision making in energy markets. Besides the risk of a single price, the dependence structure of multiple prices is often relevant. We therefore propose a generic and easy-to-implement method for creating multivariate probabilistic forecasts based on univariate point forecasts of day-ahead electricity prices. While each univariate point forecast refers to one of the day's 24 hours, the multivariate forecast distribution models dependencies across hours. The proposed method is based on simple copula techniques and an optional time series component.…
Électricité de France R&D（法国电力公司研发部）；Viking Conseil（维京咨询公司）
We focus on day-ahead electricity load forecasting of substations of the distribution network in France; therefore, our problem lies between the instability of a single consumption and the stability of a countrywide total demand. Moreover, we are interested in forecasting the loads of over one thousand substations; consequently, we are in the context of forecasting multiple time series. To that end, we rely on an adaptive methodology that provided excellent results at a national scale; the idea is to combine generalized additive models with state-space representations. However, the extension o…
In the context of capacity planning, forecasting the evolution of informatics servers usage enables companies to better manage their computational resources. We address this problem by collecting key indicator time series and propose to forecast their evolution a day-ahead. Our method assumes that data is structured by a daily seasonality, but also that there is typical evolution of indicators within a day. Then, it uses the combination of a clustering algorithm and Markov Models to produce day-ahead forecasts. Our experiments on real datasets show that the data satisfies our assumption and th…
Accurate time series prediction over long future horizons is challenging and of great interest to both practitioners and academics. As a well-known intelligent algorithm, the standard formulation of Support Vector Regression (SVR) could be taken for multi-step-ahead time series prediction, only relying either on iterated strategy or direct strategy. This study proposes a novel multiple-step-ahead time series prediction approach which employs multiple-output support vector regression (M-SVR) with multiple-input multiple-output (MIMO) prediction strategy. In addition, the rank of three leading p…
National Technical University of Athens（雅典国家技术大学）；Hellenic Air Force（希腊空军）
Delivering useful hydrological forecasts is critical for urban and agricultural water management, hydropower generation, flood protection and management, drought mitigation and alleviation, and river basin planning and management, among others. In this work, we present and appraise a new simple and flexible methodology for hydrological time series forecasting. This methodology relies on (a) at least two individual forecasting methods and (b) the median combiner of forecasts. The appraisal is made by using a big dataset consisted of 90-year-long mean annual river flow time series from approxima…
Forecasting with Deep Learning: Beyond Average of Average of Average Performance
Faculdade de Engenharia da Universidade do Porto（波尔图大学工程学院）；Laboratory for Artificial Intelligence and Computer Science (LIACC)（人工智能与计算机科学实验室（LIACC））；Fraunhofer Portugal AICOS（弗劳恩霍夫葡萄牙AICOS）
We present the winning strategy of an electricity demand forecasting competition. This competition was organized to design new forecasting methods for unstable periods such as the one starting in Spring 2020. We rely on state-space models to adapt standard statistical and machine learning models. We claim that it achieves the right compromise between two extremes. On the one hand, purely time-series models such as autoregressives are adaptive in essence but fail to capture dependence to exogenous variables. On the other hand, machine learning methods allow to learn complex dependence to explan…
University of Southern California（南加州大学）；California Institute of Technology（加州理工学院）
Spatiotemporal forecasting has various applications in neuroscience, climate and transportation domain. Traffic forecasting is one canonical example of such learning task. The task is challenging due to (1) complex spatial dependency on road networks, (2) non-linear temporal dynamics with changing road conditions and (3) inherent difficulty of long-term forecasting. To address these challenges, we propose to model the traffic flow as a diffusion process on a directed graph and introduce Diffusion Convolutional Recurrent Neural Network (DCRNN), a deep learning framework for traffic forecasting …
Postprocessing of point predictions for probabilistic forecasting of day-ahead electricity prices: The benefits of using isotonic distributional regression
Water demand is a highly important variable for operational control and decision making. Hence, the development of accurate forecasts is a valuable field of research to further improve the efficiency of water utilities. Focusing on probabilistic multi-step-ahead forecasting, a time series model is introduced, to capture typical autoregressive, calendar and seasonal effects, to account for time-varying variance, and to quantify the uncertainty and path-dependency of the water demand process. To deal with the high complexity of the water demand process a high-dimensional feature space is applied…
A Hybrid Autoencoder-Transformer Model for Robust Day-Ahead Electricity Price Forecasting under Extreme Conditions
Beijing Normal-Hong Kong Baptist University（北京师范大学-香港 Baptist大学）；Shenzhen Research Institute of Big Data（深圳大数据研究院）；School of Science and Engineering（科学与工程学院）；School of Electrical and Power Engineering（电气与电力工程学院）
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）
For the prediction with experts' advice setting, we construct forecasting algorithms that suffer loss not much more than any expert in the pool. In contrast to the standard approach, we investigate the case of long-term forecasting of time series and consider two scenarios. In the first one, at each step $t$ the learner has to combine the point forecasts of the experts issued for the time interval $[t+1, t+d]$ ahead. Our approach implies that at each time step experts issue point forecasts for arbitrary many steps ahead and then the learner (algorithm) combines these forecasts and the forecast…
MATNet: Multi-Level Fusion Transformer-Based Model for Day-Ahead PV Generation Forecasting
Department of Naval, Electrical, Electronics；Telecommunications Engineering, University of Genoa, Via all’Opera Pia 11a, 16145 Genoa, Italy；Unit of Innovation, Entrepreneurship \& Sustainability, Department of Engineering, University Campus Bio-Medico of Rome Via Alvaro del Portillo 21, 00128 Rome, Italy；Computer Systems Department of Engineering, University Campus Bio-Medico of Rome Via Alvaro del Portillo 21, 00128 Rome, Italy
Emory University（埃默里大学）；Amazon Research（亚马逊研究院）；Zalando SE
We propose Multivariate Quantile Function Forecaster (MQF$^2$), a global probabilistic forecasting method constructed using a multivariate quantile function and investigate its application to multi-horizon forecasting. Prior approaches are either autoregressive, implicitly capturing the dependency structure across time but exhibiting error accumulation with increasing forecast horizons, or multi-horizon sequence-to-sequence models, which do not exhibit error accumulation, but also do typically not model the dependency structure across time steps. MQF$^2$ combines the benefits of both approache…
We examine the problem of modeling and forecasting European Day-Ahead and Month-Ahead natural gas prices. For this, we propose two distinct probabilistic models that can be utilized in risk- and portfolio management. We use daily pricing data ranging from 2011 to 2020. Extensive descriptive data analysis shows that both time series feature heavy tails, conditional heteroscedasticity, and show asymmetric behavior in their differences. We propose state-space time series models under skewed, heavy-tailed distributions to capture all stylized facts of the data. They include the impact of autocorre…
Birla Institute of Technology and Science Pilani（比拉理工学院与科学大学皮拉尼分校）
The problem of automatic and accurate forecasting of time-series data has always been an interesting challenge for the machine learning and forecasting community. A majority of the real-world time-series problems have non-stationary characteristics that make the understanding of trend and seasonality difficult. Our interest in this paper is to study the applicability of the popular deep neural networks (DNN) as function approximators for non-stationary TSF. We evaluate the following DNN models: Multi-layer Perceptron (MLP), Convolutional Neural Network (CNN), and RNN with Long-Short Term Memor…
When modeling sub-national mortality rates, it is important to incorporate any possible correlation among sub-populations to improve forecast accuracy. Moreover, forecasts at the sub-national level should aggregate consistently across the forecasts at the national level. In this study, we apply a grouped multivariate functional time series to forecast Australian regional and remote age-specific mortality rates and reconcile forecasts in a group structure using various methods. Our proposed method compares favorably to a grouped univariate functional time series forecasting method by comparing …
Suppose that we observe a short time series where each time-t-specific data-structure consists of many slightly dependent data indexed by a and that we want to estimate a feature of the law of the experiment that depends neither on t nor on a. We develop and study an algorithm to learn sequentially which base algorithm in a user-supplied collection best carries out the estimation task in terms of excess risk and oracular inequalities. The analysis, which uses dependency graph to model the amount of conditional independence within each t-specific data-structure and a concentration inequality by…
We present a regime-switching vector-autoregressive method for very-short-term wind speed forecasting at multiple locations with regimes based on large-scale meteorological phenomena. Statistical methods short-term wind forecasting out-perform numerical weather prediction for forecast horizons up to a few hours, and the spatio-temporal interdependency between geographically dispersed locations may be exploited to improve forecast skill. Here we show that conditioning spatio-temporal interdependency on `atmospheric modes' can further improve forecast performance. The modes are defined from the …
Wind speed forecasting has received a lot of attention in the recent past from researchers due to its enormous benefits in the generation of wind power and distribution. The biggest challenge still remains to be accurate prediction of wind speeds for efficient operation of a wind farm. Wind speed forecasts can be greatly improved by understanding its underlying dynamics. In this paper, we propose a method of time series partitioning where the original 10 minutes wind speed data is converted into a two-dimensional array of order (N x 144) where N denotes the number of days with 144 the daily 10…
When modeling sub-national mortality rates, we should consider three features: (1) how to incorporate any possible correlation among sub-populations to potentially improve forecast accuracy through multi-population joint modeling; (2) how to reconcile sub-national mortality forecasts so that they aggregate adequately across various levels of a group structure; (3) among the forecast reconciliation methods, how to combine their forecasts to achieve improved forecast accuracy. To address these issues, we introduce an extension of grouped univariate functional time series method. We first conside…
We present new Bayesian methodology for consumer sales forecasting. With a focus on multi-step ahead forecasting of daily sales of many supermarket items, we adapt dynamic count mixture models to forecast individual customer transactions, and introduce novel dynamic binary cascade models for predicting counts of items per transaction. These transactions-sales models can incorporate time-varying trend, seasonal, price, promotion, random effects and other outlet-specific predictors for individual items. Sequential Bayesian analysis involves fast, parallel filtering on sets of decoupled items and…
In recent years, probabilistic forecasting is an emerging topic, which is why there is a growing need of suitable methods for the evaluation of multivariate predictions. We analyze the sensitivity of the most common scoring rules, especially regarding quality of the forecasted dependency structures. Additionally, we propose scoring rules based on the copula, which uniquely describes the dependency structure for every probability distribution with continuous marginal distributions. Efficient estimation of the considered scoring rules and evaluation methods such as the Diebold-Mariano test are d…
Lung tumor tracking for radiotherapy requires real-time, multiple-step ahead forecasting of a quasi-periodic time series recording instantaneous tumor locations. We introduce a location-mixture autoregressive (LMAR) process that admits multimodal conditional distributions, fast approximate inference using the EM algorithm and accurate multiple-step ahead predictive distributions. LMAR outperforms several commonly used methods in terms of out-of-sample prediction accuracy using clinical data from lung tumor patients. With its superior predictive performance and real-time computation, the LMAR m…
Mortality rates are often disaggregated by different attributes, such as sex, state, education, religion or ethnicity. Forecasting mortality rates at the national and sub-national levels plays an important role in making social policies associated with the national and sub-national levels. However, base forecasts at the sub-national levels may not add up to the forecasts at the national level. To address this issue, we consider the problem of reconciling mortality rate forecasts from the viewpoint of grouped time-series forecasting methods (Hyndman et al., 2011). A bottom-up method and an opti…
Age-specific mortality rates are often disaggregated by different attributes, such as sex, state, ethnic group and socioeconomic status. In making social policies and pricing annuity at national and subnational levels, it is important not only to forecast mortality accurately, but also to ensure that forecasts at the subnational level add up to the forecasts at the national level. This motivates recent developments in grouped functional time series methods (Shang and Hyndman, 2017) to reconcile age-specific mortality forecasts. We extend these grouped functional time series forecasting methods…
The need to forecast solar irradiation at a specific location over short-time horizons has acquired immense importance. In this paper, we report on analyses results involving statistical and machine learning techniques to predict hourly horizontal solar irradiation at one-hour ahead horizon using data sets from three different cities in the U.S. with different climatic conditions. A simple forecast approach that assumes consecutive days are identical serves as a baseline model against which to compare competing forecast alternatives. One approach is to use seasonal ARIMA models. Surprisingly, …
An accurate prediction of crude oil prices over long future horizons is challenging and of great interest to governments, enterprises, and investors. This paper proposes a revised hybrid model built upon empirical mode decomposition (EMD) based on the feed-forward neural network (FNN) modeling framework incorporating the slope-based method (SBM), which is capable of capturing the complex dynamic of crude oil prices. Three commonly used multi-step-ahead prediction strategies proposed in the literature, including iterated strategy, direct strategy, and MIMO (multiple-input multiple-output) strat…
In this paper, we propose an improved Bayesian bidirectional long-short term memory (BiLSTM) neural networks for multi-step ahead (MSA) solar generation forecasting. The proposed technique applies alpha-beta divergence for a more appropriate consideration of outliers in the solar generation data and resulting variability of the weight parameter distribution in the neural network. The proposed method is examined on highly granular solar generation data from Ausgrid using probabilistic evaluation metrics such as Pinball loss and Winkler score. Moreover, a comparative analysis between MSA and the…
i-EM S.r.l（i-EM有限公司）；Department of Energy, Systems, Territory and Constructions Engineering, University of Pisa（比萨大学能源、系统、领土和建筑工程系）；Enel Green Power S.p.A.（意大利国家电力公司绿色电力分公司（Enel Green Power S.p.A.））；Renewable Energy Management, Enel Green Power S.p.A.（可再生能源管理，意大利国家电力公司绿色电力分公司（Enel Green Power S.p.A.））；University of Pisa（比萨大学）
The ability to accurately forecast power generation from renewable sources is nowadays recognised as a fundamental skill to improve the operation of power systems. Despite the general interest of the power community in this topic, it is not always simple to compare different forecasting methodologies, and infer the impact of single components in providing accurate predictions. In this paper we extensively compare simple forecasting methodologies with more sophisticated ones over 32 photovoltaic plants of different size and technology over a whole year. Also, we try to evaluate the impact of we…
Several approaches have been proposed to forecast day-ahead locational marginal price (daLMP) in deregulated energy markets. The rise of deep learning has motivated its use in energy price forecasts but most deep learning approaches fail to accommodate for exogenous variables, which have significant influence in the peaks and valleys of the daLMP. Accurate forecasts of the daLMP valleys are of crucial importance for power generators since one of the most important decisions they face is whether to sell power at a loss to prevent incurring in shutdown and start-up costs, or to bid at production…
University of Chinese Academy of Sciences（中国科学院大学）
Accurate day-ahead individual residential load forecasting is of great importance to various applications of smart grid on day-ahead market. Deep learning, as a powerful machine learning technology, has shown great advantages and promising application in load forecasting tasks. However, deep learning is a computationally-hungry method, and requires high costs (e.g., time, energy and CO2 emission) to train a deep learning model, which aggravates the energy crisis and incurs a substantial burden to the environment. As a consequence, the deep learning methods are difficult to be popularized and a…
In this paper, we propose a Light Gradient Boosting (LightGBM) to forecast dominant wave periods in oceanic waters. First, we use the data collected from CDIP buoys and apply various data filtering methods. The data filtering methods allow us to obtain a high-quality dataset for training and validation purposes. We then extract various wave-based features like wave heights, periods, skewness, kurtosis, etc., and atmospheric features like humidity, pressure, and air temperature for the buoys. Afterward, we train algorithms that use LightGBM and Extra Trees through a hv-block cross-validation sc…
Spatial and time-dependent data is of interest in many applications. This task is difficult due to its complex spatial dependency, long-range temporal dependency, data non-stationarity, and data heterogeneity. To address these challenges, we propose Forecaster, a graph Transformer architecture. Specifically, we start by learning the structure of the graph that parsimoniously represents the spatial dependency between the data at different locations. Based on the topology of the graph, we sparsify the Transformer to account for the strength of spatial dependency, long-range temporal dependency, …
