The interest in developing smart cities has increased dramatically in recent years. In this context an intelligent transportation system depicts a major topic. The forecast of traffic flow is indispensable for an efficient intelligent transportation system. The traffic flow forecast is a difficult task, due to its stochastic and non linear nature. Besides classical statistical methods, neural networks are a promising possibility to predict future traffic flow. In our work, this prediction is performed with various recurrent neural networks. These are trained on measurements of induction loops, which are placed in intersections of the city. We utilized data from beginning of January to the end of July in 2018. Each model incorporates sequences of the measured traffic flow from all sensors and predicts the future traffic flow for each sensor simultaneously. A variety of model architectures, forecast horizons and input data were investigated. Most often the vector output model with gated recurrent units achieved the smallest error on the test set over all considered prediction scenarios. Due to the small amount of data, generalization of the trained models is limited.
Time series forecasting is difficult. It is difficult even for recurrent neural networks with their inherent ability to learn sequentiality. This article presents a recurrent neural network based time series forecasting framework covering feature engineering, feature importances, point and interval predictions, and forecast evaluation. The description of the method is followed by an empirical study using both LSTM and GRU networks.
University of Oxford（牛津大学）；Mind Foundry Ltd.（思维铸造有限公司）
Recurrent neural networks (RNNs) are state-of-the-art in several sequential learning tasks, but they often require considerable amounts of data to generalise well. For many time series forecasting (TSF) tasks, only a few dozens of observations may be available at training time, which restricts use of this class of models. We propose a novel RNN-based model that directly addresses this problem by learning a shared feature embedding over the space of many quantised time series. We show how this enables our RNN framework to accurately and reliably forecast unseen time series, even when there is l…
University of Southern California（南加州大学）；California Institute of Technology（加州理工学院）
Spatiotemporal forecasting has various applications in neuroscience, climate and transportation domain. Traffic forecasting is one canonical example of such learning task. The task is challenging due to (1) complex spatial dependency on road networks, (2) non-linear temporal dynamics with changing road conditions and (3) inherent difficulty of long-term forecasting. To address these challenges, we propose to model the traffic flow as a diffusion process on a directed graph and introduce Diffusion Convolutional Recurrent Neural Network (DCRNN), a deep learning framework for traffic forecasting …
In this paper, we present a monitoring system that allows increasing road safety by predicting ice formation. The system consists of a network of road weather stations and intelligence data processing program module. The results were achieved by combining physical models for forecasting road conditions based on measurements from stations and machine learning models for detecting incorrect data and forecast correction.
We study epidemic forecasting on real-world health data by a graph-structured recurrent neural network (GSRNN). We achieve state-of-the-art forecasting accuracy on the benchmark CDC dataset. To improve model efficiency, we sparsify the network weights via transformed-$\ell_1$ penalty and maintain prediction accuracy at the same level with 70% of the network weights being zero.
Time series modeling has entered an era of unprecedented growth in the size and complexity of data which require new modeling approaches. While many new general purpose machine learning approaches have emerged, they remain poorly understand and irreconcilable with more traditional statistical modeling approaches. We present a general class of exponential smoothed recurrent neural networks (RNNs) which are well suited to modeling non-stationary dynamical systems arising in industrial applications. In particular, we analyze their capacity to characterize the non-linear partial autocorrelation st…
Deep learning applied to weather forecasting has started gaining popularity because of the progress achieved by data-driven models. The present paper compares two different deep learning architectures to perform weather prediction on daily data gathered from 18 cities across Europe and spanned over a period of 15 years. We propose the Deep Attention Unistream Multistream (DAUM) networks that investigate different types of input representations (i.e. tensorial unistream vs. multistream ) as well as the incorporation of the attention mechanism. In particular, we show that adding a self-attention…
Probabilistic forecasting, i.e. estimating the probability distribution of a time series' future given its past, is a key enabler for optimizing business processes. In retail businesses, for example, forecasting demand is crucial for having the right inventory available at the right time at the right place. In this paper we propose DeepAR, a methodology for producing accurate probabilistic forecasts, based on training an auto regressive recurrent network model on a large number of related time series. We demonstrate how by applying deep learning techniques to forecasting, one can overcome many…
Argonne National Laboratory（阿贡国家实验室）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）
Traffic forecasting approaches are critical to developing adaptive strategies for mobility. Traffic patterns have complex spatial and temporal dependencies that make accurate forecasting on large highway networks a challenging task. Recently, diffusion convolutional recurrent neural networks (DCRNNs) have achieved state-of-the-art results in traffic forecasting by capturing the spatiotemporal dynamics of the traffic. Despite the promising results, however, applying DCRNNs for large highway networks still remains elusive because of computational and memory bottlenecks. We present an approach fo…
The increasing complexity of mobility plus the growing population in cities, together with the importance of privacy when sharing data from vehicles or any device, makes traffic forecasting that uses data from infrastructure and citizens an open and challenging task. In this paper, we introduce a novel approach to deal with predictions of speed, volume, and main traffic direction, in a new aggregated way of traffic data presented as videos. The approach leverages the continuity in a sequence of frames and its dynamics, learning to predict changing areas in a low dimensional space and then, rec…
Enhancing Prediction and Analysis of UK Road Traffic Accident Severity Using AI: Integration of Machine Learning, Econometric Techniques, and Time Series Forecasting in Public Health Research
In this paper, we propose two nonparametric methods used in the forecasting of functional time-dependent data, namely functional singular spectrum analysis recurrent forecasting and vector forecasting. Both algorithms utilize the results of functional singular spectrum analysis and past observations in order to predict future data points where recurrent forecasting predicts one function at a time and the vector forecasting makes predictions using functional vectors. We compare our forecasting methods to a gold standard algorithm used in the prediction of functional, time-dependent data by way …
University of Washington（华盛顿大学）；INRIX, Inc.（因瑞克斯公司）
Traffic forecasting is a particularly challenging application of spatiotemporal forecasting, due to the time-varying traffic patterns and the complicated spatial dependencies on road networks. To address this challenge, we learn the traffic network as a graph and propose a novel deep learning framework, Traffic Graph Convolutional Long Short-Term Memory Neural Network (TGC-LSTM), to learn the interactions between roadways in the traffic network and forecast the network-wide traffic state. We define the traffic graph convolution based on the physical network topology. The relationship between t…
Short-term traffic flow prediction is one of the crucial issues in intelligent transportation system, which is an important part of smart cities. Accurate predictions can enable both the drivers and the passengers to make better decisions about their travel route, departure time and travel origin selection, which can be helpful in traffic management. Multiple models and algorithms based on time series prediction and machine learning were applied to this issue and achieved acceptable results. Recently, the availability of sufficient data and computational power, motivates us to improve the pred…
Dalian University of Technology（大连理工大学）；Shenyang Blower Works Group Corporation（沈阳鼓风机集团股份有限公司）
The demand of probabilistic time series forecasting has been recently raised in various dynamic system scenarios, for example, system identification and prognostic and health management of machines. To this end, we combine the advances in both deep generative models and state space model (SSM) to come up with a novel, data-driven deep probabilistic sequence model. Specifically, we follow the popular encoder-decoder generative structure to build the recurrent neural networks (RNN) assisted variational sequence model on an augmented recurrent input space, which could induce rich stochastic seque…
University of Minnesota, Twin Cities（明尼苏达大学双城分校）；IBM Research, T. J. Watson Research Center（IBM研究院托马斯·J·沃森研究中心）
Multivariate time-series modeling and forecasting is an important problem with numerous applications. Traditional approaches such as VAR (vector auto-regressive) models and more recent approaches such as RNNs (recurrent neural networks) are indispensable tools in modeling time-series data. In many multivariate time series modeling problems, there is usually a significant linear dependency component, for which VARs are suitable, and a nonlinear component, for which RNNs are suitable. Modeling such times series with only VAR or only RNNs can lead to poor predictive performance or complex models …
Single layer Feedforward Neural Network(FNN) is used many a time as a last layer in models such as seq2seq or could be a simple RNN network. The importance of such layer is to transform the output to our required dimensions. When it comes to weights and biases initialization, there is no such specific technique that could speed up the learning process. We could depend on deep network initialization techniques such as Xavier or He initialization. But such initialization fails to show much improvement in learning speed or accuracy. In this paper we propose Zero Initialization (ZI) for weights of…
Peking University（北京大学）；Hangzhou Dianzi University（杭州电子科技大学）；Beijing Institute of Big Data Research (BIBDR)（北京大数据研究院）
Spatio-temporal prediction plays an important role in many application areas especially in traffic domain. However, due to complicated spatio-temporal dependency and high non-linear dynamics in road networks, traffic prediction task is still challenging. Existing works either exhibit heavy training cost or fail to accurately capture the spatio-temporal patterns, also ignore the correlation between distant roads that share the similar patterns. In this paper, we propose a novel deep learning framework to overcome these issues: 3D Temporal Graph Convolutional Networks (3D-TGCN). Two novel compon…
Recurrent Neural Networks (RNN) have become competitive forecasting methods, as most notably shown in the winning method of the recent M4 competition. However, established statistical models such as ETS and ARIMA gain their popularity not only from their high accuracy, but they are also suitable for non-expert users as they are robust, efficient, and automatic. In these areas, RNNs have still a long way to go. We present an extensive empirical study and an open-source software framework of existing RNN architectures for forecasting, that allow us to develop guidelines and best practices for th…
West Transmission Planning（西部输电规划部门）；Robotics Institute（机器人研究所）；American Electric Power（美国电力公司）；Carnegie Mellon University（卡内基梅隆大学）
The non-stationarity characteristic of the solar power renders traditional point forecasting methods to be less useful due to large prediction errors. This results in increased uncertainties in the grid operation, thereby negatively affecting the reliability and increased cost of operation. This research paper proposes a unified architecture for multi-time-horizon predictions for short and long-term solar forecasting using Recurrent Neural Networks (RNN). The paper describes an end-to-end pipeline to implement the architecture along with the methods to test and validate the performance of the …
Spatio-temporal dynamics of physical processes are generally modeled using partial differential equations (PDEs). Though the core dynamics follows some principles of physics, real-world physical processes are often driven by unknown external sources. In such cases, developing a purely analytical model becomes very difficult and data-driven modeling can be of assistance. In this paper, we present a hybrid framework combining physics-based numerical models with deep learning for source identification and forecasting of spatio-temporal dynamical systems with unobservable time-varying external sou…
StateFlow: Dual-State Recurrent Modeling for Long-Horizon Time Series Forecasting
Department of Computer Science, Illinois Institute of Technology, Chicago, IL, USA（伊利诺伊理工学院计算机科学系）；Department of Computer Science, Emory University, Atlanta, GA, USA（埃默里大学计算机科学系）
Centre National de la Recherche Scientifique (CNRS)（法国国家科学研究中心）
The notion of memory capacity, originally introduced for echo state and linear networks with independent inputs, is generalized to nonlinear recurrent networks with stationary but dependent inputs. The presence of dependence in the inputs makes natural the introduction of the network forecasting capacity, that measures the possibility of forecasting time series values using network states. Generic bounds for memory and forecasting capacities are formulated in terms of the number of neurons of the nonlinear recurrent network and the autocovariance function or the spectral density of the input. …
We present ARU, an Adaptive Recurrent Unit for streaming adaptation of deep globally trained time-series forecasting models. The ARU combines the advantages of learning complex data transformations across multiple time series from deep global models, with per-series localization offered by closed-form linear models. Unlike existing methods of adaptation that are either memory-intensive or non-responsive after training, ARUs require only fixed sized state and adapt to streaming data via an easy RNN-like update operation. The core principle driving ARU is simple --- maintain sufficient statistic…
Georgia Institute of Technology（佐治亚理工学院）；Pennsylvania State University（宾夕法尼亚州立大学）
Many systems used by society are extremely vulnerable to space weather events such as solar flares and geomagnetic storms which could potentially cause catastrophic damage. In recent years, many works have emerged to provide early warning to such systems by forecasting these events through some proxy, but these approaches have largely focused on a specific phenomenon. We present a sequence-to-sequence learning approach to the problem of forecasting global space weather conditions at an hourly resolution. This approach improves upon other work in this field by simultaneously forecasting several…
Time series forecasting based on deep architectures has been gaining popularity in recent years due to their ability to model complex non-linear temporal dynamics. The recurrent neural network is one such model capable of handling variable-length input and output. In this paper, we leverage recent advances in deep generative models and the concept of state space models to propose a stochastic adaptation of the recurrent neural network for multistep-ahead time series forecasting, which is trained with stochastic gradient variational Bayes. In our model design, the transition function of the rec…
Delft University of Technology（代尔夫特理工大学）；City University of Hong Kong（香港城市大学）；Virginia Polytechnic Institute and State University（弗吉尼亚理工学院暨州立大学）
Short-term forecasting of passenger flow is critical for transit management and crowd regulation. Spatial dependencies, temporal dependencies, inter-station correlations driven by other latent factors, and exogenous factors bring challenges to the short-term forecasts of passenger flow of urban rail transit networks. An innovative deep learning approach, Multi-Graph Convolutional-Recurrent Neural Network (MGC-RNN) is proposed to forecast passenger flow in urban rail transit systems to incorporate these complex factors. We propose to use multiple graphs to encode the spatial and other heterogen…
Mediamobile（Mediamobile 公司）；Institut de Mathématiques de Toulouse（图卢兹数学研究所）；INSA（法国国立应用科学学院）
In this paper, we propose deep learning architectures (FNN, CNN and LSTM) to forecast a regression model for time dependent data. These algorithm's are designed to handle Floating Car Data (FCD) historic speeds to predict road traffic data. For this we aggregate the speeds into the network inputs in an innovative way. We compare the RMSE thus obtained with the results of a simpler physical model, and show that the latter achieves better RMSE accuracy. We also propose a new indicator, which evaluates the algorithms improvement when compared to a benchmark prediction. We conclude by questioning …
Amirkabir University of Technology (Tehran Polytechnic)（阿米尔卡比尔理工大学（德黑兰理工学院））
This thesis serves three primary purposes, first of which is to forecast two stocks, i.e. Goldman Sachs (GS) and General Electric (GE). In order to forecast stock prices, we used a long short-term memory (LSTM) model in which we inputted the prices of two other stocks that lie in rather close correlation with GS. Other models such as ARIMA were used as benchmark. Empirical results manifest the practical challenges when using LSTM for forecasting stocks. One of the main upheavals was a recurring lag which we called "forecasting lag". The second purpose is to develop a more general and objective…
Forecasting the actual amount of electricity with respect to the need/demand of the load is always been a challenging task for each power plants based generating stations. Due to uncertain demand of electricity at receiving end of station causes several challenges such as: reduction in performance parameters of generating and receiving end stations, minimization in revenue, increases the jeopardize for the utility to predict the future energy need for a company etc. With this issues, the precise forecasting of load at the receiving end station is very consequential parameter to establish the i…
As an important part of intelligent transportation systems, traffic forecasting has attracted tremendous attention from academia and industry. Despite a lot of methods being proposed for traffic forecasting, it is still difficult to model complex spatial-temporal dependency. Temporal dependency includes short-term dependency and long-term dependency, and the latter is often overlooked. Spatial dependency can be divided into two parts: distance-based spatial dependency and hidden spatial dependency. To model complex spatial-temporal dependency, we propose a novel framework for traffic forecasti…
With the advent of Big Data, nowadays in many applications databases containing large quantities of similar time series are available. Forecasting time series in these domains with traditional univariate forecasting procedures leaves great potentials for producing accurate forecasts untapped. Recurrent neural networks (RNNs), and in particular Long Short-Term Memory (LSTM) networks, have proven recently that they are able to outperform state-of-the-art univariate time series forecasting methods in this context when trained across all available time series. However, if the time series database …
Swami Vivekananda Institute of Science & Technology（斯瓦米·维韦卡南达科学技术学院）；Department of Computer Science and Engineering（计算机科学与工程系）
Photovoltaic power is a renewable source of energy which is highly used in industries. In economically struggling countries it can be a potential source of electric energy as other non-renewable resources are already exhausting. Now if installation of a photovoltaic cell in a region is done prior to research, it may not provide the desired energy output required for running that region. Hence forecasting is required which can elicit the output from a particular region considering its geometrical coordinates, solar parameter like GHI and weather parameters like temperature and wind speed etc. O…
NAVER Corp（NAVER公司）；Ulsan National Institute of Science and Technology (UNIST)（蔚山科学技术院（UNIST））；Korea University（高丽大学）；National Research University Higher School of Economics (NRU-HSE)（俄罗斯国立高等经济大学（NRU-HSE））；Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院（KAIST））
Predicting road traffic speed is a challenging task due to different types of roads, abrupt speed change and spatial dependencies between roads; it requires the modeling of dynamically changing spatial dependencies among roads and temporal patterns over long input sequences. This paper proposes a novel spatio-temporal graph attention (ST-GRAT) that effectively captures the spatio-temporal dynamics in road networks. The novel aspects of our approach mainly include spatial attention, temporal attention, and spatial sentinel vectors. The spatial attention takes the graph structure information (e.…
Probabilistic Multi-Regional Solar Power Forecasting with Any-Quantile Recurrent Neural Networks
Faculty of Electrical Engineering, Czestochowa University of Technology（捷斯托霍瓦技术大学电气工程学院）；Faculty of Mathematics and Computer Science, University of Łódź（卢布林大学数学与计算机科学学院）；CAMINO - Centre for Data Analysis, Modelling and Computational Sciences, University of Łódź（卢布林大学CAMINO数据与建模计算科学中心）
XRMDN: An Extended Recurrent Mixture Density Network for Short-Term Probabilistic Rider Demand Forecasting with High Volatility
Concordia University（康考迪亚大学）；Concordia Institute for Information Systems Engineering (CIISE)（康考迪亚信息系统工程研究所）；Concordia John Molson School of Business (JMSB)（康考迪亚约翰莫尔森商学院）
针对 MoD 需求受内外因素影响且高波动、传统点预测难以刻画不确定性的问题，提出 XRMDN 深度概率预测框架，并在真实数据集上验证其优于基准模型。
RIPCN: A Road Impedance Principal Component Network for Probabilistic Traffic Flow Forecasting
School of Computer Science；Technology Beijing Jiaotong University Beijing China；Department of Computer Science Aalborg University Aalborg Denmark；Key Laboratory of Big Data \& Artificial Intelligence in Transportation, Ministry of Education Beijing China；Beijing Key Laboratory of Traffic Data Mining；Beijing Jiaotong University；Aalborg University；Key Laboratory of Big Data \& Artificial Intelligence in Transportation, Ministry of Education
Facebook；Czestochowa University of Technology（琴斯托霍瓦技术大学）
Short-term load forecasting (STLF) is challenging due to complex time series (TS) which express three seasonal patterns and a nonlinear trend. This paper proposes a novel hybrid hierarchical deep learning model that deals with multiple seasonality and produces both point forecasts and predictive intervals (PIs). It combines exponential smoothing (ES) and a recurrent neural network (RNN). ES extracts dynamically the main components of each individual TS and enables on-the-fly deseasonalization, which is particularly useful when operating on a relatively small data set. A multi-layer RNN is equi…
Rethinking Recurrent Neural Networks for Time Series Forecasting: A Reinforced Recurrent Encoder with Prediction-Oriented Proximal Policy Optimization
School of Management, Huazhong University of Science and Technology（华中科技大学管理学院）；Center for Applied Mathematics, École des Ponts ParisTech（巴黎高等理工学院应用数学中心）；School of Mechanical Engineering, Dalian Jiaotong University（大连交通大学机械工程学院）；School of Computer Science and Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）
We consider a setting where multiple entities inter-act with each other over time and the time-varying statuses of the entities are represented as multiple correlated time series. For example, speed sensors are deployed in different locations in a road network, where the speed of a specific location across time is captured by the corresponding sensor as a time series, resulting in multiple speed time series from different locations, which are often correlated. To enable accurate forecasting on correlated time series, we proposes graph attention recurrent neural networks.First, we build a graph…
Qom University of Technology（库姆理工大学）；London South Bank University（伦敦南岸大学）
Integration of renewable energy sources and emerging loads like electric vehicles to smart grids brings more uncertainty to the distribution system management. Demand Side Management (DSM) is one of the approaches to reduce the uncertainty. Some applications like Nonintrusive Load Monitoring (NILM) can support DSM, however they require accurate forecasting on high resolution data. This is challenging when it comes to single loads like one residential household due to its high volatility. In this paper, we review some of the existing Deep Learning-based methods and present our solution using Ti…
BMW Group（宝马集团）；University of the Federal Armed Forces Munich（慕尼黑联邦国防军大学）
Most traffic state forecast algorithms when applied to urban road networks consider only the links in close proximity to the target location. However, for longer-term forecasts also the traffic state of more distant links or regions of the network are expected to provide valuable information for a data-driven algorithm. This paper studies these expectations of using a network clustering algorithm and one year of Floating Car (FCD) collected by a large fleet of vehicles. First, a clustering algorithm is applied to the data in order to extract congestion-prone regions in the Munich city network.…
Short-term traffic forecasting based on deep learning methods, especially recurrent neural networks (RNN), has received much attention in recent years. However, the potential of RNN-based models in traffic forecasting has not yet been fully exploited in terms of the predictive power of spatial-temporal data and the capability of handling missing data. In this paper, we focus on RNN-based models and attempt to reformulate the way to incorporate RNN and its variants into traffic prediction models. A stacked bidirectional and unidirectional LSTM network architecture (SBU-LSTM) is proposed to assi…
Recurrent Neural Networks (RNNs) represent the de facto standard machine learning tool for sequence modelling, owing to their expressive power and memory. However, when dealing with large dimensional data, the corresponding exponential increase in the number of parameters imposes a computational bottleneck. The necessity to equip RNNs with the ability to deal with the curse of dimensionality, such as through the parameter compression ability inherent to tensors, has led to the development of the Tensor-Train RNN (TT-RNN). Despite achieving promising results in many applications, the full poten…
EPFL（洛桑联邦理工学院）；Cyber-Defence Campus, armasuisse Science and Technology（瑞士联邦国防采购局科技网络防御园区）；University of Geneva（日内瓦大学）；University of Applied Sciences of Western Switzerland（瑞士西部应用科学大学）
Because of the considerable heterogeneity and complexity of the technological landscape, building accurate models to forecast is a challenging endeavor. Due to their high prevalence in many complex systems, S-curves are a popular forecasting approach in previous work. However, their forecasting performance has not been directly compared to other technology forecasting approaches. Additionally, recent developments in time series forecasting that claim to improve forecasting accuracy are yet to be applied to technological development data. This work addresses both research gaps by comparing the …
