The problem of probabilistic forecasting and online simulation of real-time electricity market with stochastic generation and demand is considered. By exploiting the parametric structure of the direct current optimal power flow, a new technique based on online dictionary learning (ODL) is proposed. The ODL approach incorporates real-time measurements and historical traces to produce forecasts of joint and marginal probability distributions of future locational marginal prices, power flows, and dispatch levels, conditional on the system state at the time of forecasting. Compared with standard Monte Carlo simulation techniques, the ODL approach offers several orders of magnitude improvement in computation time, making it feasible for online forecasting of market operations. Numerical simulations on large and moderate size power systems illustrate its performance and complexity features and its potential as a tool for system operators.
Electricity prices strongly depend on seasonality of different time scales, therefore any forecasting of electricity prices has to account for it. Neural networks have proven successful in short-term price-forecasting, but complicated architectures like LSTM are used to integrate the seasonal behaviour. This paper shows that simple neural network architectures like DNNs with an embedding layer for seasonality information can generate a competitive forecast. The embedding-based processing of calendar information additionally opens up new applications for neural networks in electricity trading, …
The world is facing major challenges related to global warming and emissions of greenhouse gases is a major causing factor. In 2017, energy industries accounted for 46% of all CO2 emissions globally, which shows a large potential for reduction. This paper proposes a novel short-term CO2 emissions forecast to enable intelligent scheduling of flexible electricity consumption to minimize the resulting CO2 emissions. Two proposed time series decomposition methods are developed for short-term forecasting of the CO2 emissions of electricity. These are in turn bench-marked against a set of state-of-t…
We present the winning strategy of an electricity demand forecasting competition. This competition was organized to design new forecasting methods for unstable periods such as the one starting in Spring 2020. We rely on state-space models to adapt standard statistical and machine learning models. We claim that it achieves the right compromise between two extremes. On the one hand, purely time-series models such as autoregressives are adaptive in essence but fail to capture dependence to exogenous variables. On the other hand, machine learning methods allow to learn complex dependence to explan…
Classical time series models have serious difficulties in modeling and forecasting the enormous fluctuations of electricity spot prices. Markov regime switch models belong to the most often used models in the electricity literature. These models try to capture the fluctuations of electricity spot prices by using different regimes, each with its own mean and covariance structure. Usually one regime is dedicated to moderate prices and another is dedicated to high prices. However, these models show poor performance and there is no theoretical justification for this kind of classification. The mer…
Highly accurate interval forecasting of electricity demand is fundamental to the success of reducing the risk when making power system planning and operational decisions by providing a range rather than point estimation. In this study, a novel modeling framework integrating bivariate empirical mode decomposition (BEMD) and support vector regression (SVR), extended from the well-established empirical mode decomposition (EMD) based time series modeling framework in the energy demand forecasting literature, is proposed for interval forecasting of electricity demand. The novelty of this study aris…
Probabilistic forecasting of electricity load curves is of fundamental importance for effective scheduling and decision making in the increasingly volatile and competitive energy markets. We propose a novel approach to construct probabilistic predictors for curves (PPC), which leads to a natural and new definition of quantiles in the context of curve-to-curve linear regression. There are three types of PPC: a predictive set, a predictive band and a predictive quantile, all of which are defined at a pre-specified nominal probability level. In the simulation study, the PPC achieve promising cove…
Universidad Carlos III de Madrid（马德里卡洛斯三世大学）；UC3M-Santander Big Data Institute (IBiDat)（马德里卡洛斯三世大学-桑坦德大数据研究所（IBiDat））
Most electricity systems worldwide are deploying advanced metering infrastructures to collect relevant operational data. In particular, smart meters allow tracking electricity load consumption at a very disaggregated level and at high frequency rates. This data opens the possibility of developing new forecasting models with a potential positive impact in electricity systems. We present a general methodology that is able to process and forecast a large number of smart meter time series. Instead of using traditional and univariate approaches, we develop a single but complex recurrent neural-netw…
The increasing importance of renewable energy, especially solar and wind power, has led to new forces in the formation of electricity prices. Hence, this paper introduces an econometric model for the hourly time series of electricity prices of the European Power Exchange (EPEX) which incorporates specific features like renewable energy. The model consists of several sophisticated and established approaches and can be regarded as a periodic VAR-TARCH with wind power, solar power, and load as influences on the time series. It is able to map the distinct and well-known features of electricity pri…
Budapest University of Technology and Economics（布达佩斯技术与经济大学）
Energy price forecasting is a relevant yet hard task in the field of multi-step time series forecasting. In this paper we compare a well-known and established method, ARMA with exogenous variables with a relatively new technique Gradient Boosting Regression. The method was tested on data from Global Energy Forecasting Competition 2014 with a year long rolling window forecast. The results from the experiment reveal that a multi-model approach is significantly better performing in terms of error metrics. Gradient Boosting can deal with seasonality and auto-correlation out-of-the box and achieve …
Future grid management systems will coordinate distributed production and storage resources to manage, in a cost effective fashion, the increased load and variability brought by the electrification of transportation and by a higher share of weather dependent production. Electricity demand forecasts at a low level of aggregation will be key inputs for such systems. We focus on forecasting demand at the individual household level, which is more challenging than forecasting aggregate demand, due to the lower signal-to-noise ratio and to the heterogeneity of consumption patterns across households.…
The exponential growth of renewable energy capacity has brought much uncertainty to electricity prices and to electricity generation. To address this challenge, the energy exchanges have been developing further trading possibilities, especially the intraday and balancing markets. For an energy trader participating in both markets, the forecasting of imbalance prices is of particular interest. Therefore, in this manuscript we conduct a very short-term probabilistic forecasting of imbalance prices, contributing to the scarce literature in this novel subject. The forecasting is performed 30 minut…
This work presents ensemble forecasting of monthly electricity demand using pattern similarity-based forecasting methods (PSFMs). PSFMs applied in this study include $k$-nearest neighbor model, fuzzy neighborhood model, kernel regression model, and general regression neural network. An integral part of PSFMs is a time series representation using patterns of time series sequences. Pattern representation ensures the input and output data unification through filtering a trend and equalizing variance. Two types of ensembles are created: heterogeneous and homogeneous. The former consists of differe…
House of Energy Markets and Finance（能源市场与金融研究院）；University of Duisburg-Essen（杜伊斯堡-埃森大学）
We present a winning method of the IEEE DataPort Competition on Day-Ahead Electricity Demand Forecasting: Post-COVID Paradigm. The day-ahead load forecasting approach is based on online forecast combination of multiple point prediction models. It contains four steps: i) data cleaning and preprocessing, ii) a holiday adjustment procedure, iii) training of individual forecasting models, iv) forecast combination by smoothed Bernstein Online Aggregation (BOA). The approach is flexible and can quickly adopt to new energy system situations as they occurred during and after COVID-19 shutdowns. The po…
Postprocessing of point predictions for probabilistic forecasting of day-ahead electricity prices: The benefits of using isotonic distributional regression
Électricité de France R&D（法国电力公司研发部）；Viking Conseil（维京咨询公司）
We focus on day-ahead electricity load forecasting of substations of the distribution network in France; therefore, our problem lies between the instability of a single consumption and the stability of a countrywide total demand. Moreover, we are interested in forecasting the loads of over one thousand substations; consequently, we are in the context of forecasting multiple time series. To that end, we rely on an adaptive methodology that provided excellent results at a national scale; the idea is to combine generalized additive models with state-space representations. However, the extension o…
The liberalization of electricity markets and the development of renewable energy sources has led to new challenges for decision makers. These challenges are accompanied by an increasing uncertainty about future electricity price movements. The increasing amount of papers, which aim to model and predict electricity prices for a short period of time provided new opportunities for market participants. However, the electricity price literature seem to be very scarce on the issue of medium- to long-term price forecasting, which is mandatory for investment and political decisions. Our paper closes …
Modeling price risks is crucial for economic decision making in energy markets. Besides the risk of a single price, the dependence structure of multiple prices is often relevant. We therefore propose a generic and easy-to-implement method for creating multivariate probabilistic forecasts based on univariate point forecasts of day-ahead electricity prices. While each univariate point forecast refers to one of the day's 24 hours, the multivariate forecast distribution models dependencies across hours. The proposed method is based on simple copula techniques and an optional time series component.…
Carnegie Mellon University（卡内基梅隆大学）；Wrocław University of Science and Technology（弗罗茨瓦夫理工大学）
We extend the neural basis expansion analysis (NBEATS) to incorporate exogenous factors. The resulting method, called NBEATSx, improves on a well performing deep learning model, extending its capabilities by including exogenous variables and allowing it to integrate multiple sources of useful information. To showcase the utility of the NBEATSx model, we conduct a comprehensive study of its application to electricity price forecasting (EPF) tasks across a broad range of years and markets. We observe state-of-the-art performance, significantly improving the forecast accuracy by nearly 20% over t…
Electricity is bought and sold in wholesale markets at prices that fluctuate significantly. Short-term forecasting of electricity prices is an important endeavor because it helps electric utilities control risk and because it influences competitive strategy for generators. As the "smart grid" grows, short-term price forecasts are becoming an important input to bidding and control algorithms for battery operators and demand response aggregators. While the statistics and machine learning literature offers many proposed methods for electricity price prediction, there is no consensus supporting a …
We conduct an extensive empirical study on short-term electricity price forecasting (EPF) to address the long-standing question if the optimal model structure for EPF is univariate or multivariate. We provide evidence that despite a minor edge in predictive performance overall, the multivariate modeling framework does not uniformly outperform the univariate one across all 12 considered datasets, seasons of the year or hours of the day, and at times is outperformed by the latter. This is an indication that combining advanced structures or the corresponding forecasts from both modeling approache…
The coronavirus disease 2019 (COVID-19) pandemic has urged many governments in the world to enforce a strict lockdown where all nonessential businesses are closed and citizens are ordered to stay at home. One of the consequences of this policy is a significant change in electricity consumption patterns. Since load forecasting models rely on calendar or meteorological information and are trained on historical data, they fail to capture the significant break caused by the lockdown and have exhibited poor performances since the beginning of the pandemic. This makes the scheduling of the electrici…
The recent abundance of data on electricity consumption at different scales opens new challenges and highlights the need for new techniques to leverage information present at finer scales in order to improve forecasts at wider scales. In this work, we take advantage of the similarity between this hierarchical prediction problem and multi-scale transfer learning. We develop two methods for hierarchical transfer learning, based respectively on the stacking of generalized additive models and random forests, and on the use of aggregation of experts. We apply these methods to two problems of electr…
The reliable estimation of forecast uncertainties is crucial for risk-sensitive optimal decision making. In this paper, we propose implicit generative ensemble post-processing, a novel framework for multivariate probabilistic electricity price forecasting. We use a likelihood-free implicit generative model based on an ensemble of point forecasting models to generate multivariate electricity price scenarios with a coherent dependency structure as a representation of the joint predictive distribution. Our ensemble post-processing method outperforms well-established model combination benchmarks. …
Probabilistic intraday electricity price forecasting using generative machine learning
Institute of Statistics, Karlsruhe Institute of Technology（卡尔斯鲁厄技术大学统计研究所）；Heidelberg Institute for Theoretical Studies（海德堡理论研究所）；Department of Mathematics and Computer Science, Marburg University（马堡大学数学与计算机科学系）；Department of Operations Research and Business Intelligence, Wrocław University of Science and Technology（波兹南大学 of Science and Technology 运筹学与商业智能系）
In our paper we analyze the relationship between the day-ahead electricity price of the Energy Exchange Austria (EXAA) and other day-ahead electricity prices in Europe. We focus on markets, which settle their prices after the EXAA, which enables traders to include the EXAA price into their calculations. For each market we employ econometric models to incorporate the EXAA price and compare them with their counterparts without the price of the Austrian exchange. By employing a forecasting study, we find that electricity price models can be improved when EXAA prices are considered.
Intra-day price spreads are of interest to electricity traders, storage and electric vehicle operators. This paper formulates dynamic density functions, based upon skewed-t and similar representations, to model and forecast the German electricity price spreads between different hours of the day, as revealed in the day-ahead auctions. The four specifications of the density functions are dynamic and conditional upon exogenous drivers, thereby permitting the location, scale and shape parameters of the densities to respond hourly to such factors as weather and demand forecasts. The best fitting an…
In the following paper, we analyse the ID$_3$-Price in the German Intraday Continuous electricity market using an econometric time series model. A multivariate approach is conducted for hourly and quarter-hourly products separately. We estimate the model using lasso and elastic net techniques and perform an out-of-sample, very short-term forecasting study. The model's performance is compared with benchmark models and is discussed in detail. Forecasting results provide new insights to the German Intraday Continuous electricity market regarding its efficiency and to the ID$_3$-Price behaviour.
We propose a simple empirical scaling law that describes load forecasting accuracy at different levels of aggregation. The model is justified based on a simple decomposition of individual consumption patterns. We show that for different forecasting methods and horizons, aggregating more customers improves the relative forecasting performance up to specific point. Beyond this point, no more improvement in relative performance can be obtained.
We consider the setting of sequential prediction of arbitrary sequences based on specialized experts. We first provide a review of the relevant literature and present two theoretical contributions: a general analysis of the specialist aggregation rule of Freund et al. (1997) and an adaptation of fixed-share rules of Herbster and Warmuth (1998) in this setting. We then apply these rules to the sequential short-term (one-day-ahead) forecasting of electricity consumption; to do so, we consider two data sets, a Slovakian one and a French one, respectively concerned with hourly and half-hourly pred…
Purpose: Trading on electricity markets occurs such that the price settlement takes place before delivery, often day-ahead. In practice, these prices are highly volatile as they largely depend upon a range of variables such as electricity demand and the feed-in from renewable energy sources. Hence, accurate forecasts are demanded. Approach: This paper aims at comparing different predictors stemming from supply-side (solar and wind power generation), demand-side, fuel-related and economic influences. For this reason, we implement a broad range of non-linear models from machine learning and draw…
Day Ahead Electricity Markets (DAMs) in India are thin but growing. Consistent price forecasts are important for their utilization in portfolio optimization models. Univariate or multivariate models with standard exogenous variables such as special day effects etc. are not always useful. Drivers of demand and supply include weather variations over large geographic areas, outages of power system elements and sudden changes in contracts which lead the players to access power exchanges. These needs to be considered in forecasting models. Such models are observed to considerably reduce forecasting…
We consider the problem of short- and medium-term electricity load forecasting by using past loads and daily weather forecast information. Conventionally, many researchers have directly applied regression analysis. However, interpreting the effect of weather on these loads is difficult with the existing methods. In this study, we build a statistical model that resolves this interpretation issue. A varying coefficient model with basis expansion is used to capture the nonlinear structure of the weather effect. This approach results in an interpretable model when the regression coefficients are n…
We employ a recently proposed change-point detection algorithm, the Narrowest-Over-Threshold (NOT) method, to select subperiods of past observations that are similar to the currently recorded values. Then, contrarily to the traditional time series approach in which the most recent $τ$ observations are taken as the calibration sample, we estimate autoregressive models only for data in these subperiods. We illustrate our approach using a challenging dataset - day-ahead electricity prices in the German EPEX SPOT market - and observe a significant improvement in forecasting accuracy compared to co…
We are interested in the online prediction of the electricity load, within the Bayesian framework of dynamic models. We offer a review of sequential Monte Carlo methods, and provide the calculations needed for the derivation of so-called particles filters. We also discuss the practical issues arising from their use, and some of the variants proposed in the literature to deal with them, giving detailed algorithms whenever possible for an easy implementation. We propose an additional step to help make basic particle filters more robust with regard to outlying observations. Finally we use such a …
Short Term Load Forecast (STLF) is necessary for effective scheduling, operation optimization trading, and decision-making for electricity consumers. Modern and efficient machine learning methods are recalled nowadays to manage complicated structural big datasets, which are characterized by having a nonlinear temporal dependence structure. We propose different statistical nonlinear models to manage these challenges of hard type datasets and forecast 15-min frequency electricity load up to 2-days ahead. We show that the Long-short Term Memory (LSTM) and the Gated Recurrent Unit (GRU) models app…
