The large integration of variable energy resources is expected to shift a large part of the energy exchanges closer to real-time, where more accurate forecasts are available. In this context, the short-term electricity markets and in particular the intraday market are considered a suitable trading floor for these exchanges to occur. A key component for the successful renewable energy sources integration is the usage of energy storage. In this paper, we propose a novel modelling framework for the strategic participation of energy storage in the European continuous intraday market where exchanges occur through a centralized order book. The goal of the storage device operator is the maximization of the profits received over the entire trading horizon, while taking into account the operational constraints of the unit. The sequential decision-making problem of trading in the intraday market is modelled as a Markov Decision Process. An asynchronous distributed version of the fitted Q iteration algorithm is chosen for solving this problem due to its sample efficiency. The large and variable number of the existing orders in the order book motivates the use of high-level actions and an alternative state representation. Historical data are used for the generation of a large number of artificial trajectories in order to address exploration issues during the learning process. The resulting policy is back-tested and compared against a benchmark strategy that is the current industrial standard. Results indicate that the agent converges to a policy that achieves in average higher total revenues than the benchmark strategy.
Reinforcement learning (RL) techniques have shown great success in many challenging quantitative trading tasks, such as portfolio management and algorithmic trading. Especially, intraday trading is one of the most profitable and risky tasks because of the intraday behaviors of the financial market that reflect billions of rapidly fluctuating capitals. However, a vast majority of existing RL methods focus on the relatively low frequency trading scenarios (e.g., day-level) and fail to capture the fleeting intraday investment opportunities due to two major challenges: 1) how to effectively train …
Prosumer operators are dealing with extensive challenges to participate in short-term electricity markets while taking uncertainties into account. Challenges such as variation in demand, solar energy, wind power, and electricity prices as well as faster response time in intraday electricity markets. Machine learning approaches could resolve these challenges due to their ability to continuous learning of complex relations and providing a real-time response. Such approaches are applicable with presence of the high performance computing and big data. To tackle these challenges, a Markov decision …
Driven by the global decarbonization effort, the rapid integration of renewable energy into the conventional electricity grid presents new challenges and opportunities for the battery energy storage system (BESS) participating in the energy market. Energy arbitrage can be a significant source of revenue for the BESS due to the increasing price volatility in the spot market caused by the mismatch between renewable generation and electricity demand. In addition, the Frequency Control Ancillary Services (FCAS) markets established to stabilize the grid can offer higher returns for the BESS due to …
Real-Time Bidding (RTB) is an important mechanism in modern online advertising systems. Advertisers employ bidding strategies in RTB to optimize their advertising effects subject to various financial requirements, especially the return-on-investment (ROI) constraint. ROIs change non-monotonically during the sequential bidding process, and often induce a see-saw effect between constraint satisfaction and objective optimization. While some existing approaches show promising results in static or mildly changing ad markets, they fail to generalize to highly dynamic ad markets with ROI constraints,…
Finding optimal bidding strategies for generation units in electricity markets would result in higher profit. However, it is a challenging problem due to the system uncertainty which is due to the unknown other generation units' strategies. Distributed optimization, where each entity or agent decides on its bid individually, has become state of the art. However, it cannot overcome the challenges of system uncertainties. Deep reinforcement learning is a promising approach to learn the optimal strategy in uncertain environments. Nevertheless, it is not able to integrate the information on the sp…
Load serving entities with storage units reach sizes and performances that can significantly impact clearing prices in electricity markets. Nevertheless, price endogeneity is rarely considered in storage bidding strategies and modeling the electricity market is a challenging task. Meanwhile, model-free reinforcement learning such as the Actor-Critic are becoming increasingly popular for designing energy system controllers. Yet implementation frequently requires lengthy, data-intense, and unsafe trial-and-error training. To fill these gaps, we implement an online Supervised Actor-Critic (SAC) a…
This paper presents a multi-agent reinforcement learning algorithm to represent strategic bidding behavior in freight transport markets. Using this algorithm, we investigate whether feasible market equilibriums arise without any central control or communication between agents. Studying behavior in such environments may serve as a stepping stone towards self-organizing logistics systems like the Physical Internet. We model an agent-based environment in which a shipper and a carrier actively learn bidding strategies using policy gradient methods, posing bid- and ask prices at the individual cont…
The increasing penetration of renewable generations, along with the deregulation and marketization of power industry, promotes the transformation of power market operation paradigms. The optimal bidding strategy and dispatching methodology under these new paradigms are prioritized concerns for both market participants and power system operators, with obstacles of uncertain characteristics, computational efficiency, as well as requirements of hyperopic decision-making. To tackle these problems, the Reinforcement Learning (RL), as an emerging machine learning technique with advantages compared w…
Foster School of Business, University of Washington（华盛顿大学福斯特商学院）；Department of Operations Research and Financial Engineering, Princeton University（普林斯顿大学运筹学与金融工程系）
Truckload brokerages, a $100 billion/year industry in the U.S., plays the critical role of matching shippers with carriers, often to move loads several days into the future. Brokerages not only have to find companies that will agree to move a load, the brokerage often has to find a price that both the shipper and carrier will agree to. The price not only varies by shipper and carrier, but also by the traffic lanes and other variables such as commodity type. Brokerages have to learn about shipper and carrier response functions by offering a price and observing whether each accepts the quote. We…
This paper investigates the effects of a price limit change on the volatility of the Korean stock market's (KRX) intraday stock price process. Based on the most recent transaction data from the KRX, which experienced a change in the price limit on June 15, 2015, we examine the change in realized variance after the price limit change to investigate the overall effects of the change on the intraday market volatility. We then analyze the effects in more detail by applying the discrete Fourier transform (DFT) to the data set. We find evidence that the market becomes more volatile in the intraday h…
This article comes up with an intraday trading strategy under T+1 using Markowitz optimization and Multilayer Perceptron (MLP) with published stock data obtained from the Shenzhen Stock Exchange and Shanghai Stock Exchange. The empirical results reveal the profitability of Markowitz portfolio optimization and validate the intraday stock price prediction using MLP. The findings further combine the Markowitz optimization, an MLP with the trading strategy, to clarify this strategy's feasibility.
We develop a tractable equilibrium model for price formation in intraday electricity markets in the presence of intermittent renewable generation. Using stochastic control theory, we identify the optimal strategies of agents with market impact and exhibit the Nash equilibrium in closed form for a finite number of agents as well as in the asymptotic framework of mean field games. Our model reproduces the empirical features of intraday market prices, such as increasing price volatility at the approach of the delivery date and the correlation between price and renewable infeed forecasts, and rela…
We study price formation in intraday electricity markets in the presence of intermittent renewable generation. We consider the setting where a major producer may interact strategically with a large number of small producers. Using stochastic control theory we identify the optimal strategies of agents with market impact and exhibit the Nash equilibrium in closed form in the asymptotic framework of mean field games with a major player. This is a companion paper to [Féron, Tankov, and Tinsi, Price formation and optimal trading in intraday electricity markets, arXiv:2009.04786, 2020], where a simi…
We consider the problem of optimal trading for a power producer in the context of intraday electricity markets. The aim is to minimize the imbalance cost induced by the random residual demand in electricity, i.e. the consumption from the clients minus the production from renewable energy. For a simple linear price impact model and a quadratic criterion, we explicitly obtain approximate optimal strategies in the intraday market and thermal power generation, and exhibit some remarkable properties of the trading rate. Furthermore, we study the case when there are jumps on the demand forecast and …
We propose the application of a high-speed maximum likelihood clustering algorithm to detect temporal financial market states, using correlation matrices estimated from intraday market microstructure features. We first determine the ex-ante intraday temporal cluster configurations to identify market states, and then study the identified temporal state features to extract state signature vectors which enable online state detection. The state signature vectors serve as low-dimensional state descriptors which can be used in learning algorithms for optimal planning in the high-frequency trading do…
We study the problem of the intraday short-term volume forecasting in cryptocurrency exchange markets. The predictions are built by using transaction and order book data from different markets where the exchange takes place. Methodologically, we propose a temporal mixture ensemble, capable of adaptively exploiting, for the forecasting, different sources of data and providing a volume point estimate, as well as its uncertainty. We provide evidence of the outperformance of our model by comparing its outcomes with those obtained with different time series and machine learning methods. Finally, we…
This research analyses high-frequency data of the cryptocurrency market in regards to intraday trading patterns related to algorithmic trading and its impact on the European cryptocurrency market. We study trading quantitatives such as returns, traded volumes, volatility periodicity, and provide summary statistics of return correlations to CRIX (CRyptocurrency IndeX), as well as respective overall high-frequency based market statistics with respect to temporal aspects. Our results provide mandatory insight into a market, where the grand scale employment of automated trading algorithms and the …
Optimal execution, i.e., the determination of the most cost-effective way to trade volumes in continuous trading sessions, has been a topic of interest in the equity trading world for years. Electricity intraday trading slowly follows this trend but is far from being well-researched. The underlying problem is a very complex one. Energy traders, producers, and electricity wholesale companies receive various position updates from customer businesses, renewable energy production, or plant outages and need to trade these positions in intraday markets. They have a variety of options when it comes t…
The world's stock markets display a strikingly suspicious pattern of overnight and intraday returns. Overnight returns to major stock market indices over the past few decades have been wildly positive, while intraday returns have been disturbingly negative. The cause of these astonishingly consistent return patterns is unknown. We highlight the features of these extraordinary patterns that have hindered the construction of any plausible innocuous explanation. We then use those same features to deduce the only plausible explanation so far advanced for these strikingly suspicious returns.
In this paper, we study the dynamics of absolute return, trading volume and bid-ask spread after the trading halts using high-frequency data from the Shanghai Stock Exchange. We deal with all three types of trading halts, namely intraday halts, one-day halts and inter-day halts, of 203 stocks in Shanghai Stock Exchange from August 2009 to August 2011. We find that absolute return, trading volume, and in case of bid-ask spread around intraday halts share the same pattern with a sharp peak and a power law relaxation after that. While for different types of trading halts, the peaks' height and th…
We consider a 2-dimensional marked Hawkes process with increasing baseline intensity in order to model prices on electricity intraday markets. This model allows to represent different empirical facts such as increasing market activity, random jump sizes but above all microstructure noise through the signature plot. This last feature is of particular importance for practitioners and has not yet been modeled on those particular markets. We provide analytic formulas for first and second moments and for the signature plot, extending the classic results of Bacry et al. (2013) in the context of Hawk…
We discuss a concept denoted as Conformal Prediction (CP) in this paper. While initially stemming from the world of machine learning, it was never applied or analyzed in the context of short-term electricity price forecasting. Therefore, we elaborate the aspects that render Conformal Prediction worthwhile to know and explain why its simple yet very efficient idea has worked in other fields of application and why its characteristics are promising for short-term power applications as well. We compare its performance with different state-of-the-art electricity price forecasting models such as qua…
Both in practice and in the academic literature, models for setting margin requirements in futures markets classically use daily closing price changes. However, as well documented by research on high-frequency data, financial markets have recently shown high intraday volatility, which could bring more risk than expected. This paper tries to answer two questions relevant for margin committees in practice: is it right to compute margin levels based on closing prices and ignoring intraday dynamics? Is it justified to implement intraday margin calls? The paper focuses on the impact of intraday dyn…
Periodic Double Auctions (PDAs) are commonly used in the real world for trading, e.g. in stock markets to determine stock opening prices, and energy markets to trade energy in order to balance net demand in smart grids, involving trillions of dollars in the process. A bidder, participating in such PDAs, has to plan for bids in the current auction as well as for the future auctions, which highlights the necessity of good bidding strategies. In this paper, we perform an equilibrium analysis of single unit single-shot double auctions with a certain clearing price and payment rule, which we refer …
