Automated Creation of a High-Performing Algorithmic Trader via Deep Learning on Level-2 Limit Order Book Data
Aaron Wray, Matthew Meades, Dave Cliff
备注：
To be presented at the IEEE Symposium on Computational Intelligence in Financial Engineering (CIFEr2020) in Canberra, Australia, 1-4 December 2020; 8 pages; 4 figures
We present results demonstrating that an appropriately configured deep learning neural network (DLNN) can automatically learn to be a high-performing algorithmic trading system, operating purely from training-data inputs generated by passive observation of an existing successful trader T. That is, we can point our black-box DLNN system at trader T and successfully have it learn from T's trading activity, such that it trades at least as well as T. Our system, called DeepTrader, takes inputs derived from Level-2 market data, i.e. the market's Limit Order Book (LOB) or Ladder for a tradeable asset. Unusually, DeepTrader makes no explicit prediction of future prices. Instead, we train it purely on input-output pairs where in each pair the input is a snapshot S of Level-2 LOB data taken at the time when T issued a quote Q (i.e. a bid or an ask order) to the market; and DeepTrader's desired output is to produce Q when it is shown S. That is, we train our DLNN by showing it the LOB data S that T saw at the time when T issued quote Q, and in doing so our system comes to behave like T, acting as an algorithmic trader issuing specific quotes in response to specific LOB conditions. We train DeepTrader on large numbers of these S/Q snapshot/quote pairs, and then test it in a variety of market scenarios, evaluating it against other algorithmic trading systems in the public-domain literature, including two that have repeatedly been shown to outperform human traders. Our results demonstrate that DeepTrader learns to match or outperform such existing algorithmic trading systems. We analyse the successful DeepTrader network to identify what features it is relying on, and which features can be ignored. We propose that our methods can in principle create an explainable copy of an arbitrary trader T via "black-box" deep learning methods.
We consider a single security market based on a limit order book and two investors, with different speeds of trade execution. If the fast investor can front-run the slower investor, we show that this allows the fast trader to obtain risk free profits, but that these profits cannot be scaled. We derive the fast trader's optimal behaviour when she has only distributional knowledge of the slow trader's actions, with few restrictions on the possible prior distributions. We also consider the slower trader's response to the presence of a fast trader in a market, and the effects of the introduction o…
The modeling of the limit order book is directly related to the assumptions on the behavior of real market participants. This paper is twofold. We first present empirical findings that lay the ground for two improvements to these models.The first one is concerned with market participants by adding the additional dimension of informed market makers, whereas the second, and maybe more original one, addresses the race in the book between informed traders and informed market makers leading to different shapes of the order book. Namely we build an agent-based model for the order book with four type…
There is a pervasive assumption that low latency access to an exchange is a key factor in the profitability of many high-frequency trading strategies. This belief is evidenced by the "arms race" undertaken by certain financial firms to co-locate with exchange servers. To the best of our knowledge, our study is the first to validate and quantify this assumption in a continuous double auction market with a single exchange similar to the New York Stock Exchange. It is not feasible to conduct this exploration with historical data in which trader identity and location are not reported. Accordingly,…
We report successful results from using deep learning neural networks (DLNNs) to learn, purely by observation, the behavior of profitable traders in an electronic market closely modelled on the limit-order-book (LOB) market mechanisms that are commonly found in the real-world global financial markets for equities (stocks & shares), currencies, bonds, commodities, and derivatives. Successful real human traders, and advanced automated algorithmic trading systems, learn from experience and adapt over time as market conditions change; our DLNN learns to copy this adaptive trading behavior. A novel…
Informed traders need to trade fast in order to profit from their private information before it becomes public. Fast electronic markets provide such liquidity. Slow markets provide execution in an auction based trading floor. Hybrid markets combine both execution venues. In its main result, the paper shows that to compensate for their slow and risky executions, trading floors need to be at least twice as deep as the sweeping facility. Furthermore, when a stand-alone trading floor is enhanced with the addition of a sweeping facility, overall informed trading will decline because it is easier fo…
We study the optimal order placement strategy with the presence of a liquidity cost. In this problem, a stock trader wishes to clear her large inventory by a predetermined time horizon $T$. A trader uses both limit and market orders, and a large market order faces an adverse price movement caused by the liquidity risk. First, we study a single period model where the trader places a limit order and/or a market order at the beginning. We show the behavior of optimal amount of market order, $m^*$, and optimal placement of limit order, $y^*$, under different market conditions. Next, we extend it t…
This paper studies four trading algorithms of a professional trader at a multilateral trading facility, observing a realistic two-sided limit order book whose dynamics are driven by the order book events. The identity of the trader can be either internalizing or regular, either a hedge fund or a brokery agency. The speed and cost of trading can be balanced by properly choosing active strategies on the displayed orders in the book and passive strategies on the hidden orders within the spread. We shall show that the price switching algorithms provide lower and upper bounds of the mixed trading a…
An ability to postpone one's execution without penalty provides an important strategic advantage in high-frequency trading. To elucidate competition between traders one has to formulate to a quantitative theory of formation of the execution price from market expectations and quotes. This theory was provided in 2005 by Foucault, Kadan and Kandel. I derive asymptotic distribution of the bids/offers as a function of the ratio of patient and impatient traders using the dynamic version of the Foucault, Kadan and Kandel dynamic Limit Order Book (LOB) model. Dynamic version of the LOB model allows st…
We investigate the statistical properties of the EBS order book for the EUR/USD and USD/JPY currency pairs and the impact of a ten-fold tick size reduction on its dynamics. A large fraction of limit orders are still placed right at or halfway between the old allowed prices. This generates price barriers where the best quotes lie for much of the time, which causes the emergence of distinct peaks in the average shape of the book at round distances. Furthermore, we argue that this clustering is mainly due to manual traders who remained set to the old price resolution. Automatic traders easily tak…
J.P. Morgan AI Research（摩根大通人工智能研究院）；Balyasny Asset Management, L.P.（巴利亚斯尼资产管理有限合伙公司）
Multi-agent market simulators usually require careful calibration to emulate real markets, which includes the number and the type of agents. Poorly calibrated simulators can lead to misleading conclusions, potentially causing severe loss when employed by investment banks, hedge funds, and traders to study and evaluate trading strategies. In this paper, we propose a world model simulator that accurately emulates a limit order book market -- it requires no agent calibration but rather learns the simulated market behavior directly from historical data. Traditional approaches fail short to learn a…
A combination of a priority queueing model and mean field theory shows the emergence of traders' swarm behavior, even when each has a subjective prediction of the market driven by a limit order book. Using a nonlinear Markov model, we analyze the dynamics of traders who select a favorable order price taking into account the waiting cost incurred by others. We find swarm behavior emerges because of the delay in trader reactions to the market, and the direction of the swarm is decided by the current market position and the intensity of zero-intelligent random behavior, rather than subjective tra…
Modern financial exchanges use an electronic limit order book (LOB) to store bid and ask orders for a specific financial asset. As the most fine-grained information depicting the demand and supply of an asset, LOB data is essential in understanding market dynamics. Therefore, realistic LOB simulations offer a valuable methodology for explaining empirical properties of markets. Mainstream simulation models include agent-based models (ABMs) and stochastic models (SMs). However, ABMs tend not to be grounded on real historical data, while SMs tend not to enable dynamic agent-interaction. To overco…
We present a novel approach to describing the microstructure of high frequency trading using two key elements. First we introduce a new notion of informed trader which we starkly contrast to current informed trader models. We describe the exact nature of the `superior information' high frequency traders have access to, and how these agents differ from the more standard `insider traders' described in past papers. This then leads to a model and an empirical analysis of the data which strongly supports our claims. The second key element is a rigorous description of clearing conditions on a limit …
We present a simple order book mechanism that regulates an artificial financial market with self-organized criticality dynamics and fat tails of returns distribution. The model shows the role played by individual imitation in determining trading decisions, while fruitfully replicates typical aggregate market behavior as the "self-fulfilling prophecy". We also address the role of random traders as a possible decentralized solution to dampen market fluctuations.
Recently, most stock exchanges in the U.S. employ maker-taker fees, in which an exchange pays rebates to traders placing orders in the order book and charges fees to traders taking orders from the order book. Maker-taker fees encourage traders to place many orders that provide market liquidity to the exchange. However, it is not clear how maker-taker fees affect the total cost of a taking order, including all the charged fees and the market impact. In this study, we investigated the effect of maker-taker fees on the total cost of a taking order with our artificial market model, which is an age…
In recent years, machine learning has become prevalent in numerous tasks, including algorithmic trading. Stock market traders utilize machine learning models to predict the market's behavior and execute an investment strategy accordingly. However, machine learning models have been shown to be susceptible to input manipulations called adversarial examples. Despite this risk, the trading domain remains largely unexplored in the context of adversarial learning. In this study, we present a realistic scenario in which an attacker influences algorithmic trading systems by using adversarial learning …
We build an agent-based model to study how the interplay between low- and high-frequency trading affects asset price dynamics. Our main goal is to investigate whether high-frequency trading exacerbates market volatility and generates flash crashes. In the model, low-frequency agents adopt trading rules based on chronological time and can switch between fundamentalist and chartist strategies. On the contrary, high-frequency traders activation is event-driven and depends on price fluctuations. High-frequency traders use directional strategies to exploit market information produced by low-frequen…
We study the optimal placement problem of a stock trader who wishes to clear his/her inventory by a predetermined time horizon t, by using a limit order or a market order. For a diffusive market, we characterize the optimal limit order placement policy and analyze its behavior under different market conditions. In particular, we show that, in the presence of a negative drift, there exists a critical time t0>0 such that, for any time horizon t>t0, there exists an optimal placement, which, contrary to earlier work, is different from one that is placed "infinitesimally" close to the best ask, suc…
This paper deals with a stochastic order-driven market model with waiting costs, for order books with heterogenous traders. Offer and demand of liquidity drives price formation and traders anticipate future evolutions of the order book. The natural framework we use is mean field game theory, a class of stochastic differential games with a continuum of anonymous players. Several sources of heterogeneity are considered including the mean size of orders. Thus we are able to consider the coexistence of Institutional Investors and High Frequency Traders (HFT). We provide both analytical solutions a…
In this paper we develop a model of an order-driven market where traders set bids and asks and post market or limit orders according to exogenously fixed rules. Agents are assumed to have three components to the expectation of future asset returns, namely-fundamentalist, chartist and noise trader. Furthermore agents differ in the characteristics describing these components, such as time horizon, risk aversion and the weights given to the various components. The model developed here extends a great deal of earlier literature in that the order submissions of agents are determined by utility maxi…
A microscopic model is established for financial Brownian motion from the direct observation of the dynamics of high-frequency traders (HFTs) in a foreign exchange market. Furthermore, a theoretical framework parallel to molecular kinetic theory is developed for the systematic description of the financial market from microscopic dynamics of HFTs. We report first on a microscopic empirical law of traders' trend-following behavior by tracking the trajectories of all individuals, which quantifies the collective motion of HFTs but has not been captured in conventional order-book models. We next in…
The Tick library allows researchers in market microstructure to simulate and learn Hawkes process in high-frequency data, with optimized parametric and non-parametric learners. But one challenge is to take into account the correct causality of order book events considering latency: the only way one order book event can influence another is if the time difference between them (by the central order book timestamps) is greater than the minimum amount of time for an event to be (i) published in the order book, (ii) reach the trader responsible for the second event, (iii) influence the decision (pr…
High Frequency Trading (HFT) represents an ever growing proportion of all financial transactions as most markets have now switched to electronic order book systems. The main goal of the paper is to propose continuous time equations which generalize the self-financing relationships of frictionless markets to electronic markets with limit order books. We use NASDAQ ITCH data to identify significant empirical features such as price impact and recovery, rough paths of inventories and vanishing bid-ask spreads. Starting from these features, we identify microscopic identities holding on the trade cl…
We analyze a tractable model of a limit order book on short time scales, where the dynamics are driven by stochastic fluctuations between supply and demand. We establish the existence of a limiting distribution for the highest bid, and for the lowest ask, where the limiting distributions are confined between two thresholds. We make extensive use of fluid limits in order to establish recurrence properties of the model. We use the model to analyze various high-frequency trading strategies, and comment on the Nash equilibria that emerge between high-frequency traders when a market in continuous t…
We show that the statistics of spreads in real order books is characterized by an intrinsic asymmetry due to discreteness effects for even or odd values of the spread. An analysis of data from the NYSE order book points out that traders' strategies contribute to this asymmetry. We also investigate this phenomenon in the framework of a microscopic model and, by introducing a non-uniform deposition mechanism for limit orders, we are able to quantitatively reproduce the asymmetry found in the experimental data. Simulations of our model also show a realistic dynamics with a sort of intermittent be…
The two-body stochastic dealer model is revisited to provide an exact solution to the average order-book profile using the kinetic approach. The dealer model is a microscopic financial model where individual traders make decisions on limit-order prices stochastically and then reach agreements on transactions. In the literature, this model was solved for several cases: an exact solution for two-body traders $N=2$ and a mean-field solution for many traders $N\gg 1$. Remarkably, while kinetic theory plays a significant role in the mean-field analysis for $N\gg 1$, its role is still elusive for th…
We describe a bottom-up framework, based on the identification of appropriate order parameters and determination of phase diagrams, for understanding progressively refined agent-based models and simulations of financial markets. We illustrate this framework by starting with a deterministic toy model, whereby $N$ independent traders buy and sell $M$ stocks through an order book that acts as a clearing house. The price of a stock increases whenever it is bought and decreases whenever it is sold. Price changes are updated by the order book before the next transaction takes place. In this determin…
Market making is one of the most important aspects of algorithmic trading, and it has been studied quite extensively from a theoretical point of view. The practical implementation of so-called "optimal strategies" however suffers from the failure of most order book models to faithfully reproduce the behaviour of real market participants. This paper is twofold. First, some important statistical properties of order driven markets are identified, advocating against the use of purely Markovian order book models. Then, market making strategies are designed and their performances are compared, based…
In order-driven markets, limit-order book (LOB) resiliency is an important microscopic indicator of market quality when the order book is hit by a liquidity shock and plays an essential role in the design of optimal submission strategies of large orders. However, the evolutionary behavior of LOB resilience around liquidity shocks is not well understood empirically. Using order flow data sets of Chinese stocks, we quantify and compare the LOB dynamics characterized by the bid-ask spread, the LOB depth and the order intensity surrounding effective market orders with different aggressiveness. We …
Algorithmic stock trading has become a staple in today's financial market, the majority of trades being now fully automated. Deep Reinforcement Learning (DRL) agents proved to be to a force to be reckon with in many complex games like Chess and Go. We can look at the stock market historical price series and movements as a complex imperfect information environment in which we try to maximize return - profit and minimize risk. This paper reviews the progress made so far with deep reinforcement learning in the subdomain of AI in finance, more precisely, automated low-frequency quantitative stock …
We consider issues of time in automated trading strategies in simulated financial markets containing a single exchange with public limit order book and continuous double auction matching. In particular, we explore two effects: (i) reaction speed - the time taken for trading strategies to calculate a response to market events; and (ii) trading urgency - the sensitivity of trading strategies to approaching deadlines. Much of the literature on trading agents focuses on optimising pricing strategies only and ignores the effects of time, while real-world markets continue to experience a race to zer…
We propose a microstructural modeling framework for studying optimal market making policies in a FIFO (first in first out) limit order book (LOB). In this context, the limit orders, market orders, and cancel orders arrivals in the LOB are modeled as Cox point processes with intensities that only depend on the state of the LOB. These are high-dimensional models which are realistic from a micro-structure point of view and have been recently developed in the literature. In this context, we consider a market maker who stands ready to buy and sell stock on a regular and continuous basis at a public…
Traders in a stock market exchange stock shares and form a stock trading network. Trades at different positions of the stock trading network may contain different information. We construct stock trading networks based on the limit order book data and classify traders into $k$ classes using the $k$-shell decomposition method. We investigate the influences of trading behaviors on the price impact by comparing a closed national market (A-shares) with an international market (B-shares), individuals and institutions, partially filled and filled trades, buyer-initiated and seller-initiated trades, a…
We propose a framework to study optimal trading policies in a one-tick pro-rata limit order book, as typically arises in short-term interest rate futures contracts. The high-frequency trader has the choice to trade via market orders or limit orders, which are represented respectively by impulse controls and regular controls. We model and discuss the consequences of the two main features of this particular microstructure: first, the limit orders sent by the high frequency trader are only partially executed, and therefore she has no control on the executed quantity. For this purpose, cumulative …
Optimal execution of portfolio transactions is the essential part of algorithmic trading. In this paper we present in simple analytical form the optimal trajectory for risk-averse trader with the assumption of exponential market recovery and short-time investment horizon.
We present an agent based model of a single asset financial market that is capable of replicating several non-trivial statistical properties observed in real financial markets, generically referred to as stylized facts. While previous models reported in the literature are also capable of replicating some of these statistical properties, in general, they tend to oversimplify either the trading mechanisms or the behavior of the agents. In our model, we strived to capture the most important characteristics of both aspects to create agents that employ strategies inspired on those used in real mark…
Order positions are key variables in algorithmic trading. This paper studies the limiting behavior of order positions and related queues in a limit order book. In addition to the fluid and diffusion limits for the processes, fluctuations of order positions and related queues around their fluid limits are analyzed. As a corollary, explicit analytical expressions for various quantities of interests in a limit order book are derived.
The algorithmic trading comes from digitalisation of the processing of trading assets on financial markets. Since 1980 the computerization of the stock market offers real time processing of financial information. This technological revolution has offered processes and mathematic methods to identify best return on transactions. Current research relates to autonomous transaction systems programmed in certain periods and some algorithms. This offers return opportunities where traders can not intervene. There are about thirty algorithms to assist the traders, the best known are the VWAP, the TWAP,…
University of Toronto（多伦多大学）；Department of Computer Science（计算机科学系）
With the proliferation of algorithmic high-frequency trading in financial markets, the Limit Order Book has generated increased research interest. Research is still at an early stage and there is much we do not understand about the dynamics of Limit Order Books. In this paper, we employ a machine learning approach to investigate Limit Order Book features and their potential to predict short term price movements. This is an initial broad-based investigation that results in some novel observations about LOB dynamics and identifies several promising directions for further research. Furthermore, w…
This work's purpose is to understand the dynamics of limit order books in order-driven markets. We try to illustrate a dynamical trading mechanism attached to the microstructure of limit order markets. We capture the iterative nature of trading processes, which is critical in the dynamics of bid-ask pairs and the switching laws between different traders' types and their orders. In general, after introducing the atomic trading scheme, we study a general iterated trading process in both combinatorial and stochastic ways, and state a few results on the stability of a dynamical trading system. We …
This paper describes the design, implementation, and successful use of the Bristol Stock Exchange (BSE), a novel minimal simulation of a centralised financial market, based on a Limit Order Book (LOB) such as is common in major stock exchanges. Construction of BSE was motivated by the fact that most of the world's major financial markets have automated, with trading activity that previously was the responsibility of human traders now being performed by high-speed autonomous automated trading systems. Research aimed at understanding the dynamics of this new style of financial market is hampered…
