\noindent We address the issue of market making on electronic markets when taking into account the clustering and long memory properties of market order flows. We consider a market model with one market maker and order flows driven by general Hawkes processes. We formulate the market maker's objective as a stochastic control problem. We characterize an optimal control by proving existence and uniqueness of a viscosity solution to the associated Hamilton-Jacobi-Bellman equation. Finally we propose a fully consistent numerical method allowing to implement this optimal strategy in practice.
The Invisible Handshake: Persistent Overpricing by Adaptive Market Agents
Department of Computer Science University of Milan（米兰大学计算机科学系）；Institute of Finance University of Lugano（卢加诺大学金融研究所）；LLUI；Department of Computer Science Vienna University of Technology（技术大学维也纳计算机科学系）
Market makers play an essential role in financial markets. A successful market maker should control inventory and adverse selection risks and provide liquidity to the market. As an important methodology in control problems, Reinforcement Learning enjoys the advantage of data-driven and less rigid assumptions, receiving great attention in the market-making field since 2018. However, although the China Commodity market has the biggest trading volume on agricultural products, nonferrous metals, and some other sectors, the study of applying RL to Market Making in China market is still rare. In thi…
Robustness in Sequential Decision Making under Evolving Uncertainty: Evidence from High-Frequency Market Making
National University of Singapore（新加坡国立大学）；Grasshopper Asset Management Pte Ltd（蚱蜢资产管理有限公司）；NUS (Chongqing) Research Institute（新加坡国立大学（重庆）研究院）；Tsinghua University（清华大学）
Market making is one of the most important aspects of algorithmic trading, and it has been studied quite extensively from a theoretical point of view. The practical implementation of so-called "optimal strategies" however suffers from the failure of most order book models to faithfully reproduce the behaviour of real market participants. This paper is twofold. First, some important statistical properties of order driven markets are identified, advocating against the use of purely Markovian order book models. Then, market making strategies are designed and their performances are compared, based…
We present an automated market-making (AMM) cross-settlement mechanism for digital assets on interoperable blockchains, focusing on central bank digital currencies (CBDCs) and stable coins. We develop an innovative approach for generating fair exchange rates for on-chain assets consistent with traditional off-chain markets. We illustrate the efficacy of our approach on realized FX rates for G-10 currencies.
In most OTC markets, a small number of market makers provide liquidity to other market participants. More precisely, for a list of assets, they set prices at which they agree to buy and sell. Market makers face therefore an interesting optimization problem: they need to choose bid and ask prices for making money while mitigating the risk associated with holding inventory in a volatile market. Many market making models have been proposed in the academic literature, most of them dealing with single-asset market making whereas market makers are usually in charge of a long list of assets. The rare…
This paper studies optimal market making for large-tick assets in the presence of latency. We consider a random walk model for the asset price, and formulate the market maker's optimization problem using Markov Decision Processes (MDP). We characterize the value of an order and show that it plays the role of one-period reward in the MDP model. Based on this characterization, we provide explicit criteria for assessing the profitability of market making when there is latency. Under our model, we show that a market maker can earn a positive expected profit if there are sufficient uninformed marke…
We study the relationship between price spread, volatility and trading volume. We find that spread forms as a result of interplay between order liquidity and order impact. When trading volume is small adding more liquidity helps improve price accuracy and reduce spread, but after some point additional liquidity begins to deteriorate price. The model allows to connect the bid-ask spread and high-low bars to measurable microstructural parameters and express their dependence on trading volume, volatility and time horizon. Using the established relations, we address the operating spread optimizati…
University of Zagreb（萨格勒布大学）；Faculty of Electrical Engineering and Computing, University of Zagreb（萨格勒布大学电气工程与计算学院）
The stochastic control problem of optimal market making is among the central problems in quantitative finance. In this paper, a deep reinforcement learning-based controller is trained on a weakly consistent, multivariate Hawkes process-based limit order book simulator to obtain market making controls. The proposed approach leverages the advantages of Monte Carlo backtesting and contributes to the line of research on market making under weakly consistent limit order book models. The ensuing deep reinforcement learning controller is compared to multiple market making benchmarks, with the results…
Given the promising results on joint modeling of SPX/VIX smiles of the recently introduced quadratic rough Heston model, we consider a multi-asset market making problem on SPX and its derivatives, e.g. VIX futures, SPX and VIX options. The market maker tries to maximize its profit from spread capturing while controlling the portfolio's inventory risk, which can be fully explained by the value change of SPX under the particular setting of the quadratic rough Heston model. The high dimensionality of the resulting optimization problem is relaxed by several approximations. An asymptotic closed-for…
In this paper, reinforcement learning is applied to the problem of optimizing market making. A multi-agent reinforcement learning framework is used to optimally place limit orders that lead to successful trades. The framework consists of two agents. The macro-agent optimizes on making the decision to buy, sell, or hold an asset. The micro-agent optimizes on placing limit orders within the limit order book. For the context of this paper, the proposed framework is applied and studied on the Bitcoin cryptocurrency market. The goal of this paper is to show that reinforcement learning is a viable s…
Pari-mutuel markets are trading platforms through which the common market maker simultaneously clears multiple contingent claims markets. This market has several distinctive properties that began attracting the attention of the financial industry in the 2000s. For example, the platform aggregates liquidity from the individual contingent claims market into the common pool while shielding the market maker from potential financial loss. The contribution of this paper is two-fold. First, we provide a new economic interpretation of the market-clearing strategy of a pari-mutuel market that is well k…
Traditional market makers are losing their importance as automated systems have largely assumed the role of liquidity provision in markets. We update the model of Glosten and Milgrom (1985) to analyze this new world: we add multiple securities and introduce an automated market maker who uses the relationships between securities to price order flow. This new automated participant transacts the majority of orders, sets prices that are more efficient, and increases informed and decreases uninformed traders' transaction costs. These results can explain the recent dominance of high frequency tradin…
We develop a new market-making model, from the ground up, which is tailored towards high-frequency trading under a limit order book (LOB), based on the well-known classification of order types in market microstructure. Our flexible framework allows arbitrary order volume, price jump, and bid-ask spread distributions as well as the use of market orders. It also honors the consistency of price movements upon arrivals of different order types. For example, it is apparent that prices should never go down on buy market orders. In addition, it respects the price-time priority of LOB. In contrast to …
Ensuring sufficient liquidity is one of the key challenges for designers of prediction markets. Various market making algorithms have been proposed in the literature and deployed in practice, but there has been little effort to evaluate their benefits and disadvantages in a systematic manner. We introduce a novel experimental design for comparing market structures in live trading that ensures fair comparison between two different microstructures with the same trading population. Participants trade on outcomes related to a two-dimensional random walk that they observe on their computer screens.…
This paper sets forth a framework for deep reinforcement learning as applied to market making (DRLMM) for cryptocurrencies. Two advanced policy gradient-based algorithms were selected as agents to interact with an environment that represents the observation space through limit order book data, and order flow arrival statistics. Within the experiment, a forward-feed neural network is used as the function approximator and two reward functions are compared. The performance of each combination of agent and reward function is evaluated by daily and average trade returns. Using this DRLMM framework,…
In dealer markets, dealers provide prices at which they agree to buy and sell the assets and securities they have in their scope. With ever increasing trading volume, this quoting task has to be done algorithmically in most markets such as foreign exchange markets or corporate bond markets. Over the last ten years, many mathematical models have been designed that can be the basis of quoting algorithms in dealer markets. Nevertheless, in most (if not all) models, the dealer is a pure internalizer, setting quotes and waiting for clients. However, on many dealer markets, dealers also have access …
High-frequency market making is a liquidity-providing trading strategy that simultaneously generates many bids and asks for a security at ultra-low latency while maintaining a relatively neutral position. The strategy makes a profit from the bid-ask spread for every buy and sell transaction, against the risk of adverse selection, uncertain execution and inventory risk. We design realistic simulations of limit order markets and develop a high-frequency market making strategy in which agents process order book information to post the optimal price, order type and execution time. By introducing t…
Managing a book of options on several underlying involves controlling positions of several thousands of financial assets. It is one of the most challenging financial problems involving both pricing and microstructural modeling. An options market maker has to manage both long- and short-dated options having very different dynamics. In particular, short-dated options inventories cannot be managed as a part of an aggregated inventory, which prevents the use of dimensionality reduction techniques such as a factorial approach or first-order Greeks approximation. In this paper, we show that a simple…
University of Liverpool（利物浦大学）；Stratagem Technologies Ltd（斯特拉塔吉姆科技有限公司）；University College London（伦敦大学学院）
Market making is a fundamental trading problem in which an agent provides liquidity by continually offering to buy and sell a security. The problem is challenging due to inventory risk, the risk of accumulating an unfavourable position and ultimately losing money. In this paper, we develop a high-fidelity simulation of limit order book markets, and use it to design a market making agent using temporal-difference reinforcement learning. We use a linear combination of tile codings as a value function approximator, and design a custom reward function that controls inventory risk. We demonstrate t…
There has been a recent surge in interest in the application of artificial intelligence to automated trading. Reinforcement learning has been applied to single- and multi-instrument use cases, such as market making or portfolio management. This paper proposes a new approach to framing cryptocurrency market making as a reinforcement learning challenge by introducing an event-based environment wherein an event is defined as a change in price greater or less than a given threshold, as opposed to by tick or time-based events (e.g., every minute, hour, day, etc.). Two policy-based agents are traine…
Market makers provide liquidity to other market participants: they propose prices at which they stand ready to buy and sell a wide variety of assets. They face a complex optimization problem with both static and dynamic components. They need indeed to propose bid and offer/ask prices in an optimal way for making money out of the difference between these two prices (their bid-ask spread). Since they seldom buy and sell simultaneously, and therefore hold long and/or short inventories, they also need to mitigate the risk associated with price changes, and subsequently skew their quotes dynamicall…
We consider a simple model for the evolution of a limit order book in which limit orders of unit size arrive according to independent Poisson processes. The frequencies of buy limit orders below a given price level, respectively sell limit orders above a given level are described by fixed demand and supply functions. Buy (resp. sell) limit orders that arrive above (resp. below) the current ask (resp. bid) price are converted into market orders. There is no cancellation of limit orders. This model has independently been reinvented by several authors, including Stigler in 1964 and Luckock in 200…
We consider an exchange who wishes to set suitable make-take fees to attract liquidity on its platform. Using a principal-agent approach, we are able to describe in quasi-explicit form the optimal contract to propose to a market maker. This contract depends essentially on the market maker inventory trajectory and on the volatility of the asset. We also provide the optimal quotes that should be displayed by the market maker. The simplicity of our formulas allows us to analyze in details the effects of optimal contracting with an exchange, compared to a situation without contract. We show in par…
We provide an explicit characterization of the optimal market making strategy in a discrete-time Limit Order Book (LOB). In our model, the number of filled orders during each period depends linearly on the distance between the fundamental price and the market maker's limit order quotes, with random slope and intercept coefficients. The high-frequency market maker (HFM) incurs an end-of-the-day liquidation cost resulting from linear price impact. The optimal placement strategy incorporates in a novel and parsimonious way forecasts about future changes in the asset's fundamental price. We show t…
JPMorgan AI Research（摩根大通人工智能研究院）；JPMorgan Quantitative Research（摩根大通量化研究部）
Market makers play an important role in providing liquidity to markets by continuously quoting prices at which they are willing to buy and sell, and managing inventory risk. In this paper, we build a multi-agent simulation of a dealer market and demonstrate that it can be used to understand the behavior of a reinforcement learning (RL) based market maker agent. We use the simulator to train an RL-based market maker agent with different competitive scenarios, reward formulations and market price trends (drifts). We show that the reinforcement learning agent is able to learn about its competitor…
In a semi-realistic market simulator, independent reinforcement learning algorithms may facilitate market makers to maintain wide spreads even without communication. This unexpected outcome challenges the current antitrust law framework. We study the effectiveness of maker-taker fee models in preventing cooperation via algorithms. After modeling market making as a repeated general-sum game, we experimentally show that the relation between net transaction costs and maker rebates is not necessarily monotone. Besides an upper bound on taker fees, we may also need a lower bound on maker rebates to…
In this paper we complete and extend our previous work on stochastic control applied to high frequency market-making with inventory constraints and directional bets. Our new model admits several state variables (e.g. market spread, stochastic volatility and intensities of market orders) provided the full system is Markov. The solution of the corresponding HJB equation is exact in the case of zero inventory risk. The inventory risk enters into play in two ways: a path-dependent penalty based on the volatility and a penalty at expiry based on the market spread. We perform perturbation methods on…
University of Liverpool（利物浦大学）；L3S Research Center（L3S研究中心）；Leibniz University Hannover（汉诺威莱布尼茨大学）
This paper introduces a new representation for the actions of a market maker in an order-driven market. This representation uses scaled beta distributions, and generalises three approaches taken in the artificial intelligence for market making literature: single price-level selection, ladder strategies and "market making at the touch". Ladder strategies place uniform volume across an interval of contiguous prices. Scaled beta distribution based policies generalise these, allowing volume to be skewed across the price interval. We demonstrate that this flexibility is useful for inventory managem…
In this paper we extend the market-making models with inventory constraints of Avellaneda and Stoikov ("High-frequency trading in a limit-order book", Quantitative Finance Vol.8 No.3 2008) and Gueant, Lehalle and Fernandez-Tapia ("Dealing with inventory risk", Preprint 2011) to the case of a rather general class of mid-price processes, under either exponential or linear PNL utility functions, and we add an inventory-risk-aversion parameter that penalises the marker-maker if she finishes her day with a non-zero inventory. This general, non-martingale framework allows a market-maker to make dire…
Since they were authorized by the U.S. Security and Exchange Commission in 1998, electronic exchanges have boomed, and by 2010 high frequency trading accounted for over 70% of equity trades in the US. Such markets are thought to increase liquidity because of the presence of market makers, who are willing to trade as counterparties at any time, in exchange for a fee, the bid-ask spread. In this paper, we propose an equilibrium model showing how such market makers provide liquidity. The model relies on a codebook for client trades, the implied alpha. After solving the individual clients optimiza…
It has been suggested that marked point processes might be good candidates for the modelling of financial high-frequency data. A special class of point processes, Hawkes processes, has been the subject of various investigations in the financial community. In this paper, we propose to enhance a basic zero-intelligence order book simulator with arrival times of limit and market orders following mutually (asymmetrically) exciting Hawkes processes. Modelling is based on empirical observations on time intervals between orders that we verify on several markets (equity, bond futures, index futures). …
We consider the issue of a market maker acting at the same time in the lit and dark pools of an exchange. The exchange wishes to establish a suitable make-take fees policy to attract transactions on its venues. We first solve the stochastic control problem of the market maker without the intervention of the exchange. Then we derive the equations defining the optimal contract to be set between the market maker and the exchange. This contract depends on the trading flows generated by the market maker's activity on the two venues. In both cases, we show existence and uniqueness, in the viscosity …
Crypto-currency market uncertainty drives the need to find adaptive solutions to maximise gain or at least to avoid loss throughout the periods of trading activity. Given the high dimensionality and complexity of the state-action space in this domain, it can be treated as a "Narrow AGI" problem with the scope of goals and environments bound to financial markets. Adaptive Multi-Strategy Agent approach for market-making introduces a new solution to maximise positive "alpha" in long-term handling limit order book (LOB) positions by using multiple sub-agents implementing different strategies with …
University of Southern California（南加州大学）；BITS Pilani, Goa Campus（博拉理工学院果阿校区）
Automated Market Makers (AMMs) have cemented themselves as an integral part of the decentralized finance (DeFi) space. AMMs are a type of exchange that allows users to trade assets without the need for a centralized exchange. They form the foundation for numerous decentralized exchanges (DEXs), which help facilitate the quick and efficient exchange of on-chain tokens. All present-day popular DEXs are static protocols, with fixed parameters controlling the fee and the curvature - they suffer from invariance and cannot adapt to quickly changing market conditions. This characteristic may cause tr…
In corporate bond markets, which are mainly OTC markets, market makers play a central role by providing bid and ask prices for a large number of bonds to asset managers from all around the globe. Determining the optimal bid and ask quotes that a market maker should set for a given universe of bonds is a complex task. Useful models exist, most of them inspired by that of Avellaneda and Stoikov. These models describe the complex optimization problem faced by market makers: proposing bid and ask prices in an optimal way for making money out of the difference between bid and ask prices while mitig…
A large proportion of market making models derive from the seminal model of Avellaneda and Stoikov. The numerical approximation of the value function and the optimal quotes in these models remains a challenge when the number of assets is large. In this article, we propose closed-form approximations for the value functions of many multi-asset extensions of the Avellaneda-Stoikov model. These approximations or proxies can be used (i) as heuristic evaluation functions, (ii) as initial value functions in reinforcement learning algorithms, and/or (iii) directly to design quoting strategies through …
Market makers continuously set bid and ask quotes for the stocks they have under consideration. Hence they face a complex optimization problem in which their return, based on the bid-ask spread they quote and the frequency at which they indeed provide liquidity, is challenged by the price risk they bear due to their inventory. In this paper, we consider a stochastic control problem similar to the one introduced by Ho and Stoll and formalized mathematically by Avellaneda and Stoikov. The market is modeled using a reference price $S_t$ following a Brownian motion with standard deviation $σ$, arr…
