Deep Reinforcement Learning solutions have been applied to different control problems with outperforming and promising results. In this research work we have applied Proximal Policy Optimization, Soft Actor-Critic and Generative Adversarial Imitation Learning to strategy design problem of three cryptocurrency markets. Our input data includes price data and technical indicators. We have implemented a Gym environment based on cryptocurrency markets to be used with the algorithms. Our test results on unseen data shows a great potential for this approach in helping investors with an expert system to exploit the market and gain profit. Our highest gain for an unseen 66 day span is 4850 US dollars per 10000 US dollars investment. We also discuss on how a specific hyperparameter in the environment design can be used to adjust risk in the generated strategies.
Designing profitable and reliable trading strategies is challenging in the highly volatile cryptocurrency market. Existing works applied deep reinforcement learning methods and optimistically reported increased profits in backtesting, which may suffer from the false positive issue due to overfitting. In this paper, we propose a practical approach to address backtest overfitting for cryptocurrency trading using deep reinforcement learning. First, we formulate the detection of backtest overfitting as a hypothesis test. Then, we train the DRL agents, estimate the probability of overfitting, and r…
K. N. Toosi University of Technology（K. N. 托西理工大学）
AI and data driven solutions have been applied to different fields and achieved outperforming and promising results. In this research work we apply k-Nearest Neighbours, eXtreme Gradient Boosting and Random Forest classifiers for detecting the trend problem of three cryptocurrency markets. We use these classifiers to design a strategy to trade in those markets. Our input data in the experiments include price data with and without technical indicators in separate tests to see the effect of using them. Our test results on unseen data are very promising and show a great potential for this approac…
Portfolio management is the decision-making process of allocating an amount of fund into different financial investment products. Cryptocurrencies are electronic and decentralized alternatives to government-issued money, with Bitcoin as the best-known example of a cryptocurrency. This paper presents a model-less convolutional neural network with historic prices of a set of financial assets as its input, outputting portfolio weights of the set. The network is trained with 0.7 years' price data from a cryptocurrency exchange. The training is done in a reinforcement manner, maximizing the accumul…
BITS Pilani University（pilani大学）；K.K. Birla Goa Campus（K.K.比尔拉果阿校区）；ISI Calcutta（加尔各答印度统计研究所）
In this paper we explore the usage of deep reinforcement learning algorithms to automatically generate consistently profitable, robust, uncorrelated trading signals in any general financial market. In order to do this, we present a novel Markov decision process (MDP) model to capture the financial trading markets. We review and propose various modifications to existing approaches and explore different techniques like the usage of technical indicators, to succinctly capture the market dynamics to model the markets. We then go on to use deep reinforcement learning to enable the agent (the algori…
We consider a trading marketplace that is populated by traders with diverse trading strategies and objectives. The marketplace allows the suppliers to list their goods and facilitates matching between buyers and sellers. In return, such a marketplace typically charges fees for facilitating trade. The goal of this work is to design a dynamic fee schedule for the marketplace that is equitable and profitable to all traders while being profitable to the marketplace at the same time (from charging fees). Since the traders adapt their strategies to the fee schedule, we present a reinforcement learni…
King’s College London（伦敦国王学院）；University of Essex（埃塞克斯大学）；University College London（伦敦大学学院）；Turing Intelligence Technology Limited（图灵智能科技有限公司）
The cryptocurrency market is amongst the fastest-growing of all the financial markets in the world. Unlike traditional markets, such as equities, foreign exchange and commodities, cryptocurrency market is considered to have larger volatility and illiquidity. This paper is inspired by the recent success of using deep learning for stock market prediction. In this work, we analyze and present the characteristics of the cryptocurrency market in a high-frequency setting. In particular, we applied a deep learning approach to predict the direction of the mid-price changes on the upcoming tick. We mon…
This paper presents an agent-based artificial cryptocurrency market in which heterogeneous agents buy or sell cryptocurrencies, in particular Bitcoins. In this market, there are two typologies of agents, Random Traders and Chartists, which interact with each other by trading Bitcoins. Each agent is initially endowed with a finite amount of crypto and/or fiat cash and issues buy and sell orders, according to her strategy and resources. The number of Bitcoins increases over time with a rate proportional to the real one, even if the mining process is not explicitly modelled. The model proposed is…
We study the problem of the intraday short-term volume forecasting in cryptocurrency exchange markets. The predictions are built by using transaction and order book data from different markets where the exchange takes place. Methodologically, we propose a temporal mixture ensemble, capable of adaptively exploiting, for the forecasting, different sources of data and providing a volume point estimate, as well as its uncertainty. We provide evidence of the outperformance of our model by comparing its outcomes with those obtained with different time series and machine learning methods. Finally, we…
This paper sets forth a framework for deep reinforcement learning as applied to market making (DRLMM) for cryptocurrencies. Two advanced policy gradient-based algorithms were selected as agents to interact with an environment that represents the observation space through limit order book data, and order flow arrival statistics. Within the experiment, a forward-feed neural network is used as the function approximator and two reward functions are compared. The performance of each combination of agent and reward function is evaluated by daily and average trade returns. Using this DRLMM framework,…
In this paper we extend the analysis of an agent-based model for adaptive trading, called asynchronous stochastic price pump (ASPP) introduced by Perepelitsa and Timofeyev (2019), to the model with heterogeneous distribution of psychological parameters of speculative optimism and pessimism across the population of traders. We show that the new model has a range of qualitatively different dynamics when the correlation between those factors ranges from low negative to large positive values. A statistical parameter estimation suggests a heterogeneous ASPP with negative correlation as a model of p…
There has been a recent surge in interest in the application of artificial intelligence to automated trading. Reinforcement learning has been applied to single- and multi-instrument use cases, such as market making or portfolio management. This paper proposes a new approach to framing cryptocurrency market making as a reinforcement learning challenge by introducing an event-based environment wherein an event is defined as a change in price greater or less than a given threshold, as opposed to by tick or time-based events (e.g., every minute, hour, day, etc.). Two policy-based agents are traine…
Technical University of Denmark（丹麦技术大学）；City, University of London（伦敦大学城市学院）；Nokia Bell Labs（诺基亚贝尔实验室）；University College London（伦敦大学学院）
Machine learning and AI-assisted trading have attracted growing interest for the past few years. Here, we use this approach to test the hypothesis that the inefficiency of the cryptocurrency market can be exploited to generate abnormal profits. We analyse daily data for $1,681$ cryptocurrencies for the period between Nov. 2015 and Apr. 2018. We show that simple trading strategies assisted by state-of-the-art machine learning algorithms outperform standard benchmarks. Our results show that nontrivial, but ultimately simple, algorithmic mechanisms can help anticipate the short-term evolution of …
Georgia Institute of Technology（佐治亚理工学院）；J.P. Morgan AI Research（摩根大通人工智能研究院）
In finance, the weak form of the Efficient Market Hypothesis asserts that historic stock price and volume data cannot inform predictions of future prices. In this paper we show that, to the contrary, future intra-day stock prices could be predicted effectively until 2009. We demonstrate this using two different profitable machine learning-based trading strategies. However, the effectiveness of both approaches diminish over time, and neither of them are profitable after 2009. We present our implementation and results in detail for the period 2003-2017 and propose a novel idea: the use of such f…
École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）；Imperial College London（伦敦帝国理工学院）；Harvard University（哈佛大学）；UCL Centre for Blockchain Technologies（伦敦大学学院区块链技术中心）；Brave Software（Brave软件公司）
While pump-and-dump schemes have attracted the attention of cryptocurrency observers and regulators alike, this paper represents the first detailed empirical query of pump-and-dump activities in cryptocurrency markets. We present a case study of a recent pump-and-dump event, investigate 412 pump-and-dump activities organized in Telegram channels from June 17, 2018 to February 26, 2019, and discover patterns in crypto-markets associated with pump-and-dump schemes. We then build a model that predicts the pump likelihood of all coins listed in a crypto-exchange prior to a pump. The model exhibits…
Few assets in financial history have been as notoriously volatile as cryptocurrencies. While the long term outlook for this asset class remains unclear, we are successful in making short term price predictions for several major crypto assets. Using historical data from July 2015 to November 2019, we develop a large number of technical indicators to capture patterns in the cryptocurrency market. We then test various classification methods to forecast short-term future price movements based on these indicators. On both PPV and NPV metrics, our classifiers do well in identifying up and down marke…
One of the exciting recent developments in decentralized finance (DeFi) has been the development of decentralized cryptocurrency exchanges that can autonomously handle conversion between different cryptocurrencies. Decentralized exchange protocols such as Uniswap, Curve and other types of Automated Market Makers (AMMs) maintain a liquidity pool (LP) of two or more assets constrained to maintain at all times a mathematical relation to each other, defined by a given function or curve. Examples of such functions are the constant-sum and constant-product AMMs. Existing systems however suffer from …
This research aims to demonstrate a dynamic cointegration-based pairs trading strategy, including an optimal look-back window framework in the cryptocurrency market, and evaluate its return and risk by applying three different scenarios. We employ the Engle-Granger methodology, the Kapetanios-Snell-Shin (KSS) test, and the Johansen test as cointegration tests in different scenarios. We calibrate the mean-reversion speed of the Ornstein-Uhlenbeck process to obtain the half-life used for the asset selection phase and look-back window estimation. By considering the main limitations in the market …
This research analyses high-frequency data of the cryptocurrency market in regards to intraday trading patterns related to algorithmic trading and its impact on the European cryptocurrency market. We study trading quantitatives such as returns, traded volumes, volatility periodicity, and provide summary statistics of return correlations to CRIX (CRyptocurrency IndeX), as well as respective overall high-frequency based market statistics with respect to temporal aspects. Our results provide mandatory insight into a market, where the grand scale employment of automated trading algorithms and the …
We study the dependency and causality structure of the cryptocurrency market investigating collective movements of both prices and social sentiment related to almost two thousand cryptocurrencies traded during the first six months of 2018. This is the first study of the whole cryptocurrency market structure. It introduces several rigorous innovative methodologies applicable to this and to several other complex systems where a large number of variables interact in a non-linear way, which is a distinctive feature of the digital economy. The analysis of the dependency structure reveals that price…
Time-dependent weighted directed networks of cryptocurrency interaction from high-frequency returns
Indian Institute of Science Education and Research Tirupati（印度科学教育研究学院蒂鲁帕蒂分校）；RIKEN Center for Interdisciplinary Theoretical and Mathematical Sciences（理化学研究所跨学科理论与数学科学中心）
Algorithmic trading is well studied in traditional financial markets. However, it has received less attention in centralized cryptocurrency exchanges. The Commodity Futures Trading Commission (CFTC) attributed the $2010$ flash crash, one of the most turbulent periods in the history of financial markets that saw the Dow Jones Industrial Average lose $9\%$ of its value within minutes, to automated order "spoofing" algorithms. In this paper, we build a set of methodologies to characterize and empirically measure different algorithmic trading strategies in Binance, a large centralized cryptocurren…
