Unfair stock trading strategies have been shown to be one of the most negative perceptions that customers can have concerning trading and may result in long-term losses for a company. Investment banks usually place trading orders for multiple clients with the same target assets but different order sizes and diverse requirements such as time frame and risk aversion level, thereby total earning and individual earning cannot be optimized at the same time. Orders executed earlier would affect the market price level, so late execution usually means additional implementation cost. In this paper, we propose a novel scheme that utilizes multi-agent reinforcement learning systems to derive stock trading strategies for all clients which keep a balance between revenue and fairness. First, we demonstrate that Reinforcement learning (RL) is able to learn from experience and adapt the trading strategies to the complex market environment. Secondly, we show that the Multi-agent RL system allows developing trading strategies for all clients individually, thus optimizing individual revenue. Thirdly, we use the Generalized Gini Index (GGI) aggregation function to control the fairness level of the revenue across all clients. Lastly, we empirically demonstrate the superiority of the novel scheme in improving fairness meanwhile maintaining optimization of revenue.
IITB-Monash Research Academy（IITB-莫纳什研究院）；IIT Bombay（印度理工学院孟买分校）；Monash University（莫纳什大学）
In stock trading, feature extraction and trading strategy design are the two important tasks to achieve long-term benefits using machine learning techniques. Several methods have been proposed to design trading strategy by acquiring trading signals to maximize the rewards. In the present paper the theory of deep reinforcement learning is applied for stock trading strategy and investment decisions to Indian markets. The experiments are performed systematically with three classical Deep Reinforcement Learning models Deep Q-Network, Double Deep Q-Network and Dueling Double Deep Q-Network on ten I…
Columbia University（哥伦比亚大学）；Nokia-Bell Labs（诺基亚贝尔实验室）
Stock trading strategy plays a crucial role in investment companies. However, it is challenging to obtain optimal strategy in the complex and dynamic stock market. We explore the potential of deep reinforcement learning to optimize stock trading strategy and thus maximize investment return. 30 stocks are selected as our trading stocks and their daily prices are used as the training and trading market environment. We train a deep reinforcement learning agent and obtain an adaptive trading strategy. The agent's performance is evaluated and compared with Dow Jones Industrial Average and the tradi…
Columbia University（哥伦比亚大学）；AI4Finance LLC.（AI4Finance有限责任公司）；Ion Media Networks（Ion媒体网络公司）；Imperial College（帝国理工学院）；New York University (Shanghai)（上海纽约大学）
As deep reinforcement learning (DRL) has been recognized as an effective approach in quantitative finance, getting hands-on experiences is attractive to beginners. However, to train a practical DRL trading agent that decides where to trade, at what price, and what quantity involves error-prone and arduous development and debugging. In this paper, we introduce a DRL library FinRL that facilitates beginners to expose themselves to quantitative finance and to develop their own stock trading strategies. Along with easily-reproducible tutorials, FinRL library allows users to streamline their own de…
A Deep Reinforcement Learning Approach to Automated Stock Trading, using xLSTM Networks
Faculty of Computer Science and Engineering, Shahid Beheshti University（谢赫·贝赫什提大学计算机科学与工程学院）；Faculty of Electrical Engineering, Shahid Beheshti University（谢赫·贝赫什提大学电气工程学院）
University of Electronic Science and Technology of China（电子科技大学）；Harbin Institute of Technology（哈尔滨工业大学）
It is a difficult task for both professional investors and individual traders continuously making profit in stock market. With the development of computer science and deep reinforcement learning, Buy\&Hold (B\&H) has been oversteped by many artificial intelligence trading algorithms. However, the information and process are not enough, which limit the performance of reinforcement learning algorithms. Thus, we propose a parallel-network continuous quantitative trading model with GARCH and PPO to enrich the basical deep reinforcement learning model, where the deep learning parallel network layer…
Deep Reinforcement Learning (DRL) algorithms can scale to previously intractable problems. The automation of profit generation in the stock market is possible using DRL, by combining the financial assets price "prediction" step and the "allocation" step of the portfolio in one unified process to produce fully autonomous systems capable of interacting with their environment to make optimal decisions through trial and error. This work represents a DRL model to generate profitable trades in the stock market, effectively overcoming the limitations of supervised learning approaches. We formulate th…
Algorithmic stock trading has become a staple in today's financial market, the majority of trades being now fully automated. Deep Reinforcement Learning (DRL) agents proved to be to a force to be reckon with in many complex games like Chess and Go. We can look at the stock market historical price series and movements as a complex imperfect information environment in which we try to maximize return - profit and minimize risk. This paper reviews the progress made so far with deep reinforcement learning in the subdomain of AI in finance, more precisely, automated low-frequency quantitative stock …
Schumpeter School of Business and Economics（熊彼特商业与经济学院）；University of Wuppertal（伍珀塔尔大学）
This paper proposes a Deep Reinforcement Learning algorithm for financial portfolio trading based on Deep Q-learning. The algorithm is capable of trading high-dimensional portfolios from cross-sectional datasets of any size which may include data gaps and non-unique history lengths in the assets. We sequentially set up environments by sampling one asset for each environment while rewarding investments with the resulting asset's return and cash reservation with the average return of the set of assets. This enforces the agent to strategically assign capital to assets that it predicts to perform …
This scientific research paper presents an innovative approach based on deep reinforcement learning (DRL) to solve the algorithmic trading problem of determining the optimal trading position at any point in time during a trading activity in stock markets. It proposes a novel DRL trading strategy so as to maximise the resulting Sharpe ratio performance indicator on a broad range of stock markets. Denominated the Trading Deep Q-Network algorithm (TDQN), this new trading strategy is inspired from the popular DQN algorithm and significantly adapted to the specific algorithmic trading problem at ha…
The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））
In recent years, many practitioners in quantitative finance have attempted to use Deep Reinforcement Learning (DRL) to build better quantitative trading (QT) strategies. Nevertheless, many existing studies fail to address several serious challenges, such as the non-stationary financial environment and the bias and variance trade-off when applying DRL in the real financial market. In this work, we proposed Safe-FinRL, a novel DRL-based high-freq stock trading strategy enhanced by the near-stationary financial environment and low bias and variance estimation. Our main contributions are twofold: …
Portfolio management via reinforcement learning is at the forefront of fintech research, which explores how to optimally reallocate a fund into different financial assets over the long term by trial-and-error. Existing methods are impractical since they usually assume each reallocation can be finished immediately and thus ignoring the price slippage as part of the trading cost. To address these issues, we propose a hierarchical reinforced stock trading system for portfolio management (HRPM). Concretely, we decompose the trading process into a hierarchy of portfolio management over trade execut…
Price movement prediction has always been one of the traders' concerns in financial market trading. In order to increase their profit, they can analyze the historical data and predict the price movement. The large size of the data and complex relations between them lead us to use algorithmic trading and artificial intelligence. This paper aims to offer an approach using Twin-Delayed DDPG (TD3) and the daily close price in order to achieve a trading strategy in the stock and cryptocurrency markets. Unlike previous studies using a discrete action space reinforcement learning algorithm, the TD3 i…
Traders in a stock market exchange stock shares and form a stock trading network. Trades at different positions of the stock trading network may contain different information. We construct stock trading networks based on the limit order book data and classify traders into $k$ classes using the $k$-shell decomposition method. We investigate the influences of trading behaviors on the price impact by comparing a closed national market (A-shares) with an international market (B-shares), individuals and institutions, partially filled and filled trades, buyer-initiated and seller-initiated trades, a…
In this study, we investigate the statistical properties of the returns and the trading volume. We show a typical example of power-law distributions of the return and of the trading volume. Next, we propose an interacting agent model of stock markets inspired from statistical mechanics [24] to explore the empirical findings. We show that as the interaction among the interacting traders strengthens both the returns and the trading volume present power-law behavior.
When AI Meets Finance (StockAgent): Large Language Model-based Stock Trading in Simulated Real-world Environments
University of Liverpool（利物浦大学）；Peking University（北京大学）；Shanghai University of Finance（上海金融学院）；Rutgers University（罗格斯大学）；University of Michigan（密歇根大学）；Northwestern University（西北大学）；New York University（纽约大学）；Xi’an Jiaotong-Liverpool University（西安交通大学利物浦大学）；New Jersey Institute of Technology（新泽西理工学院）
We empirically study the market impact of trading orders. We are specifically interested in large trading orders that are executed incrementally, which we call hidden orders. These are reconstructed based on information about market member codes using data from the Spanish Stock Market and the London Stock Exchange. We find that market impact is strongly concave, approximately increasing as the square root of order size. Furthermore, as a given order is executed, the impact grows in time according to a power-law; after the order is finished, it reverts to a level of about 0.5-0.7 of its value …
We extend the framework of trading strategies of Gatheral [2010] from single stocks to a pair of stocks. Our trading strategy with the executions of two round-trip trades can be described by the trading rates of the paired stocks and the ratio of their trading periods. By minimizing the potential cost arising from cross-impacts, i.e., the price change of one stock due to the trades of another stock, we can find out an optimal strategy for executing a sequence of trades from different stocks. We further apply the model of the strategy to a specific case, where we quantify the cross-impacts of t…
This article comes up with an intraday trading strategy under T+1 using Markowitz optimization and Multilayer Perceptron (MLP) with published stock data obtained from the Shenzhen Stock Exchange and Shanghai Stock Exchange. The empirical results reveal the profitability of Markowitz portfolio optimization and validate the intraday stock price prediction using MLP. The findings further combine the Markowitz optimization, an MLP with the trading strategy, to clarify this strategy's feasibility.
Studying the micro-trading behaviors before stock price jumps is an important problem for financial regulations and investment decisions. In this study, we provide a new framework to study pre-jump trading behaviors based on multivariate time series analysis. Different from the existing literature, our methodology takes into account the temporal information embedded in the trading-related attributes and can better evaluate and compare the abnormality levels of different attributes. Moreover, it can explore the joint informativeness of the attributes as well as select a subset of highly informa…
In this paper, we propose stock trading based on the average tax basis. Recall that when selling stocks, capital gain should be taxed while capital loss can earn certain tax rebate. We learn the optimal trading strategies with and without considering taxes by reinforcement learning. The result shows that tax ignorance could induce more than 62% loss on the average portfolio returns, implying that taxes should be embedded in the environment of continuous stock trading on AI platforms.
Although technical trading rules have been widely used by practitioners in financial markets, their profitability still remains controversial. We here investigate the profitability of moving average (MA) and trading range break (TRB) rules by using the Shanghai Stock Exchange Composite Index (SHCI) from May 21, 1992 through December 31, 2013 and Shenzhen Stock Exchange Composite Index (SZCI) from April 3, 1991 through December 31, 2013. The $t$-test is adopted to check whether the mean returns which are conditioned on the trading signals are significantly different from unconditioned returns a…
This study first reconstructs three deep learning powered stock trading models and their associated strategies that are representative of distinct approaches to the problem and established upon different aspects of the many theories evolved around deep learning. It then seeks to compare the performance of these strategies from different perspectives through trading simulations ran on three scenarios when the benchmarks are kept at historical low points for extended periods of time. The results show that in extremely adverse market climates, investment portfolios managed by deep learning powere…
In this paper, we study the dynamics of absolute return, trading volume and bid-ask spread after the trading halts using high-frequency data from the Shanghai Stock Exchange. We deal with all three types of trading halts, namely intraday halts, one-day halts and inter-day halts, of 203 stocks in Shanghai Stock Exchange from August 2009 to August 2011. We find that absolute return, trading volume, and in case of bid-ask spread around intraday halts share the same pattern with a sharp peak and a power law relaxation after that. While for different types of trading halts, the peaks' height and th…
Manipulation is an important issue for both developed and emerging stock markets. For the study of manipulation, it is critical to analyze investor behavior in the stock market. In this paper, an analysis of the full transaction records of over a hundred stocks in a one-year period is conducted. For each stock, a trading network is constructed to characterize the relations among its investors. In trading networks, nodes represent investors and a directed link connects a stock seller to a buyer with the total trade size as the weight of the link, and the node strength is the sum of all edge wei…
In this report, we talked about a new quantitative strategy for choosing the optimal(s) stock(s) to trade. The basic notions are generally very known by the financial community. The key here is to understand 1) the standard score applied to a sample and 2) the correlation factor applied to different time series in real life. These notions are the core of our research. We are going to begin with the introduction section. In this part, we talked about variance, covariance, correlation factor, daily returns in stock trading and the Shapiro-Wilk test to test the normality of a time serie. Next to …
Peking University（北京大学）；Beijing Language and Culture University（北京语言大学）；Mizuho Securities Co., Ltd（瑞穗证券股份有限公司）
Volume prediction is one of the fundamental objectives in the Fintech area, which is helpful for many downstream tasks, e.g., algorithmic trading. Previous methods mostly learn a universal model for different stocks. However, this kind of practice omits the specific characteristics of individual stocks by applying the same set of parameters for different stocks. On the other hand, learning different models for each stock would face data sparsity or cold start problems for many stocks with small capitalization. To take advantage of the data scale and the various characteristics of individual st…
Geometric phases describe how in a continuous-time dynamical system the displacement of a variable (called phase variable) can be related to other variables (shape variables) undergoing a cyclic motion, according to an area rule. The aim of this paper is to show that geometric phases can exist also for discrete-time systems, and even when the cycles in shape space have zero area. A context in which this principle can be applied is stock trading. A zero-area cycle in shape space represents the type of trading operations normally carried out by high-frequency traders (entering and exiting a posi…
In this paper we use fuzzy systems theory to convert the technical trading rules commonly used by stock practitioners into excess demand functions which are then used to drive the price dynamics. The technical trading rules are recorded in natural languages where fuzzy words and vague expressions abound. In Part I of this paper, we will show the details of how to transform the technical trading heuristics into nonlinear dynamic equations. First, we define fuzzy sets to represent the fuzzy terms in the technical trading rules; second, we translate each technical trading heuristic into a group o…
Just as war is sometimes fallaciously represented as a zero sum game -- when in fact war is a negative sum game - stock market trading, a positive sum game over time, is often erroneously represented as a zero sum game. This is called the "zero sum fallacy" -- the erroneous belief that one trader in a stock market exchange can only improve their position provided some other trader's position deteriorates. However, a positive sum game in absolute terms can be recast as a zero sum game in relative terms. Similarly it appears that negative sum games in absolute terms have been recast as zero sum …
We carry out a large-scale empirical data analysis to examine the efficiency of the so-called pairs trading. On the basis of relevant three thresholds, namely, starting, profit-taking, and stop-loss for the `first-passage process' of the spread (gap) between two highly-correlated stocks, we construct an effective strategy to make a trade via `active' stock-pairs automatically. The algorithm is applied to $1,784$ stocks listed on the first section of the Tokyo Stock Exchange leading up to totally $1,590,436$ pairs. We are numerically confirmed that the asset management by means of the pairs tra…
The imbalance of buying and selling functions profoundly in the formation of market trends, however, a fine-granularity investigation of the imbalance is still missing. This paper investigates a unique transaction dataset that enables us to inspect the imbalance of buying and selling on the man-times level at high frequency, what we call 'trading polarity', for a large cross-section of stocks from Shenzhen Stock Exchange. The trading polarity measures the market sentiment toward stocks from a view of very essence of trading desire. When using the polarity to examine market crash, we find that …
The dynamics of a stock market with heterogeneous agents is discussed in the framework of a recently proposed spin model for the emergence of bubbles and crashes. We relate the log returns of stock prices to magnetization in the model and find that it is closely related to trading volume as observed in real markets. The cumulative distribution of log returns exhibits scaling with exponents steeper than 2 and scaling is observed in the distribution of transition times between bull and bear markets.
Traditional knowledge distillation in classification problems transfers the knowledge via class correlations in the soft label produced by teacher models, which are not available in regression problems like stock trading volume prediction. To remedy this, we present a novel distillation framework for training a light-weight student model to perform trading volume prediction given historical transaction data. Specifically, we turn the regression model into a probabilistic forecasting model, by training models to predict a Gaussian distribution to which the trading volume belongs. The student mo…
In Part III of this study, we apply the price dynamical model with big buyers and big sellers developed in Part I of this paper to the daily closing prices of the top 20 banking and real estate stocks listed in the Hong Kong Stock Exchange. The basic idea is to estimate the strength parameters of the big buyers and the big sellers in the model and make buy/sell decisions based on these parameter estimates. We propose two trading strategies: (i) Follow-the-Big-Buyer which buys when big buyer begins to appear and there is no sign of big sellers, holds the stock as long as the big buyer is still …
Technical trading represents a class of investment strategies for Financial Markets based on the analysis of trends and recurrent patterns of price time series. According standard economical theories these strategies should not be used because they cannot be profitable. On the contrary it is well-known that technical traders exist and operate on different time scales. In this paper we investigate if technical trading produces detectable signals in price time series and if some kind of memory effect is introduced in the price dynamics. In particular we focus on a specific figure called supports…
Our main task is to study the effect of corporate governance on the market liquidity of listed companies' stocks. We establish a theoretical model that contains the heterogeneity of investors' beliefs to explain the mechanisms by which corporate governance improves liquidity of the corporate stocks. In this process we found that the existence of noise traders who are semi-informed in the market is an important condition for corporate governance to have the effect of improving liquidity of the stocks. We further find that the strength of this effect is affected by the degree of noise traders' p…
We study trade-based manipulation of stock prices from the perspective of complex trading networks constructed by using detailed information of trades. A stock trading network consists of nodes and directed links, where every trader is a node and a link is formed from one trader to the other if the former sells shares to the latter. Specifically, three abnormal network motifs are investigated, which are found to be formed by a few traders, implying potential intention of price manipulation. We further investigate the dynamics of volatility, trading volume, average trade size and turnover aroun…
We investigate the trading behavior of Finnish individual investors trading the stocks selected to compute the OMXH25 index in 2003 by tracking the individual daily investment decisions. We verify that the set of investors is a highly heterogeneous system under many aspects. We introduce a correlation based method that is able to detect a hierarchical structure of the trading profiles of heterogeneous individual investors. We verify that the detected hierarchical structure is highly overlapping with the cluster structure obtained with the approach of statistically validated networks when an ap…
Quantitative investment is a fundamental financial task that highly relies on accurate stock prediction and profitable investment decision making. Despite recent advances in deep learning (DL) have shown stellar performance on capturing trading opportunities in the stochastic stock market, we observe that the performance of existing DL methods is sensitive to random seeds and network initialization. To design more profitable DL methods, we analyze this phenomenon and find two major limitations of existing works. First, there is a noticeable gap between accurate financial predictions and profit…
We seek a discussion about the most suitable feedback control structure for stock trading under the consideration of proportional transaction costs. Suitability refers to robustness and performance capability. Both are tested by considering different one-step ahead prediction qualities, including the ideal case, correct prediction of the direction of change in daily stock prices and the worst-case. Feedback control structures are partitioned into two general classes: stochastic model predictive control (SMPC) and genetic. For the former class three controllers are discussed, whereby it is dist…
In this article, the long-term behavior of the stock market index of the New York Stock Exchange is studied, for the period 1950 to 2013. Specifically, the CRSP Value-Weighted and CRSP Equal-Weighted index are analyzed in terms of market efficiency, using the standard ratio variance test, considering over 1600 one week rolling windows. For the equally weighted index, the null hypothesis of random walk is rejected in the whole period, while for the weighted market value index, the null hypothesis start to be accepted from the 1990s. In order to explain this difference, we raised the hypothesis …
Using a relationship between the moments of the probability distribution of times between the two consecutive trades (intertrade time distribution) and the moments of the distribution of a daily number of trades we show, that the underlying point process is essentially non-markovian. A detailed analysis of all trades in the EESR stock on the Moscow International Currency Exchange in the period January 2003 - September 2003, including that of correlation between intertrade time intervals is presented. A power-law decay of the correlation provides an additional evidence of the long-memory nature…
Crowded trades by similarly trading peers influence the dynamics of asset prices, possibly creating systemic risk. We propose a market clustering measure using granular trading data. For each stock the clustering measure captures the degree of trading overlap among any two investors in that stock. We investigate the effect of crowded trades on stock price stability and show that market clustering has a causal effect on the properties of the tails of the stock return distribution, particularly the positive tail, even after controlling for commonly considered risk drivers. Reduced investor pool …
In this paper, we build on the analysis of Muni Toke and Yoshida (2020) and conduct several empirical studies using high-frequency financial data. Muni Toke and Yoshida (2020) showed the consistency and asymptotic behavior of the Cox-type model estimators for relative intensities of orders in the limit order book, and then by using high-frequency trading data for 36 stocks traded on the Paris Stock Exchange, they carry out model selection and trading sign prediction. In this study, we add new covariates and carry out model selection and trading sign prediction using high-frequency trading data…
In this paper, a neural network-based stock price prediction and trading system using technical analysis indicators is presented. The model developed first converts the financial time series data into a series of buy-sell-hold trigger signals using the most commonly preferred technical analysis indicators. Then, a Multilayer Perceptron (MLP) artificial neural network (ANN) model is trained in the learning stage on the daily stock prices between 1997 and 2007 for all of the Dow30 stocks. Apache Spark big data framework is used in the training stage. The trained model is then tested with data fr…
The asymmetric price impact between the institutional purchases and sales of 32 liquid stocks in Chinese stock markets in year 2003 is carefully studied. We analyze the price impact in both drawup and drawdown trends with consecutive positive and negative daily price changes, and test the dependence of the price impact asymmetry on the market condition. For most of the stocks institutional sales have a larger price impact than institutional purchases, and larger impact of institutional purchases only exists in few stocks with primarily increasing tendencies. We further study the mean return of…
University of Toronto（多伦多大学）；Technion - Israel Institute of Technology（以色列理工学院）
A novel algorithm for actively trading stocks is presented. While traditional expert advice and "universal" algorithms (as well as standard technical trading heuristics) attempt to predict winners or trends, our approach relies on predictable statistical relations between all pairs of stocks in the market. Our empirical results on historical markets provide strong evidence that this type of technical trading can "beat the market" and moreover, can beat the best stock in the market. In doing so we utilize a new idea for smoothing critical parameters in the context of expert learning.
