Profitability Analysis in Stock Investment Using an LSTM-Based Deep Learning Model
Praxis Business School（普拉克斯商学院）
Jaydip Sen, Abhishek Dutta, Sidra Mehtab
备注：
This is the accepted version of our paper in the Second IEEE International Conference on Emerging Technologies (IEEE INCET 2021) which will be organized in Belgaum, Karnataka, INDIA from May 21 to May 23, 2021. The paper is eight pages long, and has fifteen tables and fourteen figures. This is not the final version of the paper
Designing robust systems for precise prediction of future prices of stocks has always been considered a very challenging research problem. Even more challenging is to build a system for constructing an optimum portfolio of stocks based on the forecasted future stock prices. We present a deep learning-based regression model built on a long-and-short-term memory network (LSTM) network that automatically scraps the web and extracts historical stock prices based on a stock's ticker name for a specified pair of start and end dates, and forecasts the future stock prices. We deploy the model on 75 significant stocks chosen from 15 critical sectors of the Indian stock market. For each of the stocks, the model is evaluated for its forecast accuracy. Moreover, the predicted values of the stock prices are used as the basis for investment decisions, and the returns on the investments are computed. Extensive results are presented on the performance of the model. The analysis of the results demonstrates the efficacy and effectiveness of the system and enables us to compare the profitability of the sectors from the point of view of the investors in the stock market.
Enhancing Profitability and Investor Confidence through Interpretable AI Models for Investment Decisions
School of Electrical Engineering and Computer Science, National University of Sciences and Technology (NUST)（国家科技大学电气工程与计算机科学学院）；Department of Computer Science, Bahria University（巴里亚大学计算机科学系）；College of Computer and Information Sciences (CCIS), Prince Sultan University（苏丹王子大学计算机与信息科学学院）
Ecole Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；University of California Los Angeles（加利福尼亚大学洛杉矶分校）；Imperial College London（伦敦帝国理工学院）
In the IEEE Investment ranking challenge 2018, participants were asked to build a model which would identify the best performing stocks based on their returns over a forward six months window. Anonymized financial predictors and semi-annual returns were provided for a group of anonymized stocks from 1996 to 2017, which were divided into 42 non-overlapping six months period. The second half of 2017 was used as an out-of-sample test of the model's performance. Metrics used were Spearman's Rank Correlation Coefficient and Normalized Discounted Cumulative Gain (NDCG) of the top 20% of a model's pr…
Sun Yat-sen University（中山大学）；Microsoft Research（微软研究院）
Stock trend forecasting, which forecasts stock prices' future trends, plays an essential role in investment. The stocks in a market can share information so that their stock prices are highly correlated. Several methods were recently proposed to mine the shared information through stock concepts (e.g., technology, Internet Retail) extracted from the Web to improve the forecasting results. However, previous work assumes the connections between stocks and concepts are stationary, and neglects the dynamic relevance between stocks and concepts, limiting the forecasting results. Moreover, existing …
Nomura Asset Management Co.（野村资产管理公司）；New York University（纽约大学）
Stock return predictability is an important research theme as it reflects our economic and social organization, and significant efforts are made to explain the dynamism therein. Statistics of strong explanative power, called "factor" have been proposed to summarize the essence of predictive stock returns. Although machine learning methods are increasingly popular in stock return prediction, an inference of the stock returns is highly elusive, and still most investors, if partly, rely on their intuition to build a better decision making. The challenge here is to make an investment strategy that…
This paper investigates the time-varying risk-premium relation of the Chinese stock markets within the framework of cross-sectional momentum and contrarian effects by adopting the Capital Asset Pricing Model and the French-Fama three factor model. The evolving arbitrage opportunities are also studied by quantifying the performance of time-varying cross-sectional momentum and contrarian effects in the Chinese stock markets. The relation between the contrarian profitability and market condition factors that could characterize the investment context is also investigated. The results reveal that t…
Southwestern University of Finance and Economics（西南财经大学）；University of Maryland, College Park（马里兰大学帕克分校）；Pennsylvania State University（宾夕法尼亚州立大学）；University of Electronic Science and Technology of China（电子科技大学）
Stock selection attempts to rank a list of stocks for optimizing investment decision making, aiming at minimizing investment risks while maximizing profit returns. Recently, researchers have developed various (recurrent) neural network-based methods to tackle this problem. Without exceptions, they primarily leverage historical market volatility to enhance the selection performance. However, these approaches greatly rely on discrete sampled market observations, which either fail to consider the uncertainty of stock fluctuations or predict continuous stock dynamics in the future. Besides, some s…
This paper proposes a method for ranking the investment attractiveness of exchange-traded stocks where investment risk is not related to the volatility indicator but instead is related to the indicator of compression of the time series of price changes. The article describes in detail the ranking algorithm, provides an example of ranking the shares of all companies included in the Dow Jones stock index. The paper additionally compares the results of ranking these stocks by volatility and compression and also shows the strengths of the second indicator, which is formed using the method of binar…
We present a deep long short-term memory (LSTM)-based neural network for predicting asset prices, together with a successful trading strategy for generating profits based on the model's predictions. Our work is motivated by the fact that the effectiveness of any prediction model is inherently coupled to the trading strategy it is used with, and vise versa. This highlights the difficulty in developing models and strategies which are jointly optimal, but also points to avenues of investigation which are broader than prevailing approaches. Our LSTM model is structurally simple and generates predi…
Although there is a wide use of technical trading rules in stock markets, the profitability of them still remains controversial. This paper first presents and proves the upper bound of cumulative return, and then introduces many of conventional technical trading rules. Furthermore, with the help of bootstrap methodology, we investigate the profitability of technical trading rules on different international stock markets, including developed markets and emerging markets. At last, the results show that the technical trading rules are hard to beat the market, and even less profitable than the ran…
Technical trading rules have been widely used by practitioners in financial markets for a long time. The profitability remains controversial and few consider the stationarity of technical indicators used in trading rules. We convert MA, KDJ and Bollinger bands into stationary processes and investigate the profitability of these trading rules by using 3 high-frequency data(15s,30s and 60s) of CSI300 Stock Index Futures from January 4th 2012 to December 31st 2016. Several performance and risk measures are adopted to assess the practical value of all trading rules directly while ADF-test is used …
The recent hype around Reddit's WallStreetBets (WSB) community has inspired research on its impact on our economy and society. Still, one important question remains: Can WSB's community of anonymous contributors actually provide valuable investment advice and possibly even outperform top financial institutions? We present a data-driven empirical study of investment recommendations of WSB in comparison to recommendations made by leading investment banks, based on more than 1.6 million WSB posts published since 2018. %enriched with stock market data. To this end, we extract and evaluate investme…
Portfolio allocation is crucial for investment companies. However, getting the best strategy in a complex and dynamic stock market is challenging. In this paper, we propose a novel Adaptive Deep Deterministic Reinforcement Learning scheme (Adaptive DDPG) for the portfolio allocation task, which incorporates optimistic or pessimistic deep reinforcement learning that is reflected in the influence from prediction errors. Dow Jones 30 component stocks are selected as our trading stocks and their daily prices are used as the training and testing data. We train the Adaptive DDPG agent and obtain a t…
This paper, for the first time, focuses on the sector-wise analysis of a stock market through multifractal analysis. We have considered Bombay Stock Exchange, India, and identified two time scales, short ($<200$ days) and long time-scale ($>200$ days) for investment. We infer that long-term investment will be more profitable. For long time scale, sectors can be separated into two categories based on the Hurst exponent values; one corresponds to stable sectors with small fluctuations, and the other with dominance of large fluctuations leading to possible downturns in those sectors.
Columbia University（哥伦比亚大学）；Beijing Institute of Technology（北京理工大学）；New York University Shanghai（上海纽约大学）
Midterm stock price prediction is crucial for value investments in the stock market. However, most deep learning models are essentially short-term and applying them to midterm predictions encounters large cumulative errors because they cannot avoid anomalies. In this paper, we propose a novel deep neural network Mid-LSTM for midterm stock prediction, which incorporates the market trend as hidden states. First, based on the autoregressive moving average model (ARMA), a midterm ARMA is formulated by taking into consideration both hidden states and the capital asset pricing model. Then, a midterm…
Stock market investment have been an ideal form of investment for many years. Investing capitals smartly in stock market yields high profit returns. But there are many companies available in a market. Currently there are more than $345$ active companies who have stocks in Dhaka Stock Exchange (DSE). Analyzing all these companies is quite impossible. However, many companies tend to move together. This study aims at finding which companies in DSE have a close connection and move alongside each other. By analyzing this relation, the investors and traders will be able to analyze a lot of companies…
We demonstrate that the tail dependence should always be taken into account as a proxy for systematic risk of loss for investments. We provide the clear statistical evidence of that the structure of investment portfolios on a regulated market should be adjusted to the price of gold. Our finding suggests that the active bartering of oil for goods would prevent collapsing the national market facing international sanctions.
Sun Yat-sen University（中山大学）；Microsoft Research（微软研究院）
Successful quantitative investment usually relies on precise predictions of the future movement of the stock price. Recently, machine learning based solutions have shown their capacity to give more accurate stock prediction and become indispensable components in modern quantitative investment systems. However, the i.i.d. assumption behind existing methods is inconsistent with the existence of diverse trading patterns in the stock market, which inevitably limits their ability to achieve better stock prediction performance. In this paper, we propose a novel architecture, Temporal Routing Adaptor…
This paper tends to define the quantitative relationship between the stock price and time as a time function. Based on the empirical evidence that the log-return of a stock is the series of white noise, a mathematical model of the integral white noise is established to describe the phenomenon of stock price movement. A deductive approach is used to derive the auto-correlation function, displacement formula and power spectral density of the stock price movement, which reveals not only the characteristics and rules of the movement but also the predictability of the stock price. The deductive fun…
We develop and implement methods for determining whether introducing new securities or relaxing investment constraints improves the investment opportunity set for prospect investors. We formulate a new testing procedure for prospect spanning for two nested portfolio sets based on subsampling and Linear Programming. In an application, we use the prospect spanning framework to evaluate whether well-known anomalies are spanned by standard factors. We find that of the strategies considered, many expand the opportunity set of the prospect type investors, thus have real economic value for them. In-s…
Different investment strategies are adopted in short-term and long-term depending on the time scales, even though time scales are adhoc in nature. Empirical mode decomposition based Hurst exponent analysis and variance technique have been applied to identify the time scales for short-term and long-term investment from the decomposed intrinsic mode functions(IMF). Hurst exponent ($H$) is around 0.5 for the IMFs with time scales from few days to 3 months, and $H\geq0.75$ for the IMFs with the time scales $\geq5$ months. Short term time series [$X_{ST}(t)$] with time scales from few days to 3 mon…
This paper proposes a novel adaptive algorithm for the automated short-term trading of financial instrument. The algorithm adopts a semantic sentiment analysis technique to inspect the Twitter posts and to use them to predict the behaviour of the stock market. Indeed, the algorithm is specifically developed to take advantage of both the sentiment and the past values of a certain financial instrument in order to choose the best investment decision. This allows the algorithm to ensure the maximization of the obtainable profits by trading on the stock market. We have conducted an investment simul…
Reddit's WallStreetBets (WSB) community has come to prominence in light of its notable role in affecting the stock prices of what are now referred to as meme stocks. Yet very little is known about the reliability of the highly speculative investment advice disseminated on WSB. This paper analyses WSB data spanning from January 2019 to April 2021 in order to assess how successful an investment strategy relying on the community's recommendations could have been. We detect buy and sell advice and identify the community's most popular stocks, based on which we define a WSB portfolio. Our evaluatio…
The evaluation of the financial markets to predict their behaviour have been attempted using a number of approaches, to make smart and profitable investment decisions. Owing to the highly non-linear trends and inter-dependencies, it is often difficult to develop a statistical approach that elucidates the market behaviour entirely. To this end, we present a long-short term memory (LSTM) based model that leverages the sequential structure of the time-series data to provide an accurate market forecast. We then develop a decision making StockBot that buys/sells stocks at the end of the day with th…
The paper presents the comparative study of the nature of stock markets in short-term and long-term time scales with and without structural break in the stock data. Structural break point has been identified by applying Zivot and Andrews structural trend break model to break the original time series (TSO) into time series before structural break (TSB) and time series after structural break (TSA). The empirical mode decomposition based Hurst exponent and variance techniques have been applied to the TSO, TSB and TSA to identify the time scales in short-term and long-term from the decomposed intr…
Stock prices are driven by various factors. In particular, many individual investors who have relatively little financial knowledge rely heavily on the information from news stories when making investment decisions in the stock market. However, these stories may not reflect future stock prices because of the subjectivity in the news; stock prices may instead affect the news contents. This study aims to discover whether it is news or stock prices that have a greater impact on the other. To achieve this, we analyze the relationship between news sentiment and stock prices based on time series ana…
It is common knowledge that leverage can increase the potential returns of an investment, at the expense of increased risk. For a passive investor in the stock market, leverage can be achieved using margin debt or leveraged-ETFs. We perform bootstrapped Monte-Carlo simulations of leveraged (and unleveraged) mixed portfolios of stocks and bonds, based on past stock market data, and show that leverage can amplify the potential returns, without significantly increasing the risk for long-term investors.
