Can deep reinforcement learning algorithms be exploited as solvers for optimal trading strategies? The aim of this work is to test reinforcement learning algorithms on conceptually simple, but mathematically non-trivial, trading environments. The environments are chosen such that an optimal or close-to-optimal trading strategy is known. We study the deep deterministic policy gradient algorithm and show that such a reinforcement learning agent can successfully recover the essential features of the optimal trading strategies and achieve close-to-optimal rewards.
LLM-based Personalized Portfolio Recommender: Integrating Large Language Models and Reinforcement Learning for Intelligent Investment Strategy Optimization
University of Electronic Science and Technology of China（电子科技大学）；Ross School of Business, University of Michigan（密歇根大学罗斯商学院）；School of Humanities and Sciences, Stanford University（斯坦福大学人文与科学学院）
A Comparative Analysis of Portfolio Optimization Using Mean-Variance, Hierarchical Risk Parity, and Reinforcement Learning Approaches on the Indian Stock Market
Reinforcement Learning for Portfolio Optimization with a Financial Goal and Defined Time Horizons
AIMS South Africa /Stellenbosch university, South Africa（AIMS南非 /斯托滕博斯大学，南非）；University of South Africa, South Africa（南非大学，南非）；Michigan State University, USA（密歇根州立大学，美国）
Stock portfolio optimization is the process of constant re-distribution of money to a pool of various stocks. In this paper, we will formulate the problem such that we can apply Reinforcement Learning for the task properly. To maintain a realistic assumption about the market, we will incorporate transaction cost and risk factor into the state as well. On top of that, we will apply various state-of-the-art Deep Reinforcement Learning algorithms for comparison. Since the action space is continuous, the realistic formulation were tested under a family of state-of-the-art continuous policy gradien…
This paper studies a robust portfolio optimization problem under the multi-factor volatility model introduced by Christoffersen et al. (2009). The optimal strategy is derived analytically under the worst-case scenario with or without derivative trading. To illustrate the effects of ambiguity, we compare our optimal robust strategy with some strategies that ignore the information of uncertainty, and provide the corresponding welfare analysis. The effects of derivative trading to the optimal portfolio selection are also discussed by considering alternative strategies. Our study is further extend…
We provide a verification and characterization result of optimal maximal sub-solutions of BSDEs in terms of fully coupled forward backward stochastic differential equations. We illustrate the application thereof in utility optimization with random endowment under probability and discounting uncertainty. We show with explicit examples how to quantify the costs of incompleteness when using utility indifference pricing, as well as a way to find optimal solutions for recursive utilities.
This paper uses simulation-based portfolio optimization to mitigate the left tail risk of the portfolio. The contribution is twofold. (i) We propose the Markov regime-switching GARCH model with multivariate normal tempered stable innovation (MRS-MNTS-GARCH) to accommodate fat tails, volatility clustering and regime switch. The volatility of each asset independently follows the regime-switch GARCH model, while the correlation of joint innovation of the GARCH models follows the Hidden Markov Model. (ii) We use tail risk measures, namely conditional value-at-risk (CVaR) and conditional drawdown-a…
We study the expected utility portfolio optimization problem in an incomplete financial market where the risky asset dynamics depend on stochastic factors and the portfolio allocation is constrained to lie within a given convex set. We employ fundamental duality results from real constrained optimization to formally derive a dual representation of the associated HJB PDE. Using this representation, we provide a condition on the market dynamics and the allocation constraints, which ensures that the solution to the HJB PDE is exponentially affine and separable. This condition is used to derive an…
This paper proposes a new method for financial portfolio optimization based on reducing simultaneous asset shocks across a collection of assets. This may be understood as an alternative approach to risk reduction in a portfolio based on a new mathematical quantity. First, we apply recently introduced semi-metrics between finite sets to determine the distance between time series' structural breaks. Then, we build on the classical portfolio optimization theory of Markowitz and use this distance between asset structural breaks for our penalty function, rather than portfolio variance. Our experime…
We apply numerical dynamic programming techniques to solve discrete-time multi-asset dynamic portfolio optimization problems with proportional transaction costs and shorting/borrowing constraints. Examples include problems with multiple assets, and many trading periods in a finite horizon problem. We also solve dynamic stochastic problems, with a portfolio including one risk-free asset, an option, and its underlying risky asset, under the existence of transaction costs and constraints. These examples show that it is now tractable to solve such problems.
In this paper, we document a novel machine learning based bottom-up approach for static and dynamic portfolio optimization on, potentially, a large number of assets. The methodology applies to general constrained optimization problems and overcomes many major difficulties arising in current optimization schemes. Taking mean-variance optimization as an example, we no longer need to compute the covariance matrix and its inverse, therefore the method is immune from the estimation error on this quantity. Moreover, no explicit calls of optimization routines are needed. Applications to equity portfo…
We consider the issue of solution uniqueness for portfolio optimization problem and its inverse for asset returns with a finite number of possible scenarios. The risk is assessed by deviation measures introduced by [Rockafellar et al., Mathematical Programming, Ser. B, 108 (2006), pp. 515-540] instead of variance as in the Markowitz optimization problem. We prove that in general one can expect uniqueness neither in forward nor in inverse problems. We discuss consequences of that non-uniqueness for several problems in risk analysis and portfolio optimization, including capital allocation, risk …
In this paper we consider a discrete-time risk sensitive portfolio optimization over a long time horizon with proportional transaction costs. We show that within the log-return i.i.d. framework the solution to a suitable Bellman equation exists under minimal assumptions and can be used to characterize the optimal strategies for both risk-averse and risk-seeking cases. Moreover, using numerical examples, we show how a Bellman equation analysis can be used to construct or refine optimal trading strategies in the presence of transaction costs.
