最优停止的深度强化学习及其在金融工程中的应用
Deep Reinforcement Learning for Optimal Stopping with Application in Financial Engineering
摘要:
最优停止是在随机系统中决定采取特定行动的正确时间的问题,以最大化期望回报。它在金融、医疗保健和统计等领域有许多应用。在本文中,我们采用深度强化学习(RL)来学习两个金融工程应用中的最优停止策略:即期权定价和最优期权行使。我们首次对三种最先进的深度RL算法确定的最佳停止策略的质量进行了全面的实证评估:双重深度Q学习(DDQN),分类分布RL(C51)和隐式分位数网络(IQN)。在期权定价的情况下,我们的研究结果表明,在理论上的布莱克-舒尔环境中,IQN成功地确定了接近最优的价格。另一方面,当面临涉及标准普尔500指数资产的看跌期权行使问题中的实际股票数据变动时,C51的表现略好于C51。更重要的是,C51算法能够确定一个最佳停止策略,该策略比四个自然基准策略中的最佳策略多获得8%的样本外回报。最后,我们讨论了我们的研究结果,这应该为相关的未来研究铺平道路。
英文摘要:
Optimal stopping is the problem of deciding the right time at which to take a particular action in a stochastic system, in order to maximize an expected reward. It has many applications in areas such as finance, healthcare, and statistics. In this paper, we employ deep Reinforcement Learning (RL) to learn optimal stopping policies in two financial engineering applications: namely option pricing, and optimal option exercise. We present for the first time a comprehensive empirical evaluation of the quality of optimal stopping policies identified by three state of the art deep RL algorithms: double deep Q-learning (DDQN), categorical distributional RL (C51), and Implicit Quantile Networks (IQN). In the case of option pricing, our findings indicate that in a theoretical Black-Schole environment, IQN successfully identifies nearly optimal prices. On the other hand, it is slightly outperformed by C51 when confronted to real stock data movements in a put option exercise problem that involves assets from the S&P500 index. More importantly, the C51 algorithm is able to identify an optimal stopping policy that achieves 8% more out-of-sample returns than the best of four natural benchmark policies. We conclude with a discussion of our findings which should pave the way for relevant future research.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。