arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2010.04404q-fin.PMq-fin.CP
收藏

深度强化学习在US股票资产配置中的应用

Deep Reinforcement Learning for Asset Allocation in US Equities

Miquel Noguer i Alonso, Sonam Srivastava

摘要:

强化学习是一种机器学习方法,通过最大化状态和动作空间中的奖励函数,以几乎无模型的方式解决动态优化问题。这一特性使其成为金融问题研究的一个令人兴奋的领域。资产配置的目标是在考虑风险和交易成本的市场给定状态下获得最大化回报的资产权重,这是一个很容易使用强化学习框架构建的问题。首先是预期收益率和协方差矩阵的预测问题,然后是收益、风险和市场冲击的优化问题。投资者和金融研究人员一直在使用均值方差优化、最小方差、风险平价和等权等方法,以及几种方法来使预期收益率和协方差矩阵的预测更加稳健。本文展示了强化学习的应用,以创建一个金融模型的自由解决方案的资产配置问题,学习解决问题使用时间序列和深度神经网络。我们在美国股市全球前24只股票的每日数据上证明了这一点,并进行了每日再平衡。我们使用不同架构的美国股票深度强化模型。我们使用长短期记忆网络、卷积神经网络和递归神经网络,并将它们与更传统的投资组合管理进行比较。深度强化学习方法比使用简单奖励函数的传统方法显示出更好的结果,并且只给出股票的时间序列。在金融中,没有训练来测试错误泛化结果。我们可以说,建模框架可以处理时间序列预测和资产配置,包括交易成本。

英文摘要:

Reinforcement learning is a machine learning approach concerned with solving dynamic optimization problems in an almost model-free way by maximizing a reward function in state and action spaces. This property makes it an exciting area of research for financial problems. Asset allocation, where the goal is to obtain the weights of the assets that maximize the rewards in a given state of the market considering risk and transaction costs, is a problem easily framed using a reinforcement learning framework. It is first a prediction problem for expected returns and covariance matrix and then an optimization problem for returns, risk, and market impact. Investors and financial researchers have been working with approaches like mean-variance optimization, minimum variance, risk parity, and equally weighted and several methods to make expected returns and covariance matrices' predictions more robust. This paper demonstrates the application of reinforcement learning to create a financial model-free solution to the asset allocation problem, learning to solve the problem using time series and deep neural networks. We demonstrate this on daily data for the top 24 stocks in the US equities universe with daily rebalancing. We use a deep reinforcement model on US stocks using different architectures. We use Long Short Term Memory networks, Convolutional Neural Networks, and Recurrent Neural Networks and compare them with more traditional portfolio management. The Deep Reinforcement Learning approach shows better results than traditional approaches using a simple reward function and only being given the time series of stocks. In Finance, no training to test error generalization results come guaranteed. We can say that the modeling framework can deal with time series prediction and asset allocation, including transaction costs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑