掌握基于风险感知循环强化学习的配对交易
Mastering Pair Trading with Risk-Aware Recurrent Reinforcement Learning
AI总结:
提出CREDIT,一种风险感知循环强化学习智能体,通过双向GRU和时间注意力机制捕捉资产价格变动的长期模式,并设计风险感知奖励,在五年美股数据上实现显著利润,优于现有RL方法。
摘要:
尽管配对交易是投资者消除市场风险的最简单对冲策略,但强化学习(RL)方法要像人类专家那样执行配对交易仍然是一个巨大的挑战。它要求RL方法做出数千个正确的动作,而这些动作与整体交易利润没有明显的关系,并且要在时变市场的无限状态中进行推理,这些状态中的大多数在历史上从未出现过。然而,现有的RL方法忽略了资产价格变动与所执行交易风险之间的时间联系。这导致了频繁的交易,产生高昂的交易成本和潜在损失,几乎无法达到人类专家的交易水平。因此,我们引入了CREDIT,这是一种风险感知智能体,能够像人类专家一样学习利用配对交易中的长期交易机会。CREDIT首次应用双向GRU以及时间注意力机制来充分考虑嵌入状态中的时间相关性,这使得CREDIT能够捕捉两种资产价格变动的长期模式以获取更高的利润。我们还设计了受经济学理论启发的风险感知奖励,对交易期间内的利润和风险进行建模。它帮助我们的智能体掌握配对交易,形成稳健的交易偏好,避免可能带来高收益和高损失的冒险交易。实验表明,它在配对交易中优于现有的强化学习方法,并在五年的美国股票数据上实现了显著的利润。
英文摘要:
Although pair trading is the simplest hedging strategy for an investor to eliminate market risk, it is still a great challenge for reinforcement learning (RL) methods to perform pair trading as human expertise. It requires RL methods to make thousands of correct actions that nevertheless have no obvious relations to the overall trading profit, and to reason over infinite states of the time-varying market most of which have never appeared in history. However, existing RL methods ignore the temporal connections between asset price movements and the risk of the performed trading. These lead to frequent tradings with high transaction costs and potential losses, which barely reach the human expertise level of trading. Therefore, we introduce CREDIT, a risk-aware agent capable of learning to exploit long-term trading opportunities in pair trading similar to a human expert. CREDIT is the first to apply bidirectional GRU along with the temporal attention mechanism to fully consider the temporal correlations embedded in the states, which allows CREDIT to capture long-term patterns of the price movements of two assets to earn higher profit. We also design the risk-aware reward inspired by the economic theory, that models both the profit and risk of the tradings during the trading period. It helps our agent to master pair trading with a robust trading preference that avoids risky trading with possible high returns and losses. Experiments show that it outperforms existing reinforcement learning methods in pair trading and achieves a significant profit over five years of U.S. stock data.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。