arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.21025q-fin.TRcs.LGecon.EMq-fin.STstat.ML
收藏

强化学习在高频做市中的应用

Reinforcement Learning in High-frequency Market Making

Princeton University(普林斯顿大学)

Yuheng Zheng, Zihan Ding

AI总结:

本文针对高频做市中的强化学习应用开展理论分析,揭示了采样频率下算法误差与复杂度的权衡关系,证明了双人场景下纳什均衡的收敛性,相关理论可推广至其他高频金融决策问题。

摘要:

本文针对强化学习(RL)在高频做市中的应用建立了全新且全面的理论分析框架,搭建起现代RL理论与高频金融经济学中连续时间统计模型之间的桥梁。与现有多数聚焦于为做市问题开发各类RL方法的方法学研究文献不同,本工作是提供该领域理论分析的先导性研究。本文聚焦采样频率的影响,发现调整时间增量$Δ$取值时,RL算法的误差与复杂度之间存在有趣的权衡关系:$Δ$越小,误差越小但复杂度越高。本文还研究了一般和博弈框架下的双人场景,证明了当$Δ\rightarrow0$时纳什均衡收敛到连续时间博弈均衡,并采用Nash Q-learning算法(一种在线多智能体RL方法)求解该均衡。本文提出的理论不仅可辅助从业者选择采样频率,还具备很强的通用性,适用于其他采用连续时间马尔可夫决策过程时间离散化的高频金融决策问题,例如最优执行。蒙特卡洛模拟结果验证了所有理论结论的正确性。

英文摘要:

This paper establishes a new and comprehensive theoretical analysis for the application of reinforcement learning (RL) in high-frequency market making. We bridge the modern RL theory and the continuous-time statistical models in high-frequency financial economics. Different with most existing literature on methodological research about developing various RL methods for market making problem, our work is a pilot to provide the theoretical analysis. We target the effects of sampling frequency, and find an interesting tradeoff between error and complexity of RL algorithm when tweaking the values of the time increment $Δ$ $-$ as $Δ$ becomes smaller, the error will be smaller but the complexity will be larger. We also study the two-player case under the general-sum game framework and establish the convergence of Nash equilibrium to the continuous-time game equilibrium as $Δ\rightarrow0$. The Nash Q-learning algorithm, which is an online multi-agent RL method, is applied to solve the equilibrium. Our theories are not only useful for practitioners to choose the sampling frequency, but also very general and applicable to other high-frequency financial decision making problems, e.g., optimal executions, as long as the time-discretization of a continuous-time markov decision process is adopted. Monte Carlo simulation evidence support all of our theories.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑