基于强化学习的多级做市策略
Multi-Level Market Making with Reinforcement Learning
中文总结 AI 辅助
该研究提出一种强化学习做市框架，通过多元逻辑正态分布建模订单分配、深度集编码器聚合特征、势能奖励塑形加速学习，在含三类交易者的模拟市场中验证了其性能。
中文摘要 AI 辅助
我们提出了一种用于订单簿做市的强化学习框架。该算法旨在通过在多个价格水平上动态提交不同规模的市价订单和限价订单，同时控制库存规模，以最大化交易收益。我们使用多元逻辑正态分布对订单分配进行建模，并采用深度集编码器将可变长度订单集的特征聚合为固定维度的潜在表示。此外，我们引入基于势能的奖励塑形，以在不改变最优策略的前提下加速学习。我们在三种模拟市场环境中验证了该方法的性能，这些环境包含提交随机交易的噪声交易者、对瞬时交易量失衡做出反应的策略性交易者，以及沿指数加权交易量失衡信号方向交易的战略性交易者。
英文摘要
We introduce a reinforcement learning framework for market making in a limit order book. Our algorithm aims to maximize trading revenue by dynamically submitting market and limit orders of varying sizes across multiple price levels while controlling inventory size. We use multivariate logistic-normal distributions to model order allocations and employ a deep-set encoder to aggregate features from variable-length order sets into a fixed-dimensional latent representation. Additionally, we incorporate potential-based reward shaping to accelerate learning without altering the optimal policy. We illustrate the performance of the method in three simulated market environments consisting of noise traders who submit random trades, tactical traders who respond to instantaneous volume imbalance, and strategic traders who trade in the direction of an exponentially weighted volume imbalance signal.
