股票交易的深度强化学习：基于前向再训练的Actor-Critic方法基准测试
Deep Reinforcement Learning for Equity Trading: Benchmarking Actor-Critic Methods with Forward Retraining
浏览论文内容
中文总结 AI 辅助
本研究基准测试五种深度强化学习actor-critic方法在股票交易中的表现，发现DDPG收益最高但风险大，TD3和SAC平衡更佳，前向再训练可提升部分方法，并揭示了端到端DRL与预测驱动策略的收益-风险权衡。
中文摘要 AI 辅助
持续盈利的交易是困难的，因为股票市场具有噪声、非平稳性，且仅能从历史数据中部分预测。我们基准测试了五种深度强化学习（DRL）actor-critic方法：A2C、PPO、DDPG、TD3和SAC，这些方法从市场状态端到端地学习交易动作，并将其与有监督的价格预测基线进行比较。使用2000年至2020年间20只标普500大市值股票的日数据，并辅以趋势跟踪技术指标和对数最小-最大缩放，我们在2000-2018年进行训练，在2019-2020年进行回测。每个智能体在仅训练一次以及在前向再训练（即在每个连续测试窗口之前利用所有可用数据重新训练）两种情况下进行评估。DDPG实现了最高的年化收益率（55.5%）、夏普比率（1.38）和alpha（0.22），但同时也具有最高的市场beta（1.24）。TD3和SAC提供了更好的风险-收益平衡，夏普比率分别为1.37和1.33，最大回撤约为25%。前向再训练改善了A2C、PPO和SAC，使TD3基本保持不变，并将DDPG的年化收益率从55.5%降至29.8%，这与TD3对超参数更强的鲁棒性一致。预测基线具有最小的最大回撤（9.6%）和最低的beta（0.31），凸显了端到端DRL的较高收益与预测驱动策略的较低风险之间的权衡。
英文摘要
Consistently profitable trading is difficult because equity markets are noisy, non-stationary, and only partially predictable from historical data. We benchmark five deep reinforcement learning (DRL) actor-critic methods: A2C, PPO, DDPG, TD3, and SAC, that learn trading actions end-to-end from market states, and compare them with a supervised price-forecasting baseline. Using daily data for 20 large-capitalization S&P 500 stocks from 2000 to 2020, enriched with trend-following technical indicators and log min-max scaling, we train on 2000-2018 and backtest on 2019-2020. Each agent is evaluated both when trained once and under forward retraining, in which it is retrained on all data available before each successive test window. DDPG achieves the highest annual return (55.5%), Sharpe ratio (1.38), and alpha (0.22), but also the highest market beta (1.24). TD3 and SAC offer a better risk-return balance, with Sharpe ratios of 1.37 and 1.33 and maximum drawdowns of about 25%. Forward retraining improves A2C, PPO, and SAC, leaves TD3 essentially unchanged, and reduces DDPG's annual return from 55.5% to 29.8%, consistent with TD3's greater robustness to hyperparameters. The forecasting baseline has the smallest maximum drawdown (9.6%) and the lowest beta (0.31), underscoring a trade-off between the higher returns of end-to-end DRL and the lower risk of forecast-driven strategies.
