可交易信用方案中逐日动态收费的深度强化学习
Deep Reinforcement Learning for Day-to-day Dynamic Tolling in Tradable Credit Schemes
AI总结：
本文针对可交易信用方案中的逐日动态收费问题，将其建模为马尔可夫决策过程，提出基于深度强化学习的求解方法，并通过正则化与迁移学习提升策略鲁棒性、可迁移性和计算效率。
摘要：
可交易信用方案（TCS）因其收入中性以及能够通过初始信用分配解决公平性问题，正成为日益受到研究的拥堵收费替代方案。对TCS进行建模以支持未来设计与实施，涉及用户和市场行为、供需动态以及控制机制等方面的挑战。本文聚焦于控制机制，研究TCS下的逐日动态收费问题，将其形式化为离散时间马尔可夫决策过程，并使用强化学习（RL）算法求解。结果表明，RL算法在出行时间和社会福利方面达到了与贝叶斯优化基准相当的水平，并且在不同容量和需求水平下具有泛化能力。我们进一步评估了RL在不同超参数下的鲁棒性，并应用正则化技术来缓解动作振荡，从而生成在逐日供需变化下具有可迁移性的实用收费策略。最后，我们讨论了诸如扩展到大规模网络等潜在挑战，并展示了如何利用迁移学习提高计算效率，促进基于RL的TCS解决方案的实际部署。
英文摘要：
Tradable credit schemes (TCS) are an increasingly studied alternative to congestion pricing, given their revenue neutrality and ability to address issues of equity through the initial credit allocation. Modeling TCS to aid future design and implementation is associated with challenges involving user and market behaviors, demand-supply dynamics, and control mechanisms. In this paper, we focus on the latter and address the day-to-day dynamic tolling problem under TCS, which is formulated as a discrete-time Markov Decision Process and solved using reinforcement learning (RL) algorithms. Our results indicate that RL algorithms achieve travel times and social welfare comparable to the Bayesian optimization benchmark, with generalization across varying capacities and demand levels. We further assess the robustness of RL under different hyperparameters and apply regularization techniques to mitigate action oscillation, which generates practical tolling strategies that are transferable under day-to-day demand and supply variability. Finally, we discuss potential challenges such as scaling to large networks, and show how transfer learning can be leveraged to improve computational efficiency and facilitate the practical deployment of RL-based TCS solutions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
