arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2512.03059cs.LG
收藏

安全且可持续的电动巴士充电调度与受约束的分层深度强化学习

Safe and Sustainable Electric Bus Charging Scheduling with Constrained Hierarchical DRL

College of Engineering, University of Guelph(圭尔夫大学工程学院);EthicalAI;College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院)

Jiaju Qi, Lei Lei, Thorsteinn Jonsson, Dusit Niyato

AI总结:

本文提出了一种安全且可持续的电动巴士充电调度方法,采用受约束的分层深度强化学习框架,通过整合拉格朗日松弛实现安全与成本的优化。

摘要:

将电动巴士(EBs)与可再生能源源如光伏(PV)板整合起来,是推动可持续和低碳公共交通的一种有前景的方法。然而,优化电动巴士充电计划以在最小化运营成本的同时确保安全运行且不导致电池耗尽仍具挑战性,特别是在现实条件下,必须考虑光伏发电的不确定性、动态电价、变化的出行时间以及有限的充电基础设施。在本文中,我们提出了一种安全的分层深度强化学习(HDRL)框架,用于在多源不确定性下解决电动巴士充电调度问题(EBCSP)。我们将问题建模为一个带选项的约束马尔可夫决策过程(CMDP),以实现时间抽象的决策。我们开发了一种新的HDRL算法,即双Actor-Critic多智能体近端策略优化拉格朗日(DAC-MAPPO-Lagrangian),该算法将拉格朗日松弛整合到双Actor-Critic(DAC)框架中。在高层,我们采用集中式PPO-Lagrangian算法来学习安全的充电器分配策略。在低层,我们采用MAPPO-Lagrangian来在集中式训练和去中心化执行(CTDE)范式下学习去中心化的充电功率决策。广泛的实验使用真实数据表明,所提出的方法在成本最小化和安全合规性方面均优于现有基线,同时保持快速收敛速度。

英文摘要:

The integration of Electric Buses (EBs) with renewable energy sources such as photovoltaic (PV) panels is a promising approach to promote sustainable and low-carbon public transportation. However, optimizing EB charging schedules to minimize operational costs while ensuring safe operation without battery depletion remains challenging - especially under real-world conditions, where uncertainties in PV generation, dynamic electricity prices, variable travel times, and limited charging infrastructure must be accounted for. In this paper, we propose a safe Hierarchical Deep Reinforcement Learning (HDRL) framework for solving the EB Charging Scheduling Problem (EBCSP) under multi-source uncertainties. We formulate the problem as a Constrained Markov Decision Process (CMDP) with options to enable temporally abstract decision-making. We develop a novel HDRL algorithm, namely Double Actor-Critic Multi-Agent Proximal Policy Optimization Lagrangian (DAC-MAPPO-Lagrangian), which integrates Lagrangian relaxation into the Double Actor-Critic (DAC) framework. At the high level, we adopt a centralized PPO-Lagrangian algorithm to learn safe charger allocation policies. At the low level, we incorporate MAPPO-Lagrangian to learn decentralized charging power decisions under the Centralized Training and Decentralized Execution (CTDE) paradigm. Extensive experiments with real-world data demonstrate that the proposed approach outperforms existing baselines in both cost minimization and safety compliance, while maintaining fast convergence speed.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑