arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2305.01461eess.SYcs.AIcs.SY
收藏

基于强化学习的混合整数最优控制:混合动力汽车能量管理的案例研究

Mixed-Integer Optimal Control via Reinforcement Learning: A Case Study on Hybrid Electric Vehicle Energy Management

Shien-Ming Wu School of Intelligent Engineering, South China University of Technology(华南理工大学吴贤铭智能工程学院);University of Waterloo(滑铁卢大学)

Jinming Xu, Nasser Lashgarian Azad, Yuan Lin

AI总结:

本文提出一种新型混合动作强化学习算法TD3AQ,用于解决混合整数最优控制问题,在PHEV能量管理案例中实现接近动态规划的最优控制且优于基线算法。

摘要:

许多最优控制问题需要同时输出离散和连续控制变量。这些问题通常被表述为混合整数最优控制(MIOC)问题,由于解空间的复杂性,求解具有挑战性。诸如分支定界之类的数值方法计算成本高昂,不适合实时控制。本文提出了一种用于MIOC问题的新型混合动作强化学习(HARL)算法——双延迟深度确定性actor-Q(TD3AQ)。TD3AQ结合了actor-critic和Q-learning方法的优势,能够同时处理离散和连续动作空间。所提出的算法在插电式混合动力汽车(PHEV)能量管理问题上进行了评估,其中离散变量(离合器接合/分离和换挡)和连续变量(发动机扭矩)的实时控制对于在满足驾驶约束的同时最大化燃油经济性至关重要。仿真结果表明,与动态规划(DP)相比,TD3AQ实现了接近最优的控制结果,差异仅为4.69%。此外,它超越了基线强化学习算法的性能。

英文摘要:

Many optimal control problems require the simultaneous output of discrete and continuous control variables. These problems are usually formulated as mixed-integer optimal control (MIOC) problems, which are challenging to solve due to the complexity of the solution space. Numerical methods such as branch-and-bound are computationally expensive and undesirable for real-time control. This paper proposes a novel hybrid-action reinforcement learning (HARL) algorithm, twin delayed deep deterministic actor-Q (TD3AQ), for MIOC problems. TD3AQ combines the advantages of both actor-critic and Q-learning methods, and can handle the discrete and continuous action spaces simultaneously. The proposed algorithm is evaluated on a plug-in hybrid electric vehicle (PHEV) energy management problem, where real-time control of the discrete variables, clutch engagement/disengagement and gear shift, and continuous variable, engine torque, is essential to maximize fuel economy while satisfying driving constraints. Simulation outcomes demonstrate that TD3AQ achieves control results close to optimality when compared with dynamic programming (DP), with just 4.69% difference. Furthermore, it surpasses the performance of baseline reinforcement learning algorithms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑