发表机构
Imperial College London; University College London（帝国理工学院; 伦敦大学学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对多时间尺度电池系统，提出MPC-RL分层控制框架，高层MPC优化长期设定点，低层RL实时跟踪，延长电池寿命84%并提高利润34%。
AI 中文摘要
多时间尺度系统提出了一个基本挑战，即快速运行决策必须与长期可持续性目标共存。在本工作中，我们提出了一种新的基于模型预测控制（MPC）和强化学习（RL）的分层控制框架，以在两个不同的时间尺度上分离决策。高层MPC在慢动态上优化长期设定点，而在快时间尺度上，一个低层预训练的RL智能体实时跟踪这些设定点以最大化短期目标。引入RL是为了学习非线性控制策略，无需依赖模型线性化，也无需承担重复求解最优控制问题所带来的繁重在线计算。该框架应用于参与调频市场的电池储能系统（BESS），以平衡快速盈利机会（秒级）与缓慢的电池退化（周至月级）。该设计采用一个离线训练的退化感知RL智能体来生成安全的长时域设定点，以及一个从该智能体微调而来的退化不知情智能体用于快速运行时设定点跟踪。与MPC基线相比，所提出的方法成功地将电池寿命延长了84%，并将运营利润提高了34%。
英文摘要
Multi-timescale systems present a fundamental challenge, where fast operational decisions must coexist with long-horizon sustainability targets. In this work, we propose a new hierarchical control framework via Model Predictive Control (MPC) and Reinforcement Learning (RL) to separate decision-making on two distinct timescales. The high-level MPC optimizes long-horizon setpoints at the slow dynamic and on a fast timescale, a low-level pretrained RL agent tracks these setpoints in real time to maximize short-term objectives. RL is introduced to learn nonlinear control policies, without relying on model linearizations or requiring the heavy online computation from solving repeated optimal control problems. The framework is applied to a Battery Energy Storage System (BESS) operating in frequency regulation markets to balance fast profit opportunities (seconds) and slow battery degradation (weeks to months). The design employs a degradation-aware RL agent trained offline to generate safe long-horizon setpoints, and a degradation-unaware agent fine-tuned from it for fast runtime setpoint tracking. Compared to MPC baselines, the proposed approach successfully extends battery lifetime by 84% and increases operational profit by 34%.
