抑制价差市场变化下基于动作掩蔽与投影的成本感知强化学习电池储能调度
Cost-Aware Reinforcement Learning with Action Masking and Projection for Battery Energy Storage Dispatch under Suppressed-Spread Market Shifts
浏览论文内容
中文总结 AI 辅助
针对价差收窄下电池储能调度，提出将PPO控制器的物理动作掩蔽与投影同经济咨询分离的方法，实验显示其提升T2利润但整体仍低于代理成本MPC，并验证了可行性保障与经济筛选的独立作用。
中文摘要 AI 辅助
电池储能系统（BESS）调度必须在价格价差下降、可用于支付循环成本的利润空间减少的同时，保持运行可行性。我们研究了一种近端策略优化（PPO）控制器，其预选择的物理动作掩蔽和紧急投影与一个因果的、基于预测的经济咨询模块相分离。所有依赖预测的方法都接收相同的因果24步预测和电网侧结算。在五个PPO随机种子下，启用咨询的净利润在336小时的T1和T2窗口内分别为30.59和18.04美元，而代理成本MPC分别为36.77和22.94美元；PPO在两个时期均低于该参考值。咨询将T2利润从16.45美元提高至18.04美元，同时降低了吞吐量，但在T1中无显著影响。在不重叠的周块上，PPO在日、周和混合季节预测下表现稳定，在持续性预测下减弱，且仍低于代理成本MPC。配对诊断将变化定位于观察到的5-10美元/MWh区间，并伴有混合的SoC依赖效应。M0-M6消融实验表明，移除掩蔽会向投影发送数千个不可行请求，而移除两个物理层则暴露了爬坡违规。证据将经济筛选与可行性执行分离，但不声称形式化安全性、生命周期最优老化或RL主导性。
英文摘要
Battery energy storage system (BESS) dispatch must preserve operational feasibility while declining price spreads reduce the margin available to pay for cycling. We study a proximal policy optimization (PPO) controller whose pre-selection physical action mask and emergency projection are separated from a causal, forecast-informed economic advisory. All forecast-dependent methods receive the same causal 24-step forecast and grid-side settlement. Across five PPO seeds, advice-on net profit is 30.59 and 18.04 USD per 336-hour T1 and T2 window, versus 36.77 and 22.94 USD for proxy-cost MPC; PPO remains below this reference in both periods. Advice raises T2 profit from 16.45 to 18.04 USD while reducing throughput, but is immaterial in T1. On disjoint weekly blocks, PPO is stable under daily, weekly, and blended seasonal forecasts, weakens under persistence, and remains below proxy-cost MPC. Paired diagnostics localize changes to the observed 5-10 USD/MWh regime with mixed SoC-dependent effects. An M0-M6 ablation shows that mask removal sends thousands of infeasible requests to projection, while removing both physical layers exposes ramp violations. The evidence separates economic screening from feasibility enforcement without claiming formal safety, lifecycle-optimal aging, or RL dominance.
发表机构
- National Tsing Hua University（国立清华大学）
机构由 AI 辅助整理，请以论文原文为准。
