AI 中文总结
针对现有DERA框架依赖负荷预测易受误差影响的问题，提出无预测RL框架，通过LSVI算法学习最优策略，在CAISO数据实验中实现高跟踪精度与稳定调节。
AI 中文摘要
可再生能源发电的日益波动性提升了对快速灵活的电网平衡机制的需求。现有分布式能源资源聚合（DERA）框架依赖净负荷的短期预测，使其性能对预测误差高度敏感。本文提出一种用于DERA分配的无预测强化学习（RL）框架，该框架直接从运行数据中学习最优策略。我们将DER动态建模为确定性线性系统，将外生净负荷建模为基于特征的线性马尔可夫过程，在无需显式预测的情况下捕捉短程时间依赖关系。我们推导了最优策略的闭式表达式，该表达式通过最小二乘值迭代（LSVI）算法利用多回合收集的数据进行学习。所提框架在保留DER模型可解释性与约束满足性的同时，通过数据驱动的更新适应随机负荷变化。基于真实加州独立系统运营商（CAISO）净负荷数据的数值实验表明，所学习的控制器在无需任何负荷预测的情况下，针对异构DER聚合器实现了高跟踪精度与稳定调节。
英文摘要
The growing variability of renewable generation increases the need for fast and flexible grid-balancing mechanisms. Existing frameworks for distributed energy resource aggregations (DERAs) rely on short-term forecasts of net demand, making their performance highly sensitive to prediction errors. In this paper we present a forecast-free reinforcement learning (RL) framework for DERA allocation that learns optimal policies directly from operational data. We model the DERA dynamics as a deterministic linear system and the exogenous net load as a feature-based linear Markov process, capturing short-range temporal dependencies without explicit forecasting. We derive a closed-form expression for the optimal policy, which is learned through a least-squares value iteration (LSVI) algorithm using data collected across episodes. The proposed framework preserves the interpretability and constraint satisfaction of DER model while adapting to stochastic demand variations through data-driven updates. Numerical experiments on real California Independent System Operator (CAISO) net-demand data demonstrate that the learned controller achieves high tracking accuracy and stable regulation across heterogeneous DER aggregators without requiring any demand prediction.
Comments5 pages. Presented at the 2026 IEEE Power & Energy Society General Meeting (PES GM)
