PIRS：基于物理信息奖励塑形的SAC建筑能源管理
PIRS: Physics-Informed Reward Shaping for SAC-Based Building Energy Management
AI总结：
针对深度强化学习中奖励函数设计缺乏物理基础的问题，提出PIRS方法，将ISO 7730 PMV公式嵌入SAC的多目标奖励中，提升可解释性和性能。
摘要：
居住者舒适度和电网感知的能效是相互竞争的目标，其联合优化关键取决于深度强化学习（DRL）控制器中奖励函数的指定方式。然而，奖励设计在很大程度上仍然是临时的：舒适度项要么是手动调整的启发式规则，要么是简单的温度偏差代理，缺乏热舒适物理的明确基础。我们提出PIRS（物理信息奖励塑形），它在用于Soft Actor-Critic（SAC）的加权多目标奖励中，用ISO 7730预测平均投票（PMV）公式替代这些临时的舒适度代理。通过将舒适度信号锚定在ISO 7730 PMV公式中，PIRS提高了奖励的可解释性，并在不改变学习流程任何其他组件的情况下，提供了一个基于标准的舒适度代理。我们在CityLearn v2.1.2（2022年挑战赛第一阶段）中评估PIRS，使用一个中央SAC智能体在五个随机种子上训练50k步，并与基于规则的控制器（RBC）、手动设计的奖励（E2）、仅能量奖励（E3）和朴素温度偏差舒适度奖励（E4）进行比较。区域级关键绩效指标（KPI）以与RBC的比率报告显示，PIRS在成本、碳和电力指标上与手动基线相当，同时显著优于非物理基础的设计——特别是在负载爬坡（1.78倍 vs. ~2.4倍RBC）和日峰值需求方面。所有DRL策略在此训练预算下仍高于RBC；我们诚实地解释这一差距，并将PIRS定位为可解释、符合标准的奖励设计基础，而非在有限计算下优于经典控制的声明。
英文摘要：
Occupant comfort and grid-aware energy efficiency are competing objectives whose joint optimization depends critically on how reward functions are specified in deep reinforcement learning (DRL) controllers for buildings. Yet reward design remains largely ad hoc: comfort terms are either hand-tuned heuristics or simple temperature-deviation proxies without explicit grounding in thermal-comfort physics. We present PIRS (Physics-Informed Reward Shaping), which replaces these ad-hoc comfort proxies with the ISO 7730 Predicted Mean Vote (PMV) formulation inside a weighted multi-objective reward for Soft Actor-Critic (SAC). By anchoring the comfort signal in the ISO 7730 PMV formulation, PIRS improves reward interpretability and provides a standards-grounded comfort proxy without changing any other component of the learning pipeline. We evaluate PIRS in CityLearn v2.1.2 (challenge 2022 phase 1) with a central SAC agent trained for 50k steps over five random seeds, and compare against a rule-based controller (RBC), a manually engineered reward (E2), an energy-only reward (E3), and a naive temperature-deviation comfort reward (E4). District-level key performance indicators (KPIs), reported as ratios versus RBC, show that PIRS attains cost, carbon, and electricity metrics on par with the manual baseline while substantially outperforming non-physics-grounded designs -- particularly on load ramping (1.78x vs. ~2.4x RBC) and daily peak demand. All DRL policies remain above RBC at this training budget; we interpret this gap honestly and position PIRS as an interpretable, standards-aligned foundation for reward design rather than a claim of dominance over classical control at limited compute.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
