AI 中文总结
研究安全强化学习中罕见灾难性尾部事件难检测的问题，提出SteinGate方法，利用核斯坦因差异进行稳健一致性检查并考虑边界原子，动态调整学习机制，实验证明该方法能降低约束违反频率和严重性且保持竞争力。
AI 中文摘要
安全强化学习通常通过限制预期累积成本来确保安全，这一标准往往无法检测到罕见但灾难性的尾部事件。为克服这些限制，本文引入了SteinGate，一种边界感知分布安全证书，它使用核斯坦因差异进行稳健一致性检查，取代脆弱的尾部拟合，同时考虑由截断成本引起的边界原子。SteinGate评估观察到的策略展开成本是否与安全参考分布保持一致，提供非参数安全证书。该证书用于动态调整学习机制：当展开与安全参考一致时，倾向于奖励改进的策略更新；当成本尾部偏离时，切换到恢复行为。在连续控制基准上的实验表明，SteinGate在训练期间显著降低了约束违反的频率和严重性，同时相对于现有基线保持了有竞争力的回报。
英文摘要
Safe reinforcement learning typically enforces safety by bounding expected cumulative costs, a criterion that often fails to detect rare but catastrophic tail events. To overcome these limitations, this paper introduces SteinGate, a boundary-aware distributional safety certificate that replaces fragile tail fitting with a robust consistency check using Kernelized Stein Discrepancy while accounting for boundary atoms induced by clipped costs. SteinGate evaluates whether observed policy rollout costs remain consistent with a safe reference distribution, providing a non-parametric safety certificate. This certificate is used to dynamically adapt the learning regime: favoring reward-improving policy updates when rollouts remain consistent with the safe reference and switching to recovery behavior when the cost tail deviates. Experiments on continuous-control benchmarks demonstrate that SteinGate significantly reduces both the frequency and severity of constraint violations during training while maintaining competitive returns relative to state-of-the-art baselines.
CommentsAccepted for Publication at the 42nd Conference on Uncertainty in Artificial Intelligence (UAI), 2026
