非平稳强化学习的主动上下文预测安全约束
Proactive Context-Forecasted Safety Constraints for Nonstationary Reinforcement Learning
浏览论文内容
中文总结 AI 辅助
针对非平稳强化学习，提出基于上下文预测的主动安全约束生成框架，通过推断和预测环境上下文来提前构建安全约束，在驾驶环境中显著减少碰撞并保持任务性能。
中文摘要 AI 辅助
在非平稳条件下确保强化学习的安全性，需要在风险导致不安全行为之前预判风险变化。现有方法通常依赖于在设计时定义的安全约束或在执行过程中被动更新的约束，并假设这些约束随时间保持有效。然而，在上下文不断演变、驾驶布局不断变化的非平稳环境中，这些假设可能失效。我们提出了一种基于上下文预测的主动安全约束生成框架。该方法从观测中推断潜在的环境上下文，预测其未来演变，并构建适应预期条件的安全约束。这使得智能体能够主动避开不安全区域，而不是仅在安全违规发生后才做出反应。我们在具有结构化上下文变化的驾驶环境中评估了该方法。实验包括对非平稳强度的扫描以及额外的留出驾驶布局，包括高速公路、交叉路口和赛道场景。结果表明，主动约束生成在已见和训练外非平稳强度下均大幅减少碰撞，并且通常在留出驾驶布局中保持有效，同时维持可用的任务性能。这些发现表明，基于上下文的约束生成是非平稳条件下安全强化学习的一种有前景的方法。
英文摘要
Ensuring safety in reinforcement learning under nonstationarity requires anticipating changes in risk before they lead to unsafe behavior. Existing approaches typically rely on safety constraints defined at design time or updated reactively during execution, assuming that such constraints remain valid over time. However, in nonstationary environments with evolving contexts and changing driving layouts, these assumptions may fail. We propose a framework for proactive safety constraint generation based on context forecasting. The approach infers latent environmental context from observations, predicts its future evolution, and constructs safety constraints adapted to anticipated conditions. This enables the agent to proactively avoid unsafe regions instead of reacting only after safety violations occur. We evaluate the method in driving environments with structured context variation. The experiments include a sweep over nonstationarity intensities and additional held-out driving layouts, including highway, intersection, and racetrack scenarios. Results show that proactive constraint generation substantially reduces collisions under both seen and out-of-training nonstationarity intensities and generally remains effective across held-out driving layouts while maintaining usable task performance. These findings suggest that context-based constraint generation is a promising approach for safe reinforcement learning under nonstationarity.
发表机构
- McMaster University（麦克马斯特大学）
机构由 AI 辅助整理，请以论文原文为准。
