VLM-SAFE: 基于世界模型的视觉-语言模型引导的安全强化学习用于自动驾驶
VLM-SAFE: Vision-Language Model-Guided Safety-Aware Reinforcement Learning with World Models for Autonomous Driving
AI总结：
本文提出VLM-SAFE框架，通过观察-想象-评估-行动闭环，利用视觉语言模型提供语义安全信号，结合世界模型预测未来轨迹，优化策略以提升自动驾驶的安全性和效率。
摘要：
自动驾驶政策学习中的强化学习（RL）本质上受限于低样本效率、弱泛化能力和对不安全在线试错交互的依赖。尽管安全RL引入了显式约束或成本，现有方法往往无法捕捉真实驾驶场景中的安全语义，导致在简单情况中行为保守，在复杂情况中风险意识不足。为了解决这一问题，我们提出了VLM-SAFE，一种离线安全RL框架，遵循人类认知循环的观察-想象-评估-行动流程。从离线驾驶数据出发，VLM-SAFE观察交通场景，并利用视觉语言模型（VLM）提供基于场景理解的语义安全信号。一个学习的世界模型然后从观察的上下文中想象未来的轨迹，使智能体能够在不与真实环境交互的情况下推断可能的后果。而不是仅使用想象的回放来估计回报，VLM-SAFE进一步利用基于VLM的安全指导来评估这些预测的未来，明确将未来预期与语义风险评估联系起来。最终的安全意识想象经验被用于通过actor-critic学习优化策略，使得行动基于预测结果和其安全影响。通过紧密整合观察、想象、评估和行动到一个统一的闭环中，VLM-SAFE实现了更安全、更高效的离线策略学习，用于自动驾驶。大量仿真实验表明，VLM-SAFE在安全性和交通密度变化下的鲁棒性以及安全性能权衡方面优于代表性基线。
英文摘要：
Autonomous driving policy learning with reinforcement learning (RL) is fundamentally limited by low sample efficiency, weak generalization, and a dependence on unsafe online trial-and-error interactions. Although safe RL introduces explicit constraints or costs, existing methods often fail to capture the semantic meaning of safety in real driving scenes, leading to conservative behaviors in simple cases and insufficient risk awareness in complex ones. To address this issue, we propose VLM-SAFE, an offline safe RL framework that follows a human cognitive loop of observe-imagine-evaluate-act. Starting from offline driving data, VLM-SAFE observes traffic scenarios and leverages a vision-language model (VLM) to provide semantic safety signals grounded in scene understanding. A learned world model then imagines future trajectories from the observed context, enabling the agent to reason about possible consequences without interacting with the real environment. Rather than using imagined rollouts solely for return estimation, VLM-SAFE further evaluates these predicted futures with VLM-based safety guidance, explicitly coupling future anticipation with semantic risk assessment. The resulting safety-aware imagined experience is finally used to optimize the policy via actor-critic learning, such that actions are chosen based on both predicted outcomes and their safety implications. By tightly integrating observation, imagination, evaluation, and action into a unified closed loop, VLM-SAFE enables safer and more efficient offline policy learning for autonomous driving. Extensive experiments in simulation show that VLM-SAFE achieves improved safety, stronger robustness under traffic-density shift, and a better safety-performance trade-off than representative baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
