SafeVLA: 通过约束学习实现视觉-语言-动作模型的安全对齐
SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learning
AI总结：
本文提出SafeVLA方法，通过约束学习整合安全约束，有效降低安全违规成本并提升任务成功率，同时确保安全性和泛化能力。
摘要：
视觉-语言-动作模型（VLAs）具有成为通用机器人策略的潜力。然而，在现实部署中，这些模型面临极端的安全挑战，包括对环境、机器人和人类的潜在危害。本文通过探索集成安全方法（ISA），系统地建模安全需求，主动引出多样化的不安全行为，通过安全强化学习有效约束VLA策略，并通过针对性评估确保其安全性。利用约束马尔可夫决策过程（CMDP）范式，ISA从min-max视角优化VLAs，以应对引出的安全风险。通过这种综合方法，策略在安全性和性能之间实现有效平衡，减少了83.58%的安全违规累积成本，同时保持任务成功率提升3.85%。此外，该方法具备缓解长尾风险和处理极端失败场景的能力，并在各种分布外扰动中表现出良好的泛化能力。本文在长周期移动操作任务上进行了有效性评估。数据、模型和新提出的基准环境可在https://pku-safevla.github.io获取。
英文摘要：
Vision-language-action models (VLAs) show potential as generalist robot policies. However, these models pose extreme safety challenges during real-world deployment, including the risk of harm to the environment, the robot itself, and humans. How can safety constraints be explicitly integrated into VLAs? We address this by exploring an integrated safety approach (ISA), systematically modeling safety requirements, then actively eliciting diverse unsafe behaviors, effectively constraining VLA policies via safe reinforcement learning, and rigorously assuring their safety through targeted evaluations. Leveraging the constrained Markov decision process (CMDP) paradigm, ISA optimizes VLAs from a min-max perspective against elicited safety risks. Thus, policies aligned through this comprehensive approach achieve the following key features: (I) effective safety-performance trade-offs, reducing the cumulative cost of safety violations by 83.58% compared to the state-of-the-art method, while also maintaining task success rate (+3.85%). (II) strong safety assurance, with the ability to mitigate long-tail risks and handle extreme failure scenarios. (III) robust generalization of learned safety behaviors to various out-of-distribution perturbations. The effectiveness is evaluated on long-horizon mobile manipulation tasks. Our data, models and newly proposed benchmark environment are available at https://pku-safevla.github.io.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
