RT-Safe：实时具身环境中的智能体安全性基准测试

中文摘要 AI 辅助

人工智能智能体的快速进展使得智能体安全性日益受到关注，大量评估工作集中在数字环境上。随着智能体进入物理世界，具身安全性变得愈发重要：故障可能导致人身伤害和昂贵的硬件损坏。除了选择安全动作外，具身智能体还必须在实时约束下运行：物理世界不会在智能体推理时暂停。在推理过程中，随着行人移动和车辆靠近，在观测时看似安全的动作可能在执行前变得不安全。因此，实时具身安全性既取决于决策质量，也取决于决策延迟。我们提出了RT-SAFE，一个用于在实时约束下评估具身智能体安全性的模拟城市基准。RT-SAFE将导航任务与移动的参与者、环境危险和交通规则相结合，同时允许世界在推理和动作执行过程中持续演化。在八个视觉语言模型（VLM）中，智能体实现了较高的任务完成率，但几乎从未安全完成：在最困难的设置下，仅有0.7%的回合未发生安全事件。更引人注目的是，匹配的静态和实时评估分别产生91.3%和94.1%的任务完成率，而实时执行使碰撞增加了12.3倍。这些结果表明，标准任务成功可能掩盖重大的安全失败，且决策延迟本身可能成为物理风险的来源。最后，我们展示了RT-SAFE能够支持离线强化学习训练，在实现强任务完成率的同时大幅降低碰撞率。

英文摘要

Rapid progress in AI agents has brought growing attention to agent safety, with extensive evaluation focused on digital environments. As agents move into the physical world, embodied safety becomes increasingly important: failures can cause human injury and costly hardware damage. Beyond selecting safe actions, embodied agents must also operate under real-time constraints: the physical world does not pause while an agent reasons. As pedestrians move and vehicles approach during inference, an action that appears safe at observation time may become unsafe before execution. Real-time embodied safety therefore depends on both decision quality and decision latency. We introduce RT-SAFE, a simulated urban benchmark for evaluating embodied-agent safety under real-time constraints. RT-SAFE combines navigation tasks with moving actors, environmental hazards, and traffic rules, while allowing the world to evolve throughout inference and action execution. Across eight VLMs, agents achieve high task completion yet almost never complete safely: in the hardest setting, only 0.7% of episodes finish without a safety event. More strikingly, matched static and real-time evaluations yield task completion rates of 91.3% and 94.1%, respectively, while real-time execution increases collisions by $12.3\times$. These results reveal that standard task success can mask substantial safety failures, and that decision latency itself can become a source of physical risk. Finally, we show that RT-SAFE can support offline RL training and substantially reduce collision rates while achieving strong task completion.

发表机构

• Cornell University（康奈尔大学）
• University of California, San Diego（加利福尼亚大学圣迭戈分校）

机构由 AI 辅助整理，请以论文原文为准。