VLESA: 用于人类活动监测的视觉语言具身安全智能体
VLESA: Vision-Language Embodied Safety Agent for Human Activity Monitoring
AI总结：
提出VLESA框架，通过自我中心视频监测人类活动，利用GRPO训练的目标条件安全Q过滤器进行实时安全干预，在ASIMOV-2.0基准上实现更高干预精度。
摘要：
随着AI系统越来越多地协助人类完成物理任务，确保安全变得至关重要——物理动作会带来即时且不可逆转的后果，而数字错误则不会。我们引入了视觉语言具身安全智能体（VLESA），这是一个从自我中心视频监测人类活动，并在预测到危险动作时触发实时安全干预的框架。VLESA处理意图依赖的安全问题，其中相同的动作可能根据上下文而安全或危险。我们引入了一个将自我中心帧与目标条件安全注释配对的数据集，使得能够通过GRPO训练一个目标条件安全Q过滤器，该过滤器在不重新训练的情况下根据推断的意图评估动作。在此基础上，提出了一个意图-动作预测智能体，用于从视频中联合推断目标并预测未来动作。在ASIMOV-2.0基准上，VLESA在精确的地面真值帧处实现了比基线更高的干预准确率，而通过目标条件约束解码，GRPO训练的Q过滤器将动作安全性提高了超过41个百分点。代码可在该网址获取。
英文摘要：
As AI systems increasingly assist humans in physical tasks, ensuring safety becomes paramount -- physical actions carry immediate and irreversible consequences that digital errors do not. We introduce the Vision-Language Embodied Safety Agent (VLESA), a framework that monitors human activities from egocentric video and triggers real-time safety interventions when dangerous actions are predicted. VLESA addresses intent-dependent safety where identical actions can be safe or dangerous depending on context. A dataset pairing egocentric frames with goal-conditioned safety annotations is introduced, enabling a goal-conditioned safety Q-filter trained via GRPO that evaluates actions with respect to inferred intent without retraining. On top of that, an intent-action prediction agent is proposed to jointly infer goals and predict future actions from video. On the ASIMOV-2.0 benchmark, VLESA achieves higher intervention accuracy at the exact ground-truth frame compared to baselines, while the GRPO-trained Q-filter improves action safety by over 41 percentage points through goal-conditioned constrained decoding. Code is available at https://github.com/HanjiangHu/VLESA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
