用于主动具身安全的自进化即时记忆

中文摘要 AI 辅助

虽然视觉语言模型（VLMs）使具身智能体能够执行复杂的家庭任务，但它们在闭环交互中难以主动应对动态出现的危险。现有安全方法常依赖运行时护栏来阻止不安全行为或导致过度谨慎，严重阻碍任务进展。为打破安全与进展的权衡，我们引入自进化即时记忆框架，将具身安全从阻碍进展的护栏转变为主动减轻危险。该框架由用于部分可观测下持续安全相关状态跟踪的风险充足拓扑信念图（RSG）、用于精确危险预测的基于智能体的事实记忆以及注入程序元技能以指导可执行、保持进展的减轻危险的经验记忆组成。此外，我们提出自动测试-验证-写入循环，让智能体在测试时能从执行轨迹中不断完善减轻危险的元技能。在IS-Bench上的实验表明，我们的框架大幅提高了多个VLM主干的安全成功率（如在Qwen3-VL-8B上提高30.3%），使智能体能够主动减轻危险而不阻碍任务进展。

英文摘要

While Vision-Language Models (VLMs) have empowered embodied agents to execute complex household tasks, they struggle to proactively handle dynamically emerging hazards during closed-loop interactions. Existing safety approaches often rely on runtime guardrails to block unsafe actions or induce excessive caution, which severely stalls task progress instead of actively resolving the underlying risks. To break this safety-progress trade-off, we introduce the Self-Evolving Just-In-Time Memory framework, which reframes embodied safety from progress-stalling guardrails to proactive hazard mitigation. The framework consists of a Risk-Sufficient Topological Belief Graph (RSG) for persistent safety-relevant state tracking under partial observability, an Agency-Grounded Factual Memory for precise hazard anticipation, and an Experience Memory that injects procedural Meta-Skills to guide executable, progress-preserving mitigation. Furthermore, we propose an automated Test-Verify-Write loop, allowing agents to continually refine their mitigation Meta-Skills from execution traces at test time. Experiments on IS-Bench demonstrate that our framework substantially boosts the Safe-Success rate across multiple VLM backbones (e.g., +30.3% on Qwen3-VL-8B), enabling agents to proactively mitigate hazards without stalling task progress. Code is available at https://github.com/DyMessi/JIT-Memory.

发表机构

• Huazhong University of Science and Technology（华中科技大学）
• Tsinghua University（清华大学）
• Shanghai Jiao Tong University（上海交通大学）

机构由 AI 辅助整理，请以论文原文为准。