BraveGuard: 从开放世界威胁到更安全的计算机使用代理
BraveGuard: From Open-World Threats to Safer Computer-Use Agents
AI总结:
提出BraveGuard框架,通过从开放世界威胁信号和真实代理轨迹中训练防护模型,实现轨迹级别的安全检测,显著提升计算机使用代理的安全性。
摘要:
计算机使用代理将语言模型从文本生成扩展到与文件、终端、浏览器和外部工具的持续交互。这种转变带来了安全风险,这些风险难以从孤立的提示或最终响应中检测出来,因为危害通常只在多步执行轨迹中显现,而单个动作在局部看似无害。我们引入了BraveGuard,一个自我进化的防御框架,用于从开放世界威胁信号和真实代理轨迹中训练防护模型。BraveGuard挖掘近期研究来源以识别新兴风险和攻击模式,将其实例化为可执行的计算机使用任务,收集代理轨迹,并为防护模型训练提供轨迹级别的监督。随着新威胁和验证失败的出现,可以重复该流程,形成一个自适应防御循环,而不是静态的、基准驱动的训练过程。我们通过训练多个防护骨干模型(包括Qwen3-Guard和Llama-Guard变体)来实例化BraveGuard,并在轨迹级别的代理安全基准上评估生成的防护模型。BraveGuard在计算机使用轨迹上持续提高了安全检测能力。在AgentHazard上,与现成的防护模型相比,它显著提高了检测准确性,在平均防护模型设置下,准确率从38.79%提升到82.38%。这些结果表明,基于开放世界威胁发现和真实代理执行的防护监督可以超越固定分类法和合成提示级数据,改进安全监控。BraveGuard为面对不断变化的现实世界风险的计算机使用代理提供了一条可扩展的自适应防御路径。
英文摘要:
Computer-use agents extend language models from text generation to sustained interaction with files, terminals, browsers, and external tools. This shift creates safety risks that are difficult to detect from isolated prompts or final responses, because harm often emerges only through multi-step execution traces whose individual actions appear locally benign. We introduce BraveGuard, a self-evolving defense framework for training guard models from open-world threat signals and realistic agent trajectories. BraveGuard mines recent research sources to identify emerging risks and attack patterns, instantiates them as executable computer-use tasks, collects agent rollouts, and derives trajectory-level supervision for guard model training. As new threats and validation failures appear, the pipeline can be repeated, yielding an adaptive defense loop rather than a static, benchmark-driven training process. We instantiate BraveGuard by training multiple guard backbones, including Qwen3-Guard and Llama-Guard variants, and evaluate the resulting guards on trajectory-level agent-safety benchmarks. BraveGuard consistently improves safety detection across computer-use trajectories. On AgentHazard, it substantially improves detection accuracy over off-the-shelf guard models, with accuracy increasing from 38.79% to 82.38% under the averaged guard-model setting. These results show that guard supervision grounded in open-world threat discovery and realistic agent execution can improve safety monitoring beyond fixed taxonomies and synthetic prompt-level data. BraveGuard offers a scalable path toward adaptive defenses for computer-use agents facing evolving real-world risks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。