SafePred: 通过世界模型为计算机使用代理构建一种预测性防护措施
SafePred: A Predictive Guardrail for Computer-Using Agents via World Models
AI总结：
SafePred通过世界模型预测未来风险，构建风险到决策的循环，提升计算机使用代理的安全性和任务效率。
摘要：
随着计算机使用代理（CUAs）在复杂现实环境中的广泛应用，普遍存在的长期风险往往导致严重且不可逆的后果。现有CUAs的防护措施大多采用反应式方法，仅在当前观察空间内限制代理行为。虽然这些防护措施可以防止即时的短期风险（例如点击钓鱼链接），但无法主动避免长期风险：看似合理的行动可能导致延迟出现的高风险后果（例如清理日志导致未来审计不可追溯），而反应式防护措施无法在当前观察空间内识别这些风险。为了解决这些局限性，我们提出了一种预测性防护方法，其核心思想是将预测的未来风险与当前决策对齐。基于这一方法，我们提出了SafePred，一种针对CUAs的预测性防护框架，建立了风险到决策的循环，以确保安全的代理行为。SafePred支持两个关键能力：（1）短期和长期风险预测：通过使用安全策略作为风险预测的基础，SafePred利用世界模型的预测能力生成短期和长期风险的语义表示，从而识别并剪枝导致高风险状态的行动；（2）决策优化：通过步骤级干预和任务级重新规划，将预测的风险转化为可操作的安全决策指导。大量实验表明，SafePred显著减少了高风险行为，实现了超过97.6%的安全性能，相比反应式基线提高了多达21.4%的任务效用。
英文摘要：
With the widespread deployment of Computer-using Agents (CUAs) in complex real-world environments, prevalent long-term risks often lead to severe and irreversible consequences. Most existing guardrails for CUAs adopt a reactive approach, constraining agent behavior only within the current observation space. While these guardrails can prevent immediate short-term risks (e.g., clicking on a phishing link), they cannot proactively avoid long-term risks: seemingly reasonable actions can lead to high-risk consequences that emerge with a delay (e.g., cleaning logs leads to future audits being untraceable), which reactive guardrails cannot identify within the current observation space. To address these limitations, we propose a predictive guardrail approach, with the core idea of aligning predicted future risks with current decisions. Based on this approach, we present SafePred, a predictive guardrail framework for CUAs that establishes a risk-to-decision loop to ensure safe agent behavior. SafePred supports two key abilities: (1) Short- and long-term risk prediction: by using safety policies as the basis for risk prediction, SafePred leverages the prediction capability of the world model to generate semantic representations of both short-term and long-term risks, thereby identifying and pruning actions that lead to high-risk states; (2) Decision optimization: translating predicted risks into actionable safe decision guidances through step-level interventions and task-level re-planning. Extensive experiments show that SafePred significantly reduces high-risk behaviors, achieving over 97.6% safety performance and improving task utility by up to 21.4% compared with reactive baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
