现实场景中工具使用LLM代理的数据泄露风险评估
An Evaluation of Data Leakage Risks in Tool-Using LLM Agents in Realistic Scenarios
AI总结：
评估了12个非对抗性任务中AI代理的数据泄露风险，发现所有代理均存在数据安全意识不足、信息过度访问等问题，表明操作数据泄露是独立于对抗性窃取的一阶安全风险。
摘要：
AI代理越来越多地被用于企业和个人场景，可以访问电子邮件、数据库、文档和其他工具，从而读取、更新和传播敏感信息。先前关于代理数据泄露风险的研究大多集中在通过提示注入和越狱进行的对抗性数据窃取。然而，敏感信息也可能在非对抗性使用中暴露，即使在用户发出良性请求时也会产生泄露风险。我们报告了新加坡AI安全研究所和韩国AI安全研究所的联合评估，检查了12个现实、非对抗性任务中的代理数据泄露，涵盖客户支持、DevOps、网络自动化以及企业和个人生产力。评估涵盖了五种风险类型：缺乏数据意识、受众意识、政策合规性、数据最小化和访问边界意识。两个研究所使用独立的测试环境和特定任务的LLM评判标准，测试了一组反映真实部署的常见场景。在测试的三个代理中，没有一个在所有场景中实现完全正确且完全安全的执行。成功的任务完成往往伴随着数据处理失败，例如访问不必要的信息或向不适当的接收者披露信息，表明能力和数据处理安全性应分开评估。定性审查还揭示了声明-行动不匹配、模拟感知行为、用户-模拟器角色反转以及自动评判中的解释差距。总体而言，结果表明操作数据泄露是独立于对抗性窃取的一阶代理安全问题，并为未来代理数据处理安全评估提供了方法论。
英文摘要：
AI agents are increasingly being adopted in enterprise and personal settings with access to emails, databases, documents, and other tools where they can read, update, and disseminate sensitive information. Much of prior research on data leakage risks in agents has focused on adversarial data exfiltration through prompt injections and jailbreaks. However, sensitive information may also be exposed during non-adversarial use, creating leakage risks even when users issue benign requests. We report a joint evaluation by the Singapore AI Safety Institute and the Korea AI Safety Institute examining agent data leakage in 12 realistic, non-adversarial tasks spanning customer support, DevOps, web automation, and enterprise and personal productivity. The evaluation covers five risk types: lack of data awareness, audience awareness, policy compliance, data minimization, and access-boundary awareness. Both institutes tested a common set of scenarios mirroring real-world deployments using independent testing environments and task-specific LLM-judge rubrics. Across the three tested agents, none achieved fully correct and fully safe execution across all scenarios. Successful task completion often coincided with data-handling failures such as accessing unnecessary information or disclosing information to inappropriate recipients, indicating that capability and data-handling safety should be evaluated separately. Qualitative review also revealed claim-action mismatches, simulation-aware behavior, user-simulator role reversal, and interpretation gaps in automated judging. Overall, the results indicate that operational data leakage is a first-order agent-safety concern distinct from adversarial exfiltration and provide a methodology for future evaluations of agent data-handling safety.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
