发表机构
University of North Carolina at Charlotte（北卡罗来纳大学夏洛特分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
提出AgentPrivArena框架，通过轨迹级指标和运行时审计方法评估LLM智能体在真实工作流中的隐私风险，实验发现现有基准忽视的显著隐私问题。
AI 中文摘要
大语言模型（LLM）智能体的快速发展使系统能够通过外部工具自主执行复杂任务，但其对个人数据日益增长的访问权限引入了显著的隐私风险。现有基准主要通过模拟轨迹和基于结果的指标来评估LLM智能体隐私，这限制了其捕捉多步智能体执行过程中出现的隐私风险的能力。在本工作中，我们提出了AgentPrivArena，一个用于评估现实LLM智能体工作流中隐私风险的框架。AgentPrivArena在可复现的执行环境中集成了真实的MCP工具和自托管服务。我们进一步提出了轨迹级隐私指标，用于量化最终响应泄露之外的冗余信息访问。基于该框架，我们引入了AgentPrivAudit，一种用于在智能体执行期间监控隐私违规的运行时审计方法。对最先进的LLM智能体进行的大量实验揭示了现有评估范式所忽视的实质性隐私风险，强调了轨迹级审计对于可信智能体部署的重要性。
英文摘要
The rapid advancement of LLM agents has enabled systems to autonomously perform complex tasks through external tools, but their growing access to personal data introduces significant privacy risks. Existing benchmarks primarily evaluate LLM agent privacy through simulated trajectories and outcome-based metrics, limiting their ability to capture privacy risks arising during multi-step agent execution. In this work, we introduce AgentPrivArena, a framework for evaluating privacy risks in realistic LLM agent workflows. AgentPrivArena integrates authentic MCP tools and self-hosted services within a reproducible execution environment. We further propose trajectory-level privacy metrics that quantify unnecessary information access beyond final response leakage. Building on this framework, we introduce AgentPrivAudit, a runtime auditing approach for monitoring privacy violations during agent execution. Extensive experiments on state-of-the-art LLM agents reveal substantial privacy risks overlooked by existing evaluation paradigms, highlighting the importance of trajectory-level auditing for trustworthy agent deployment.
