arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2606.23217cs.CLcs.AI
收藏

MuPPET:多轮对话中LLM助手上下文隐私的基准测试

MuPPET: A Benchmark for Contextual Privacy of LLM Assistants in Multi-Party Conversations

Parameter Lab(参数实验室);University of Rome Tor Vergata(罗马第二大学);University of Oxford(牛津大学);NAVER AI Lab(NAVER AI实验室);KAIST AI(韩国科学技术院人工智能研究所)

Elena Sofia Ruzzetti, Cornelius Emde, Sangdoo Yun, Seong Joon Oh, Martin Gubri

AI总结:

提出MuPPET基准,评估多用户对话中LLM代理的隐私泄露风险,发现模型在多用户场景下泄露更严重,现有防御措施效果有限。

摘要:

LLM代理越来越多地部署在多用户环境中,代表个人用户处理敏感个人数据,例如在群聊中。当这样的代理披露私人信息时,它会立即到达每个群成员。这种风险在结构上比一对一设置更难控制,因为每条私人信息必须适合群中的每个接收者。然而,所有现有的上下文隐私基准只考虑单对话者设置,忽略了多用户隐私风险。我们引入了MuPPET(多用户隐私暴露测试),这是一个用于多轮对话中上下文隐私的基准测试。我们的实验表明,模型在多用户设置中泄露的信息比一对一评估所暗示的要多得多。前沿模型易受攻击,而较小的开源模型(通常因其本地部署处理敏感数据而受到青睐)则更加脆弱。现有的上下文隐私防御措施只能提供部分保护,降低实用性,并且不能解决根本的参与者跟踪问题。

英文摘要:

LLM agents are increasingly deployed in multi-party environments, handling sensitive personal data on behalf of individual users, for instance in group chats. When such an agent discloses private information, it reaches every group member at once. This risk is structurally harder to control than in one-to-one settings, as every piece of private information must be appropriate for every recipient in the group. Yet all existing contextual privacy benchmarks consider only single-interlocutor settings, leaving multi-party privacy risks unmeasured. We introduce MuPPET (Multi-Party Privacy Exposure Testing), a benchmark for contextual privacy in multi-party conversations. Our experiments show that models leak substantially more in multi-party settings than one-to-one evaluations suggest. Frontier models are vulnerable, and smaller open-weights models, often preferred for local deployment with sensitive data, even more so. Existing contextual privacy defences offer only partial protection, degrade utility, and do not resolve the underlying party-tracking problem.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑