LLM agents are increasingly deployed in multi-party environments, handling sensitive personal data on behalf of individual users, for instance in group chats. When such an agent discloses private information, it reaches every group member at once. This risk is structurally harder to control than in one-to-one settings, as every piece of private information must be appropriate for every recipient in the group. Yet all existing contextual privacy benchmarks consider only single-interlocutor settings, leaving multi-party privacy risks unmeasured. We introduce MuPPET (Multi-Party Privacy Exposure Testing), a benchmark for contextual privacy in multi-party conversations. Our experiments show that models leak substantially more in multi-party settings than one-to-one evaluations suggest. Frontier models are vulnerable, and smaller open-weights models, often preferred for local deployment with sensitive data, even more so. Existing contextual privacy defences offer only partial protection, degrade utility, and do not resolve the underlying party-tracking problem.
Still Between Us? Evaluating and Improving Voice Assistant Robustness to Third-Party Interruptions
Interdisciplinary Program in Artificial Intelligence, Seoul National University(首尔国立大学人工智能跨学科项目);Department of EE, Yonsei University(延世大学电子工程系);Department of ECE, SNU(首尔国立大学电子与计算机工程系);Department of AI, University of Seoul(首尔大学人工智能系);AIIS, ASRI, INMC, and ISRC, SNU(首尔国立大学人工智能研究所、亚洲研究机构、智能网络研究中心和智能系统研究所以及首尔国立大学)
GateMem: Benchmarking Memory Governance in Multi-Principal Shared-Memory Agents
School of Artificial Intelligence, Jilin University(吉林大学人工智能学院);Shanghai Jiao Tong University(上海交通大学);King Abdullah University of Science and Technology (KAUST)(卡尔斯鲁厄大学);Tsinghua University(清华大学);National University of Singapore(新加坡国立大学)