Privacy-R1: 通过强化学习实现隐私意识的多LLM代理协作
Privacy-R1: Privacy-Aware Multi-LLM Agent Collaboration via Reinforcement Learning
AI总结：
本文提出Privacy-R1框架，通过强化学习动态路由文本块，在隐私泄露与任务性能间取得平衡，引入医疗数据集验证方法有效性，实现隐私-效用前沿的新状态。
摘要：
当用户将查询提交给大型语言模型（LLMs）时，其提示可能包含敏感数据，迫使做出艰难抉择：将查询发送给强大的专有LLM提供者以获得最先进的性能并有数据暴露风险，或依赖较小的本地模型以保证数据隐私但通常导致任务性能下降。先前的方法依赖于静态管道，使用LLM重写，这会破坏语言连贯性并 indiscriminately 移除隐私敏感信息，包括任务关键内容。我们把这一挑战（隐私意识委托）重新表述为一个顺序决策问题，并引入一种新的强化学习（RL）框架称为Privacy-R1来解决。我们的框架训练一个代理动态路由文本块，学习一个政策，最优平衡隐私泄露和任务性能之间的权衡。它隐式区分可替换的个人身份信息（PII）（它本地屏蔽）和任务关键PII（它战略性地发送到远程模型以最大化效用）。为了验证我们的方法在复杂场景中的有效性，我们还引入了一个新的医疗数据集，具有高PII密度。我们的框架在隐私-效用前沿上实现了新的状态，证明了在敏感环境中部署LLMs时学习适应性策略的必要性。数据集可在：https://github.com/zackhuiiiii/Privacy-R1找到。
英文摘要：
When users submit queries to Large Language Models (LLMs), their prompts can often contain sensitive data, forcing a difficult choice: Send the query to a powerful proprietary LLM providers to achieving state-of-the-art performance and risk data exposure, or relying on smaller, local models guarantees data privacy but often results in a degradation of task performance. Prior approaches have relied on static pipelines that use LLM rewriting, which shatters linguistic coherence and indiscriminately removes privacy-sensitive information, including task-critical content. We reformulate this challenge (Privacy-Conscious Delegation) as a sequential decision-making problem and introduce a novel reinforcement learning (RL) framework called Privacy-R1 to solve it. Our framework trains an agent to dynamically route text chunks, learning a policy that optimally balances the trade-off between privacy leakage and task performance. It implicitly distinguishes between replaceable Personally Identifiable Information (PII) (which it shields locally) and task-critical PII (which it strategically sends to the remote model for maximal utility). To validate our approach in complex scenarios, we also introduce a new medical dataset with high PII density. Our framework achieves a new state-of-the-art on the privacy-utility frontier, demonstrating the necessity of learned, adaptive policies for deploying LLMs in sensitive environments. Dataset can be found at: https://github.com/zackhuiiiii/Privacy-R1.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
