arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.00870cs.CRcs.AI
收藏

ConfusionPrompt:在线大型语言模型的实用隐私推断

ConfusionPrompt: Practical Private Inference for Online Large Language Models

Peihua Mai, Youjia Yang, Ran Yan, Rui Ye, Yan Pang

AI总结:

ConfusionPrompt通过分解提示和生成伪提示,提升在线LLM的隐私保护,实现隐私与效用的平衡。

摘要:

最先进的大型语言模型(LLM)通常作为在线服务部署,要求用户传输详细提示至云端服务器,引发重大隐私问题。为此,我们引入ConfusionPrompt,一种新的隐私保护框架,通过将原始提示分解为子提示,并生成伪提示与真实子提示一同发送至LLM。用户随后将服务器响应重新组合以重建最终输出。该方法相较于现有LLM隐私保护方法具有两大优势:(i)无缝集成现有黑盒LLM;(ii)相较于现有文本扰动方法,隐私-效用权衡显著提升。我们还开发了(λ, μ, ρ)-隐私模型,以规范隐私保护的提示组,并提供复杂性分析以证明提示分解的作用。我们的实证评估显示,ConfusionPrompt在使用开源模型和扰动技术时,效用显著高于本地推断方法,同时相比开源LLM减少了内存消耗。

英文摘要:

State-of-the-art large language models (LLMs) are typically deployed as online services, requiring users to transmit detailed prompts to cloud servers. This raises significant privacy concerns. In response, we introduce ConfusionPrompt, a novel framework for private LLM inference that protects user privacy by: (i) decomposing the original prompt into smaller sub-prompts, and (ii) generating pseudo-prompts alongside the genuine sub-prompts, which are then sent to the LLM. The server responses are later recomposed by the user to reconstruct the final output. This approach offers key advantages over previous LLM privacy protection methods: (i) it integrates seamlessly with existing black-box LLMs, and (ii) it delivers a significantly improved privacy-utility trade-off compared to existing text perturbation methods. We also develop a $(λ, μ, ρ)$-privacy model to formulate the requirements for a privacy-preserving group of prompts and provide a complexity analysis to justify the role of prompt decomposition. Our empirical evaluation shows that ConfusionPrompt achieves significantly higher utility than local inference methods using open-source models and perturbation-based techniques, while also reducing memory consumption compared to open-source LLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑