近年来,大语言模型(Large Language Models, LLMs)性能大幅提升,这得益于模型规模的扩大和训练数据的丰富。这一进展引发了各行业和公众的广泛关注与应用。然而,机器学习模型的训练数据记忆程度随模型规模增长而提升,这对LLMs而言尤为值得关注。被记忆的文本序列可能直接从LLMs中泄露,对数据隐私构成严重威胁。目前已开发出多种攻击LLMs以提取其训练数据的技术。随着这些模型规模持续增长,这一问题变得愈发关键。为帮助研究人员和政策制定者了解隐私攻击与缓解措施的研究现状,包括仍需开展更多工作的方向,我们提出了首个针对LLMs数据隐私的SoK(系统知识梳理)。我们:(i)确定了LLMs攻击差异的显著维度分类体系;(ii)系统化梳理现有攻击方法,利用我们的维度分类体系突出关键趋势;(iii)调研现有缓解策略,阐明其优势与局限性;(iv)指出关键研究空白,明确待解决的问题和值得关注的领域。
英文摘要:
Large Language Models (LLMs) have shown greatly enhanced performance in recent years, attributed to increased size and extensive training data. This advancement has led to widespread interest and adoption across industries and the public. However, training data memorization in Machine Learning models scales with model size, particularly concerning for LLMs. Memorized text sequences have the potential to be directly leaked from LLMs, posing a serious threat to data privacy. Various techniques have been developed to attack LLMs and extract their training data. As these models continue to grow, this issue becomes increasingly critical. To help researchers and policymakers understand the state of knowledge around privacy attacks and mitigations, including where more work is needed, we present the first SoK on data privacy for LLMs. We (i) identify a taxonomy of salient dimensions where attacks differ on LLMs, (ii) systematize existing attacks, using our taxonomy of dimensions to highlight key trends, (iii) survey existing mitigation strategies, highlighting their strengths and limitations, and (iv) identify key gaps, demonstrating open problems and areas for concern.
RAG-CT: Mitigating Privacy Risks on Retrieval-Augmented Generation Systems via Scanning Prompt Distribution
University of Massachusetts Lowell(马萨诸塞大学洛厄尔分校);City University of Hong Kong(香港城市大学);Virginia Tech(弗吉尼亚理工大学);University of South Florida(南佛罗里达大学);Rochester Institute of Technology(罗切斯特理工学院)
The Double-edged Sword of LLM-based Data Reconstruction: Understanding and Mitigating Contextual Vulnerability in Word-level Differential Privacy Text Sanitization
Technical University of Munich School of Computation, Information(慕尼黑技术大学计算、信息与技术学院)
Auditing and Mitigating Privacy Leakage in Cloud-Edge Collaborative Decoding
The Hong Kong Polytechnic University(香港理工大学);Institute for AI Industry Research, Tsinghua University(清华大学人工智能产业研究院);School of Software, Tsinghua University(清华大学软件学院)
Shadow in the Cache: Unveiling and Mitigating Privacy Risks of KV-cache in LLM Inference
State Key Laboratory of Blockchain and Data Security(区块链与数据安全国家重点实验室);Zhejiang University(浙江大学);Huawei Technology(华为技术);Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security(杭州高新技术区(滨江)区块链与数据安全研究院)
Confuse the Model, Control the Flow: Understanding and Mitigating Privacy Leakage from LLM Agents with Information Flow Control
University of California, Irvine(加州大学欧文分校);University of California, Santa Cruz(加州大学圣克鲁兹分校);University of California, Davis(加州大学戴维斯分校);University of California, Santa Barbara(加州大学圣巴巴拉分校)
Mitigating Sensitive Information Leakage in LLMs4Code through Machine Unlearning
College of Computer Science and Technology, National University of Defense Technology, Changsha, Hunan, 410073, China(计算机科学与技术学院,国防科技大学,长沙,湖南,410073,中国);Xichang Satellite Launch Center, Xichang, Sichuan, 615000, China(西昌卫星发射中心,西昌,四川,615000,中国);Academy of Military Sciences, Beijing, 100091, China(军事科学院,北京,100091,中国)
Understanding and Mitigating Cross-lingual Privacy Leakage via Language-specific and Universal Privacy Neurons
King Abdullah University of Science and Technology(国王阿卜杜勒阿齐兹科学与技术大学);Provable Responsible AI and Data Analytics (PRADA) Lab(可证明负责任的人工智能与数据分析实验室);University of Copenhagen(哥本哈根大学);University of Science and Technology of China(中国科学技术大学);State University of New York at Buffalo(纽约州立大学布法罗分校);The Hong Kong Polytechnic University(香港理工大学);Zhejiang University(浙江大学)
Parameter-Efficient Fine-Tuning with Differential Privacy for Robust Instruction Adaptation in Large Language Models
Georgia Institute of Technology(佐治亚理工学院);University of Southern California(南加州大学);Dartmouth College(达特茅斯学院);University of Michigan(密歇根大学);University of Pennsylvania(宾夕法尼亚大学)