大型语言模型微调中的隐私：攻击、防御与未来方向
Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directions
AI总结：
本文系统综述了LLM微调阶段的隐私攻击与防御方法，涵盖成员推断、数据提取、后门攻击及差分隐私、联邦学习等机制，并指出未来研究方向。
摘要：
微调已成为利用大型语言模型（LLMs）完成特定下游任务的关键过程，使这些模型能够在各个领域取得最先进的性能。然而，微调过程通常涉及敏感数据集，引入了利用该阶段独特特征的隐私风险。在本文中，我们对与微调LLMs相关的隐私挑战进行了全面综述，重点揭示了面对各种隐私攻击的脆弱性，包括成员推断攻击、数据提取攻击和后门攻击。我们进一步回顾了旨在降低微调阶段隐私风险的防御机制，例如差分隐私、联邦学习和知识遗忘，并讨论了它们在应对隐私风险和保持模型效用方面的有效性与局限性。通过识别现有研究中的关键空白，我们强调了相关挑战并提出了推进微调LLMs隐私保护方法发展的方向，以促进其在多样化应用中的负责任使用。
英文摘要：
Fine-tuning has emerged as a critical process in leveraging Large Language Models (LLMs) for specific downstream tasks, enabling these models to achieve state-of-the-art performance across various domains. However, the fine-tuning process often involves sensitive datasets, introducing privacy risks that exploit the unique characteristics of this stage. In this paper, we provide a comprehensive survey of privacy challenges associated with fine-tuning LLMs, highlighting vulnerabilities to various privacy attacks, including membership inference, data extraction, and backdoor attacks. We further review defense mechanisms designed to mitigate privacy risks in the fine-tuning phase, such as differential privacy, federated learning, and knowledge unlearning, discussing their effectiveness and limitations in addressing privacy risks and maintaining model utility. By identifying key gaps in existing research, we highlight challenges and propose directions to advance the development of privacy-preserving methods for fine-tuning LLMs, promoting their responsible use in diverse applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
