arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.12787cs.CRcs.AI
收藏

LLM-PBE:评估大型语言模型中的数据隐私

LLM-PBE: Assessing Data Privacy in Large Language Models

University of California, Berkeley(加州大学伯克利分校);University of Texas at Austin(德克萨斯大学奥斯汀分校);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);National University of Singapore(新加坡国立大学);Center for AI Safety(AI安全中心);University of Chicago(芝加哥大学)

Qinbin Li, Junyuan Hong, Chulin Xie, Jeffrey Tan, Rachel Xin, Junyi Hou, Xavier Yin, Zhun Wang, Dan Hendrycks, Zhangyang Wang, Bo Li, Bingsheng He, Dawn Song

AI总结:

针对大型语言模型训练数据泄露风险缺乏系统评估的问题,本文提出覆盖全生命周期、攻击防御、数据类型与指标的LLM-PBE工具包,并通过多模型实验分析模型规模、数据特征和时间因素。

摘要:

大型语言模型(LLM)已成为众多领域不可或缺的组成部分,显著推动了数据管理、挖掘和分析应用的发展。然而,它们在处理和解释复杂语言数据方面的强大能力,也使数据隐私方面的紧迫担忧凸显出来,尤其是训练数据意外泄露的风险。尽管这一问题至关重要,但此前尚无文献对LLM中的数据隐私风险进行全面评估。为弥补这一空白,本文提出了LLM-PBE,这是一个专门为系统评估LLM数据隐私风险而设计的工具包。LLM-PBE旨在分析LLM整个生命周期中的隐私问题,融合多种攻击与防御策略,并处理各种数据类型和指标。通过在多个LLM上进行详细实验,LLM-PBE促进了对数据隐私问题的深入探索,揭示了模型规模、数据特征以及不断演变的时间维度等影响因素。本研究不仅丰富了对LLM隐私问题的理解,也为该领域未来研究提供了重要资源。为增进该领域知识的广度,研究结果、资源以及完整技术报告均发布于https://llm-pbe.github.io/,为LLM隐私评估的学术与实践进展提供开放平台。

英文摘要:

Large Language Models (LLMs) have become integral to numerous domains, significantly advancing applications in data management, mining, and analysis. Their profound capabilities in processing and interpreting complex language data, however, bring to light pressing concerns regarding data privacy, especially the risk of unintentional training data leakage. Despite the critical nature of this issue, there has been no existing literature to offer a comprehensive assessment of data privacy risks in LLMs. Addressing this gap, our paper introduces LLM-PBE, a toolkit crafted specifically for the systematic evaluation of data privacy risks in LLMs. LLM-PBE is designed to analyze privacy across the entire lifecycle of LLMs, incorporating diverse attack and defense strategies, and handling various data types and metrics. Through detailed experimentation with multiple LLMs, LLM-PBE facilitates an in-depth exploration of data privacy concerns, shedding light on influential factors such as model size, data characteristics, and evolving temporal dimensions. This study not only enriches the understanding of privacy issues in LLMs but also serves as a vital resource for future research in the field. Aimed at enhancing the breadth of knowledge in this area, the findings, resources, and our full technical report are made available at https://llm-pbe.github.io/, providing an open platform for academic and practical advancements in LLM privacy assessment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑