arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.02718cs.CRcs.CL
收藏

“是的,我的LoRD。” 采用局部强化蒸馏引导语言模型提取

"Yes, My LoRD." Guiding Language Model Extraction with Locality Reinforced Distillation

The Hong Kong Polytechnic University(香港理工大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Hong Kong Baptist University(香港 Baptist大学)

Zi Liang, Qingqing Ye, Yanyun Wang, Sen Zhang, Yaxin Xiao, Ronghua Li, Jianliang Xu, Haibo Hu

AI总结:

针对现有大语言模型提取攻击忽略任务对齐差异导致性能不佳的问题,提出局部强化蒸馏算法LoRD,通过策略梯度式训练利用受害模型响应构建本地偏好,理论上保证对齐一致性并降低查询复杂度、削弱水印保护,实验验证其提取商用大模型的优越性。

摘要:

针对大语言模型(LLM)的模型提取攻击(MEA)在近年研究中受到越来越多的关注。然而,现有攻击方法通常采用最初为深度神经网络(DNN)设计的提取策略,忽略了MEA训练任务与LLM对齐任务之间的内在不一致性,导致攻击性能欠佳。为解决这一问题,我们提出了局部强化蒸馏(Locality Reinforced Distillation,LoRD)——一种专门为LLM设计的新型模型提取算法。具体而言,LoRD采用了一种新定义的策略梯度风格训练任务,利用受害模型的响应作为信号,引导构建本地模型的偏好。理论分析表明:其一,LoRD在模型提取中的收敛过程与LLM的对齐过程一致;其二,LoRD可通过我们基于探索的窃取方式降低查询复杂度,同时削弱水印保护机制。大量实验验证了我们的方法在提取多种当前最先进的商用LLM方面的优越性。我们的代码可在以下地址获取:https://github.com/liangzid/LoRD-MEA 。

英文摘要:

Model extraction attacks (MEAs) on large language models (LLMs) have received increasing attention in recent research. However, existing attack methods typically adapt the extraction strategies originally developed for deep neural networks (DNNs). They neglect the underlying inconsistency between the training tasks of MEA and LLM alignment, leading to suboptimal attack performance. To tackle this issue, we propose Locality Reinforced Distillation (LoRD), a novel model extraction algorithm specifically designed for LLMs. In particular, LoRD employs a newly defined policy-gradient-style training task that utilizes the responses of victim model as the signal to guide the crafting of preference for the local model. Theoretical analyses demonstrate that I) The convergence procedure of LoRD in model extraction is consistent with the alignment procedure of LLMs, and II) LoRD can reduce query complexity while mitigating watermark protection through our exploration-based stealing. Extensive experiments validate the superiority of our method in extracting various state-of-the-art commercial LLMs. Our code is available at: https://github.com/liangzid/LoRD-MEA .

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑