arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.01296cs.CRcs.AIcs.LG
收藏

激进压缩使大语言模型权重窃取成为可能

Aggressive Compression Enables LLM Weight Theft

University of Pennsylvania(宾夕法尼亚大学);Georgia Institute of Technology(佐治亚理工学院);Center for AI Safety(人工智能安全中心)

Davis Brown, Juan-Pablo Rivera, Dan Hendrycks, Mantas Mazeika

AI总结:

本文研究了通过压缩使大语言模型权重窃取风险增加的问题,提出通过放松解压缩约束来提升窃取效率,并探讨了三种防御方法以降低窃取风险。

摘要:

随着前沿AI变得越来越强大和昂贵,攻击者有越来越大的动机通过网络窃取模型权重。在本工作中,我们考虑了一种窃取模型权重的攻击,攻击者试图通过网络将模型权重从数据中心中偷偷带出。虽然窃取攻击是多步骤的网络攻击,但我们证明了一个单一因素,即模型权重的可压缩性,显著提高了大型语言模型(LLMs)的窃取风险。我们通过放松解压缩约束来专门针对窃取进行压缩,并证明攻击者可以以16倍到100倍的压缩率进行压缩,而代价很小,将攻击者从防御者服务器非法传输模型权重所需的时间从几个月减少到几天。最后,我们研究了三种不同的防御方法,以减少窃取风险:使模型更难压缩、使它们更难找到,以及通过取证水印跟踪溯源以进行事后分析。尽管所有防御措施都具有前景,但取证水印防御既有效又便宜,因此是减轻权重窃取风险特别有吸引力的杠杆。

英文摘要:

As frontier AIs become more powerful and costly to develop, adversaries have increasing incentives to steal model weights by mounting exfiltration attacks. In this work, we consider exfiltration attacks where an adversary attempts to sneak model weights out of a datacenter over a network. While exfiltration attacks are multi-step cyber attacks, we demonstrate that a single factor, the compressibility of model weights, significantly heightens exfiltration risk for large language models (LLMs). We tailor compression specifically for exfiltration by relaxing decompression constraints and demonstrate that attackers could achieve 16x to 100x compression with minimal trade-offs, reducing the time it would take for an attacker to illicitly transmit model weights from the defender's server from months to days. Finally, we study defenses designed to reduce exfiltration risk in three distinct ways: making models harder to compress, making them harder to 'find,' and tracking provenance for post-attack analysis using forensic watermarks. While all defenses are promising, the forensic watermark defense is both effective and cheap, and therefore is a particularly attractive lever for mitigating weight-exfiltration risk.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑