arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.08846cs.CRcs.CLcs.LG
收藏

指纹向量:通过向量加法实现可扩展且高效的模型指纹迁移

Fingerprint Vector: Enabling Scalable and Efficient Model Fingerprint Transfer via Vector Addition

Zhenhua Xu, Qichen Liu, Zhebo Wang, Wenpeng Xing, Dezhang Kong, Mohan Li, Meng Han

AI总结:

针对基于继承的模型指纹识别存在的局限,提出指纹向量机制,通过向量加法实现指纹在下游模型的高效迁移,在多架构及下游变体中保持优效性、无害性和鲁棒性,兼具可扩展性。

摘要:

基于后门的指纹识别已成为追踪大型语言模型所有权的有效技术。然而,在实际部署场景中,开发者常从共享基础模型实例化多个下游模型,单独对每个变体应用指纹识别会产生高昂的计算开销。虽然基于继承的方法——将指纹嵌入基础模型并期望其在微调中保留——看似有吸引力,但存在三个关键局限:后期指纹识别、指纹不稳定性及对下游适配的干扰。为应对这些挑战,我们提出一种名为指纹向量的新型机制。该方法首先通过基于后门的微调将指纹嵌入基础模型,然后通过计算带指纹模型与干净模型的差异,提取特定任务的参数增量作为指纹向量。此向量可直接添加到任何结构兼容的下游模型中,无需额外微调即可事后迁移指纹。大量实验表明,指纹向量在关键需求上的性能与直接注入相当或更优。它在不同模型架构及同一家族内的主流下游变体中均保持强有效性,多数情况下还能保留无害性和鲁棒性。即使观察到轻微的鲁棒性下降,其影响仍在可接受范围内,且被该方法的可扩展性优势所抵消。

英文摘要:

Backdoor-based fingerprinting has emerged as an effective technique for tracing the ownership of large language models. However, in real-world deployment scenarios, developers often instantiate multiple downstream models from a shared base model, and applying fingerprinting to each variant individually incurs prohibitive computational overhead. While inheritance-based approaches -- where fingerprints are embedded into the base model and expected to persist through fine-tuning -- appear attractive, they suffer from three key limitations: late-stage fingerprinting, fingerprint instability, and interference with downstream adaptation. To address these challenges, we propose a novel mechanism called the Fingerprint Vector. Our method first embeds a fingerprint into the base model via backdoor-based fine-tuning, then extracts a task-specific parameter delta as a fingerprint vector by computing the difference between the fingerprinted and clean models. This vector can be directly added to any structurally compatible downstream model, allowing the fingerprint to be transferred post hoc without additional fine-tuning. Extensive experiments show that Fingerprint Vector achieves comparable or superior performance to direct injection across key desiderata. It maintains strong effectiveness across diverse model architectures as well as mainstream downstream variants within the same family. It also preserves harmlessness and robustness in most cases. Even when slight robustness degradation is observed, the impact remains within acceptable bounds and is outweighed by the scalability benefits of our approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑