通过强化学习保障代码水印:SWaRL
SWaRL: Safeguard Code Watermarking via Reinforcement Learning
AI总结:
SWaRL通过强化学习框架实现稳健且保真度高的代码水印,保护大语言模型的知识产权,通过在生成程序中嵌入唯一可验证签名,有效抵御移除攻击并保持功能正确性。
摘要:
我们提出了SWaRL,一种稳健且保真度保持的水印框架,旨在通过在生成程序中嵌入唯一且可验证的签名来保护代码大语言模型的知识产权。现有水印方法要么依赖手工编码转换,要么在推理时操纵token生成概率,使其易受移除攻击且可能破坏功能正确性。为解决这些问题,SWaRL采用基于强化学习的联合训练框架,利用编译器反馈维持功能正确性,并使用联合训练的保密验证器作为奖励信号以保持水印可检测性。此外,SWaRL在微调过程中采用低秩适应(LoRA),使水印行为能够高效集成并跨模型更新保持可转移性。大量实验表明,SWaRL在水印检测准确性方面优于先前方法,同时完全保持水印代码的功能性。此外,SWaRL对重构和对抗性转换攻击具有强鲁棒性,能可靠地进行归因,且无需显著计算开销。
英文摘要:
We present SWaRL, a robust and fidelity-preserving watermarking framework designed to protect the intellectual property of code LLMs by embedding unique and verifiable signatures in the generated program. Existing watermarking approaches either rely on handcrafted code transformations or manipulate token generation probabilities at inference time, making them vulnerable to removal attacks or prone to breaking functional correctness. To address these challenges, SWaRL employs a reinforcement learning-based co-training framework that uses compiler feedback for functional correctness and a jointly trained confidential verifier as a reward signal to maintain watermark detectability. Furthermore, SWaRL employs low-rank adaptation (LoRA) during fine-tuning, enabling efficient integration of watermarking behavior and transferability across model updates. Extensive experiments show that SWaRL achieves strong watermark detection accuracy compared to prior methods while fully maintaining watermarked code functionality. Moreover, SWaRL exhibits strong resilience against refactoring and adversarial transformation attacks, which maintains reliable attribution without substantial computational overhead.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。