ScaleOT：结合动态LayerReplace与选择性秩压缩的隐私-效用可扩展异地调优方法
ScaleOT: Privacy-utility-scalable Offsite-tuning with Dynamic LayerReplace and Selective Rank Compression
AI总结：
针对现有大语言模型异地调优方法存在的性能下降、成本高、隐私保护强度有限的问题，本文提出ScaleOT框架，结合强化学习驱动的分层替换与选择性秩压缩，在实现近无损调优性能的同时提升隐私保护效果。
摘要：
异地调优（offsite-tuning）是一种用于大语言模型（LLM）调优的隐私保护方法，由LLM所有者将有损压缩后的仿真器共享给数据所有者，以开展下游任务调优，该方法可同时保护模型所有者与数据所有者的隐私。但现有异地调优方法通常存在适配性能下降、计算成本高、保护强度有限的问题，其原因在于这类方法要么统一删减LLM层，要么依赖成本高昂的知识蒸馏。为解决上述问题，本文提出ScaleOT——一种全新的隐私-效用可扩展异地调优框架，可有效平衡隐私保护与效用性能。ScaleOT提出了一种新颖的分层有损压缩算法，该算法利用强化学习获取每一层的重要性，采用被称为“协调器（harmonizer）”的轻量级网络替换原始LLM层。通过将重要的原始LLM层与协调器以不同比例组合，ScaleOT可生成适配不同模型规模、实现最优性能的仿真器，以增强隐私保护效果。此外，本文还提出一种秩缩减方法，用于进一步压缩原始LLM层，可在对效用影响可忽略不计的前提下显著提升隐私性。综合实验表明，与全量微调相比，ScaleOT可实现几乎无损的异地调优性能，同时具备更优的模型隐私保护能力。
英文摘要：
Offsite-tuning is a privacy-preserving method for tuning large language models (LLMs) by sharing a lossy compressed emulator from the LLM owners with data owners for downstream task tuning. This approach protects the privacy of both the model and data owners. However, current offsite tuning methods often suffer from adaptation degradation, high computational costs, and limited protection strength due to uniformly dropping LLM layers or relying on expensive knowledge distillation. To address these issues, we propose ScaleOT, a novel privacy-utility-scalable offsite-tuning framework that effectively balances privacy and utility. ScaleOT introduces a novel layerwise lossy compression algorithm that uses reinforcement learning to obtain the importance of each layer. It employs lightweight networks, termed harmonizers, to replace the raw LLM layers. By combining important original LLM layers and harmonizers in different ratios, ScaleOT generates emulators tailored for optimal performance with various model scales for enhanced privacy protection. Additionally, we present a rank reduction method to further compress the original LLM layers, significantly enhancing privacy with negligible impact on utility. Comprehensive experiments show that ScaleOT can achieve nearly lossless offsite tuning performance compared with full fine-tuning while obtaining better model privacy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
