arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.07283cs.CRcs.CLcs.LG
收藏

一种具备成本效益且自适应的LLM抖动与恢复机制框架

A Framework for Cost-Effective and Self-Adaptive LLM Shaking and Recovery Mechanism

Baidu Inc.(百度公司);Baidu Research(百度研究院)

Zhiyu Chen, Yu Li, Suochao Zhang, Jingbo Zhou, Jiwen Zhou, Chenfu Bao, Dianhai Yu

AI总结:

本研究针对LLM云部署的成本与隐私-准确率权衡问题,提出名为CypherTalk的自适应抖动调优恢复框架,通过水平、垂直抖动算子实现与SOTA隐私方案相当的准确率,是首个兼顾成本与效用-隐私权衡的LLM相关研究。

摘要:

随着大语言模型(LLM)在实际应用中取得巨大成功,越来越多用户寻求通过云服务开发并部署定制化LLM。但在部分特定领域,人们仍对成本,以及隐私问题与准确率之间的权衡存在顾虑。本研究提出了一种具备成本效益的自适应LLM抖动调优与恢复机制,名为CypherTalk。通过精心设计的水平与垂直抖动算子,我们能达到与基于密码学或差分隐私的SOTA(当前最优)隐私保护LLM方案相当的准确率结果。实验还表明,借助CypherTalk框架,用户在使用优化后的抖动算子设置时可获得可靠的准确率。据我们所知,这是首个在LLM场景中考虑成本以及模型效用与隐私之间权衡的研究。

英文摘要:

As Large Language Models (LLMs) gain great success in real-world applications, an increasing number of users are seeking to develop and deploy their customized LLMs through cloud services. Nonetheless, in some specific domains, there are still concerns regarding cost and trade-offs between privacy issues and accuracy. In this study, we introduce a cost-effective and self-adaptive LLM shaking tuning and recovery mechanism, named CypherTalk. With carefully designed horizontal and vertical shaking operators, we can achieve comparable accuracy results with SOTA privacy-preserving LLM schemes using Cryptography-based or Differential Privacy-based methods. Experiments also show that with the CypherTalk framework, users can achieve reliable accuracy when using optimized shaking operator settings. To our best knowledge, this is the first work that considers cost, and trade-off between model utility and privacy in LLM scenarios.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑