arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.01042cs.CRcs.LG
收藏

TruncFormer:仅使用截断的私有 LLM 推理

TruncFormer: Private LLM Inference Using Only Truncations

New York University(纽约大学)

Patrick Yubeaton, Jianqiao Cambridge Mo, Karthik Garimella, Nandan Kumar Jha, Brandon Reagen, Chinmay Hegde, Siddharth Garg

AI总结:

针对 LLM 私有推理中非线性函数导致高延迟的问题,本文提出 TruncFormer,用加法、乘法和截断近似非线性并静态规划截断位置,从而降低密码协议延迟。

摘要:

私有推理(PI)在用户与 LLM 等专有机器学习模型交互时,对保证用户数据隐私发挥着重要作用。然而,由于 LLM 中存在的非线性函数会带来巨大的延迟成本,PI 在实践中仍然难以实现。现有工作侧重于通过近似来改进特定 LLM 非线性操作(例如 Softmax 或 GeLU)的延迟。但是,新型非线性操作会随着新的 LLM 架构不断出现,这导致了一场持续的追赶游戏:PI 研究人员试图优化最新的非线性函数。我们提出 TruncFormer,这是一个可将任意 LLM 转换为 PI 的明文仿真的框架。我们的框架利用了这样一个事实:LLM 中的非线性操作是可微的,并且可以通过一系列加法、乘法和截断被精确近似。此外,我们将加法/乘法与截断操作解耦,并根据给定的字段大小和输入表示大小,静态确定应在何处插入截断。相较于在每次乘法操作后都强制执行截断的现有密码协议,这带来了延迟改进。我们开源了代码,供社区使用。

英文摘要:

Private inference (PI) serves an important role in guaranteeing the privacy of user data when interfacing with proprietary machine learning models such as LLMs. However, PI remains practically intractable due to the massive latency costs associated with nonlinear functions present in LLMs. Existing works have focused on improving latency of specific LLM nonlinearities (such as the Softmax, or the GeLU) via approximations. However, new types of nonlinearities are regularly introduced with new LLM architectures, and this has led to a constant game of catch-up where PI researchers attempt to optimize the newest nonlinear function. We introduce TruncFormer, a framework for taking any LLM and transforming it into a plaintext emulation of PI. Our framework leverages the fact that nonlinearities in LLMs are differentiable and can be accurately approximated with a sequence of additions, multiplications, and truncations. Further, we decouple the add/multiply and truncation operations, and statically determine where truncations should be inserted based on a given field size and input representation size. This leads to latency improvements over existing cryptographic protocols that enforce truncation after every multiplication operation. We open source our code for community use.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑