SecDTD:用于安全Transformer推理的动态令牌丢弃
SecDTD: Dynamic Token Drop for Secure Transformers Inference
AI总结:
本文提出SecDTD,通过动态令牌丢弃技术在安全计算中减少推理成本,采用Max-Centric Normalization和OMSel协议提升效率,实现4.47倍的推理加速且不降低准确性。
摘要:
Transformer-based AI的快速普及由如ChatGPT等易得模型推动,为开发者和企业提供了基于API的服务。然而,随着这些在线推理服务越来越多地处理敏感输入,隐私问题已成为重大挑战。为此,已提出安全推理框架,但其高计算和通信开销常限制实际部署。在明文环境下,令牌丢弃是一种有效降低推理成本的技术;然而,我们的分析表明,直接将其应用于密文场景效果不佳,因为安全计算中的成本分布不同。我们提出SecDTD,一种针对安全Transformer推理的动态令牌丢弃方案。SecDTD通过将丢弃过程移至更早的推理阶段,有效减少Softmax等关键组件的成本。为此,我们引入两种核心技术:Max-Centric Normalization(MCN)是一种新的、不依赖Softmax的评分方法,能够以最小的开销实现早期令牌丢弃,并改进归一化,支持更激进的丢弃而不影响准确性。OMSel:一种更快的、无偏的中位数选择协议,安全地确定重要性分数的中位数以支持令牌丢弃。与现有基于排序的方法相比,OMSel实现了16.9倍的速度提升,同时保持安全、无偏和随机性。我们通过使用BOLT和BumbleBee框架,在八个GLUE数据集上进行48次实验,评估SecDTD在不同网络设置下的表现。SecDTD在不降低准确性的情况下实现了4.47倍的端到端推理加速。
英文摘要:
The rapid adoption of Transformer-based AI has been driven by accessible models such as ChatGPT, which provide API-based services for developers and businesses. However, as these online inference services increasingly handle sensitive inputs, privacy concerns have emerged as a significant challenge. To address this, secure inference frameworks have been proposed, but their high computational and communication overhead often limit practical deployment. In plaintext settings, token drop is an effective technique for reducing inference cost; however, our analysis reveals that directly applying such methods to ciphertext scenarios is suboptimal due to distinct cost distributions in secure computation. We propose SecDTD, a dynamic token drop scheme tailored for secure Transformer inference. SecDTD advances token drop by shifting the dropping to earlier inference stages, effectively reducing the cost of key components such as Softmax. To support this, we introduce two core techniques. Max-Centric Normalization (MCN): A novel, Softmax-independent scoring method that enables early token drop with minimal overhead and improved normalization, supporting more aggressive dropping without accuracy loss. OMSel: A faster, oblivious median selection protocol that securely identifies the median of importance scores to support token drop. Compared to existing sorting-based methods, OMSel achieves a 16.9$\times$ speedup while maintaining security, obliviousness and randomness. We evaluate SecDTD through 48 experiments across eight GLUE datasets under various network settings using the BOLT and BumbleBee frameworks. SecDTD achieves 4.47 times end-to-end inference acceleration without degradation in accuracy.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。