Decepticons：被破坏的Transformer在语言模型联邦学习中破坏隐私
Decepticons: Corrupted Transformers Breach Privacy in Federated Learning for Language Models
AI总结：
本文提出一种利用恶意参数向量攻击联邦学习文本模型的新方法，通过分别提取token和位置嵌入恢复用户私有文本，揭示文本联邦学习比以往认为的更易受隐私攻击。
摘要：
联邦学习（FL）的一个核心原则是在不集中用户数据的情况下训练模型，从而保护隐私。然而，先前的工作表明，FL中使用的梯度更新可能会泄露用户信息。虽然FL在工业界的大多数应用是文本应用（例如击键预测），但几乎所有针对FL隐私的攻击都集中在简单的图像分类器上。我们提出了一种新颖的攻击方法，通过部署恶意参数向量来揭示私有用户文本，并且即使在小批量、多用户和长序列的情况下也能成功。与先前针对FL的攻击不同，该攻击利用了Transformer架构和token嵌入的特性，分别提取token和位置嵌入，以检索高保真文本。这项工作表明，历史上被认为对隐私攻击具有抵抗力的文本联邦学习，其脆弱性远超之前的认知。
英文摘要：
A central tenet of Federated learning (FL), which trains models without centralizing user data, is privacy. However, previous work has shown that the gradient updates used in FL can leak user information. While the most industrial uses of FL are for text applications (e.g. keystroke prediction), nearly all attacks on FL privacy have focused on simple image classifiers. We propose a novel attack that reveals private user text by deploying malicious parameter vectors, and which succeeds even with mini-batches, multiple users, and long sequences. Unlike previous attacks on FL, the attack exploits characteristics of both the Transformer architecture and the token embedding, separately extracting tokens and positional embeddings to retrieve high-fidelity text. This work suggests that FL on text, which has historically been resistant to privacy attacks, is far more vulnerable than previously thought.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
