Accepted to the 6th AAAI Workshop on Privacy-Preserving Artificial Intelligence (PPAI), 2025. arXiv admin note: substantial text overlap with arXiv:2410.13060
针对私有推理中非线性运算导致的高通信与延迟开销，论文用 Shannon 熵分析其对训练稳定和注意力头多样性的作用，并提出熵引导注意力、熵正则化及 PI 友好归一化方案。
摘要：
专有语言模型的普及引发了严重的隐私担忧，因此需要推进私有推理（PI）：在不泄露用户敏感信息的情况下，直接在加密数据上执行计算。尽管 PI 提供了一种有前景的解决方案，但其实际部署受到巨大通信和延迟开销的阻碍，这些开销主要来自非线性运算。为此，我们引入一个信息论框架，用以刻画仅解码器语言模型中非线性的作用，从而为针对 PI 需求优化 transformer 架构奠定有原则的基础。通过利用 Shannon 熵作为定量度量，我们揭示了非线性此前未被探索的双重意义：除确保训练稳定性外，它们对于维持注意力头多样性也至关重要。具体而言，我们发现移除非线性会触发两种关键失效模式：深层中的“熵坍缩”，使训练不稳定；以及较早层中的“熵过载”，导致多头注意力（MHA）的表示能力利用不足。我们提出一种熵引导注意力机制，并配合一种新颖的熵正则化技术来缓解熵过载。此外，我们探索了对 PI 友好的层归一化替代方案，以防止熵坍缩并稳定减少非线性的 LLM 训练。本研究弥合了信息论与架构设计之间的差距，确立了熵动态作为开发高效 PI 架构的原则性指导。代码和实现可在 https://github.com/Nandan91/entropy-guided-attention-llm 获取。
英文摘要：
The pervasiveness of proprietary language models has raised critical privacy concerns, necessitating advancements in private inference (PI), where computations are performed directly on encrypted data without revealing users' sensitive information. While PI offers a promising solution, its practical deployment is hindered by substantial communication and latency overheads, primarily stemming from nonlinear operations. To address this, we introduce an information-theoretic framework to characterize the role of nonlinearities in decoder-only language models, laying a principled foundation for optimizing transformer-architectures tailored to the demands of PI.
By leveraging Shannon's entropy as a quantitative measure, we uncover the previously unexplored dual significance of nonlinearities: beyond ensuring training stability, they are crucial for maintaining attention head diversity. Specifically, we find that their removal triggers two critical failure modes: {\em entropy collapse} in deeper layers that destabilizes training, and {\em entropic overload} in earlier layers that leads to under-utilization of Multi-Head Attention's (MHA) representational capacity.
We propose an entropy-guided attention mechanism paired with a novel entropy regularization technique to mitigate entropic overload. Additionally, we explore PI-friendly alternatives to layer normalization for preventing entropy collapse and stabilizing the training of LLMs with reduced-nonlinearities. Our study bridges the gap between information theory and architectural design, establishing entropy dynamics as a principled guide for developing efficient PI architectures. The code and implementation are available at https://github.com/Nandan91/entropy-guided-attention-llm
University of Florida（佛罗里达大学）；New Jersey Institute of Technology（新泽西理工学院）
Recent development in the field of explainable artificial intelligence (XAI) has helped improve trust in Machine-Learning-as-a-Service (MLaaS) systems, in which an explanation is provided together with the model prediction in response to each query. However, XAI also opens a door for adversaries to gain insights into the black-box models in MLaaS, thereby making the models more vulnerable to several attacks. For example, feature-based explanations (e.g., SHAP) could expose the top important features that a black-box model focuses on. Such disclosure has been exploited to craft effective backdo…
Private-RAG: Answering Multiple Queries with LLMs while Keeping Your Data Private
Computer Science and Engineering University of California, San Diego（计算机科学与工程大学加州大学圣地亚哥分校）；Halıcıoğlu Data Science Institute University of California, San Diego（Halıcıoğlu数据科学研究所加州大学圣地亚哥分校）；Department of Computer Science University of California, Los Angeles（计算机科学系加州大学洛杉矶分校）
University of California San Diego（加州大学圣迭戈分校）；Microsoft Research（微软研究院）；Microsoft（微软）
Recent papers have shown that large pre-trained language models (LLMs) such as BERT, GPT-2 can be fine-tuned on private data to achieve performance comparable to non-private models for many downstream Natural Language Processing (NLP) tasks while simultaneously guaranteeing differential privacy. The inference cost of these models -- which consist of hundreds of millions of parameters -- however, can be prohibitively large. Hence, often in practice, LLMs are compressed before they are deployed in specific applications. In this paper, we initiate the study of differentially private model compres…
Northwestern University（西北大学）；George Washington University（乔治华盛顿大学）；Yale University（耶鲁大学）；University of Miami（迈阿密大学）
Motivated by the human-machine interaction such as training chatbots for improving customer satisfaction, we study human-guided human-machine interaction involving private information. We model this interaction as a two-player turn-based game, where one player (Alice, a human) guides the other player (Bob, a machine) towards a common goal. Specifically, we focus on offline reinforcement learning (RL) in this game, where the goal is to find a policy pair for Alice and Bob that maximizes their expected total rewards based on an offline dataset collected a priori. The offline setting presents two…
Recent large-scale natural language processing (NLP) systems use a pre-trained Large Language Model (LLM) on massive and diverse corpora as a headstart. In practice, the pre-trained model is adapted to a wide array of tasks via fine-tuning on task-specific datasets. LLMs, while effective, have been shown to memorize instances of training data thereby potentially revealing private information processed during pre-training. The potential leakage might further propagate to the downstream tasks for which LLMs are fine-tuned. On the other hand, privacy-preserving algorithms usually involve retraini…
Reconstruction of Differentially Private Text Sanitization via Large Language Models
Nanjing University of Science and Technology（南京理工大学）；Western Sydney University（西澳大学）；Institute of Information Engineering（信息工程研究所）；Chinese Academy of Sciences（中国科学院）；Adelaide University（阿德莱德大学）；Responsible AI Research (RAIR) Centre, The University of Adelaide（负责任人工智能研究中心，阿德莱德大学）
Gradient Transformer: Learning to Generate Updates for LLMs
Department of Data Science, New Jersey Institute of Technology, Newark, NJ, USA（数据科学系，新泽西理工学院，新泽西州诺克斯维尔）；Qatar Computing Research Institute, HBKU, Doha, Qatar（卡塔尔计算研究所，HBKU，多哈）
HESTIA: A Hessian-Guided Differentiable Quantization-Aware Training Framework for Extremely Low-Bit LLMs
School of Software and Microelectronics, Peking University, Beijing, China（软件与微电子学院，北京大学，北京，中国）；School of Computer Science, Peking University, Beijing, China（计算机科学学院，北京大学，北京，中国）
