The nonparametric variational information bottleneck (NVIB) provides the foundation for nonparametric variational differential privacy (NVDP), a framework for building privacy-preserving language models. However, the learned latent representations can drift into regions with high information content, leading to poor privacy guarantees, but also low utility due to numerical instability during training. In this work, we introduce a principled parameter clipping strategy to directly address this issue. Our method is mathematically derived from the objective of minimizing the Rényi Divergence (RD) upper bound, yielding specific, theoretically grounded constraints on the posterior mean, variance, and mixture weight parameters. We apply our technique to an NVIB based model and empirically compare it against an unconstrained baseline. Our findings demonstrate that the clipped model consistently achieves tighter RD bounds, implying stronger privacy, while simultaneously attaining higher performance on several downstream tasks. This work presents a simple yet effective method for improving the privacy-utility trade-off in variational models, making them more robust and practical.
When Gradient Clipping Becomes a Control Mechanism for Differential Privacy in Deep Learning
Computer Science, University of California, Merced, CA 95343, USA（计算机科学，加州大学默塞德分校）；Department of Applied Mathematics, University of California, Merced, CA 95343, USA（应用数学系，加州大学默塞德分校）；Department of Innovation（创新部门）；Research, North Carolina State University, Raleigh, NC, USA（研究，北卡罗来纳州立大学，拉洛城）；Automation (MESA) Lab, Department of Mechanical Engineering, School of Engineering, University of California, Merced, CA 95343, USA（自动化（MESA）实验室，机械工程系，工程学院，加州大学默塞德分校）
When Differential Privacy Meets Wireless Federated Learning: An Improved Analysis for Privacy and Convergence
School of Informatics, Xiamen University, China（厦门大学信息学院，中国）；Internet of Things Thrust, Hong Kong University of Science and Technology (Guangzhou), China（香港科技大学（广州）物联网研究组，中国）
Taming Noise-Induced Prototype Degradation for Privacy-Preserving Personalized Federated Fine-Tuning
School of Artificial Intelligence, Beihang University（北京航空航天大学人工智能学院）；School of Statistics, Renmin University of China（中国人民大学统计学院）；School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）
DC-SGD: Differentially Private SGD with Dynamic Clipping through Gradient Norm Distribution Estimation
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Ant Group（蚂蚁集团）；School of Engineering, University of Virginia（弗吉尼亚大学工程学院）
Mitigating Disparate Impact of Differentially Private Learning through Bounded Adaptive Clipping
Department of Computer Science, University of Helsinki（计算机科学系，赫尔辛基大学）；Department of Electrical and Computer Engineering, American University of Beirut（电气与计算机工程系，贝鲁特美国大学）
Double Momentum and Error Feedback for Clipping with Fast Rates and Differential Privacy
University of Basel（巴塞尔大学）；Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)（穆罕默德·本·扎耶德人工智能大学（MBZUAI））；King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学（KAUST））
Exploring the Limits of Differentially Private Deep Learning with Group-wise Clipping
University of Science and Technology of China（中国科学技术大学）；Stanford University（斯坦福大学）；Sun Yat-sen University（中山大学）；Microsoft Research Asia（微软亚洲研究院）；Microsoft Research（微软研究院）
Robust Estimation of Sparse Numerical Vectors under Local Differential Privacy
Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；Zhejiang University（浙江大学）；Guangzhou University（广州大学）；National Interdisciplinary Research Center of Engineering Physics（国家工程物理学跨学科研究中心）；Anhui University of Science and Technology（安徽理工大学）
A Differentially Private Kaplan-Meier Estimator for Privacy-Preserving Survival Analysis
Cancer Registry of Norway（挪威癌症登记处）；Norwegian Institute of Public Health（挪威公共卫生研究院）；University of Southern California（南加州大学）；The Arctic University of Norway（挪威北极大学）
Robust Federated Learning Over the Air: Combating Heavy-Tailed Noise with Median Anchored Clipping
ZJU-UIUC Institute, Zhejiang University, Haining 314400, China（浙江大学-UIUC研究院，浙江大学，海宁314400，中国）；Singapore University of Technology and Design（新加坡科技与设计大学）
DP-aware AdaLN-Zero: Taming Conditioning-Induced Heavy-Tailed Gradients in Differentially Private Diffusion
School of Computer Science（计算机科学学院）；Big Data, Minjiang University, Fuzhou, Fujian, China.（大数据，闽江大学，福州，福建，中国）；School of Information, Renmin University of China, Beijing, China（信息学院，中国人民大学，北京，中国）
Deep Learning under Fractional-Order Differential Privacy
MESA Lab, Dept. of EECS University of California, Merced（加州大学默塞德分校电子工程与计算机科学系MESA实验室）；Dept. of Applied Mathematics University of California, Merced（加州大学默塞德分校应用数学系）
