We show that differentially private stochastic gradient descent (DP-SGD) can yield poorly calibrated, overconfident deep learning models. This represents a serious issue for safety-critical applications, e.g. in medical diagnosis. We highlight and exploit parallels between stochastic gradient Langevin dynamics, a scalable Bayesian inference technique for training deep neural networks, and DP-SGD, in order to train differentially private, Bayesian neural networks with minor adjustments to the original (DP-SGD) algorithm. Our approach provides considerably more reliable uncertainty estimates than DP-SGD, as demonstrated empirically by a reduction in expected calibration error (MNIST $\sim{5}$-fold, Pediatric Pneumonia Dataset $\sim{2}$-fold).
Tighter Generalization Bounds for Iterative Differentially Private Learning Algorithms
UBTECH Sydney AI Centre, School of Computer Science, Faculty of Engineering, the University of Sydney(悉尼大学工程学院计算机科学学院优必选悉尼人工智能中心);University of Science and Technology of China(中国科学技术大学)
Communication-Efficient and Differentially Private Vertical Federated Learning with Zeroth-Order Optimization
Elmore Family School of Electrical and Computer Engineering, Purdue University(电气与计算机工程学院,普渡大学);Department of Computer Science and Engineering, Yonsei University(计算机科学与工程系,延世大学)
ScoreShield: Differentially Private Release of Similarity Scores
School of Engineering and Applied Sciences, Harvard University(哈佛大学工程与应用科学学院);School of Engineering, École Polytechnique Fédérale de Lausanne (EPFL)(洛桑联邦理工学院工程学院)
Differentially Private Decentralized Dataset Synthesis Through Randomized Mixing with Correlated Noise
School of Data and Sciences(数据与科学学院);BRAC University(布拉克大学);Electrical and Computer Engineering(电气与计算机工程);University of California, Santa Barbara(加州大学圣巴巴拉分校);Electrical and Electronic Engineering(电气与电子工程);Bangladesh University Engineering and Technology(孟加拉工程与技术大学)
$(ε, δ)$-Differentially Private Partial Least Squares Regression
Software Competence Center Hagenberg (SCCH) GmbH(哈根伯格软件能力中心有限公司);Faculty of Computer Science and Electrical Engineering(计算机科学与电气工程学院);University of Rostock(罗斯托克大学)
Calibrated Adaptation: Bayesian Stiefel Manifold Priors for Reliable Parameter-Efficient Fine-Tuning
Department of Computer Science, Iowa State University(计算机科学系,爱荷华州立大学);Department of Civil, Construction and Environmental Engineering, Iowa State University(土木、建设与环境工程系,爱荷华州立大学)
SMA-DP: Spectral Memory-Aware Differential Privacy for Deep Learning
Department of Electrical Engineering and Computer Science(电气工程与计算机科学系);University of California, Merced(加州大学默塞德分校);Department of Applied Mathematics(应用数学系)
Content-Aware Differential Privacy with Conditional Invertible Neural Networks
Heidelberg University Hospital(海德堡大学医院);Ruprecht-Karls University Heidelberg(海德堡鲁普雷希特-卡尔斯大学);DZHK (German Centre for Cardiovascular Research)(德国心血管研究中心)
Stabilization of Perturbed Loss Function: Differential Privacy without Gradient Noise
Department of Computer Science and Engineering, University of Texas at Arlington(计算机科学与工程系,德克萨斯大学阿灵顿分校);School of Electrical, Computer and Energy Engineering, Arizona State University(电气、计算机与能源工程学院,亚利桑那州立大学)
Validated Synthetic Patient Generation for Small Longitudinal Cohorts: Coagulation Dynamics Across Pregnancy
Robert F. Smith School of Chemical and Biomolecular Engineering, Cornell University(康奈尔大学罗伯特·F·史密斯化学与生物分子工程学院);Department of Biochemistry, The Robert Larner, M.D. College of Medicine, University of Vermont Medical Center(佛蒙特大学医学中心罗伯特·拉纳医学院生物化学系);Department of Obstetrics, Gynecology, and Reproductive Sciences, The Robert Larner, M.D. College of Medicine, University of Vermont Medical Center(佛蒙特大学医学中心罗伯特·拉纳医学院妇产科与生殖科学系)
DECODE: Dual-Enhanced Conditioned Diffusion for EEG Forecasting
Concordia Institute for Information Systems Engineering (CIISE), Concordia University, Canada. Department of Electrical;Computer Engineering, University of Toronto, Canada