arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1905.08464stat.MLcs.AIcs.LGmath.DS
收藏

基于梯度共轭先验的深度神经网络抗离群点鲁棒性

Robustness Against Outliers For Deep Neural Networks By Gradient Conjugate Priors

Peoples’ Friendship University of Russia(俄罗斯人民友谊大学);Free University Berlin(柏林自由大学)

Pavel Gurevich, Hannes Stuke

摘要:

我们分析了一种新的鲁棒的方法重建的概率分布的观测数据中存在的输出离群值。它基于所谓的梯度共轭先验(GCP)网络,该网络输出先验的参数。通过严格研究GCP学习过程的动力学,我们导出了一个明确的公式,用于校正所获得的边缘分布的方差,并消除训练集中离群值引起的偏差。假设一个高斯(输入依赖)地面真值分布污染的比例$\varepsilon$的离群值,我们表明,拟合的平均值是在一个$c e^{-1/\varepsilon}$ -附近的地面真值平均值和校正方差是在一个$b\varepsilon$ -附近的地面真值方差,而未经校正的方差的边缘分布甚至可以是无限的。我们明确地发现$b$作为GCP网络的输出的函数,而没有离群值(可能依赖于输入)分布的先验知识。合成和真实世界的数据集的实验表明,GCP网络与标准的优化器拟合优于其他强大的回归方法。

英文摘要:

We analyze a new robust method for the reconstruction of probability distributions of observed data in the presence of output outliers. It is based on a so-called gradient conjugate prior (GCP) network which outputs the parameters of a prior. By rigorously studying the dynamics of the GCP learning process, we derive an explicit formula for correcting the obtained variance of the marginal distribution and removing the bias caused by outliers in the training set. Assuming a Gaussian (input-dependent) ground truth distribution contaminated with a proportion $\varepsilon$ of outliers, we show that the fitted mean is in a $c e^{-1/\varepsilon}$-neighborhood of the ground truth mean and the corrected variance is in a $b\varepsilon$-neighborhood of the ground truth variance, whereas the uncorrected variance of the marginal distribution can even be infinite. We explicitly find $b$ as a function of the output of the GCP network, without a priori knowledge of the outliers (possibly input-dependent) distribution. Experiments with synthetic and real-world data sets indicate that the GCP network fitted with a standard optimizer outperforms other robust methods for regression.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑