arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1903.09668stat.MLcs.LGstat.ME
收藏

贝叶斯深度学习的数据增强

Data Augmentation for Bayesian Deep Learning

University of Chicago(芝加哥大学);Booth School of Business(布斯商学院);George Mason University(乔治梅森大学);Volgenau School of Engineering(沃尔根瑙工程学院)

Yuexi Wang, Nicholas G. Polson, Vadim O. Sokolov

摘要:

深度学习(DL)方法已成为函数逼近和预测的最强大工具之一。虽然DL的表示属性已经得到了很好的研究,但不确定性量化仍然具有挑战性,并且在很大程度上尚未探索。数据增强技术是一种自然的方法,提供不确定性量化,并将随机蒙特卡罗搜索到随机梯度下降(SGD)方法。我们的论文的目的是表明,训练DL架构与数据增强导致效率的提高。我们使用法线的尺度混合理论来推导深度学习的数据增强策略。这使得期望最大化和MCMC算法的变体可以应用于这些高维非线性深度学习模型。为了展示我们的方法,我们为各种常用的激活函数开发了数据增强算法:logit,ReLU,leaky ReLU和SVM。我们的方法相比,传统的随机梯度下降与反向传播。我们的优化过程导致一个版本的迭代重新加权最小二乘法,并可以实现规模与加速线性代数方法提供了显着的速度提高。我们在一些标准数据集上说明了我们的方法。最后,我们总结了未来研究的方向。

英文摘要:

Deep Learning (DL) methods have emerged as one of the most powerful tools for functional approximation and prediction. While the representation properties of DL have been well studied, uncertainty quantification remains challenging and largely unexplored. Data augmentation techniques are a natural approach to provide uncertainty quantification and to incorporate stochastic Monte Carlo search into stochastic gradient descent (SGD) methods. The purpose of our paper is to show that training DL architectures with data augmentation leads to efficiency gains. We use the theory of scale mixtures of normals to derive data augmentation strategies for deep learning. This allows variants of the expectation-maximization and MCMC algorithms to be brought to bear on these high dimensional nonlinear deep learning models. To demonstrate our methodology, we develop data augmentation algorithms for a variety of commonly used activation functions: logit, ReLU, leaky ReLU and SVM. Our methodology is compared to traditional stochastic gradient descent with back-propagation. Our optimization procedure leads to a version of iteratively re-weighted least squares and can be implemented at scale with accelerated linear algebra methods providing substantial improvement in speed. We illustrate our methodology on a number of standard datasets. Finally, we conclude with directions for future research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑