arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2006.02734cs.LGstat.ML
收藏

深度学习中的鲁棒采样

Robust Sampling in Deep Learning

Universidad Carlos III de Madrid(马德里卡洛斯三世大学);Swiss Data Science Institute (ETHZ/EPFL)(瑞士数据科学研究所(苏黎世联邦理工学院/洛桑联邦理工学院))

Aurora Cobo Aguilera, Antonio Artés-Rodríguez, Fernando Pérez-Cruz, Pablo Martínez Olmos

摘要:

深度学习需要正则化机制来减少过拟合并提高泛化能力。我们解决这个问题的一个新的正则化方法的基础上分布鲁棒优化。其核心思想是修改每个样本的贡献,以收紧经验风险界。在随机训练过程中,样本的选择是根据它们的准确度来完成的,这样一来,表现最差的样本就是在优化中贡献最大的样本。我们研究了不同的场景,并展示了它可以使收敛速度更快或提高精度的场景。

英文摘要:

Deep learning requires regularization mechanisms to reduce overfitting and improve generalization. We address this problem by a new regularization method based on distributional robust optimization. The key idea is to modify the contribution from each sample for tightening the empirical risk bound. During the stochastic training, the selection of samples is done according to their accuracy in such a way that the worst performed samples are the ones that contribute the most in the optimization. We study different scenarios and show the ones where it can make the convergence faster or increase the accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑