arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1907.08982stat.MLcs.LG
收藏

条件密度估计的噪声正则化

Noise Regularization for Conditional Density Estimation

ETH Zurich(苏黎世联邦理工学院);Karlsruhe Institute of Technology (KIT)(卡尔斯鲁厄理工学院)

Jonas Rothfuss, Fabio Ferreira, Simon Boehm, Simon Walther, Maxim Ulrich, Tamim Asfour, Andreas Krause

摘要:

在许多情况下,对条件均值之外的统计关系进行建模至关重要。条件密度估计(CDE)的目的是从数据中学习完整的条件概率密度。尽管基于CDE模型的神经网络具有高度表达性,但在使用极大似然目标进行训练时,会遭受严重的过拟合。由于这些模型的固有结构,在参数空间中的经典正则化方法变得无效。为了解决这个问题,我们为CDE开发了一种模型无关的噪声正则化方法,该方法在训练过程中向数据添加随机扰动。我们证明了所提出的方法对应于光滑正则化,并证明了其渐近一致性。在我们的实验中,噪声正则化在七个数据集和三个CDE模型上显著且一致地优于其他正则化方法。噪声正则化的有效性使得基于神经网络的CDE优于以前的非参数和半参数方法,即使在训练数据很少的情况下。

英文摘要:

Modelling statistical relationships beyond the conditional mean is crucial in many settings. Conditional density estimation (CDE) aims to learn the full conditional probability density from data. Though highly expressive, neural network based CDE models can suffer from severe over-fitting when trained with the maximum likelihood objective. Due to the inherent structure of such models, classical regularization approaches in the parameter space are rendered ineffective. To address this issue, we develop a model-agnostic noise regularization method for CDE that adds random perturbations to the data during training. We demonstrate that the proposed approach corresponds to a smoothness regularization and prove its asymptotic consistency. In our experiments, noise regularization significantly and consistently outperforms other regularization methods across seven data sets and three CDE models. The effectiveness of noise regularization makes neural network based CDE the preferable method over previous non- and semi-parametric approaches, even when training data is scarce.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑