用于深度集成的广义负相关学习
Generalized Negative Correlation Learning for Deep Ensembling
摘要：
包围算法在许多机器学习应用中提供最先进的性能。一个常见的解释是，他们的出色表现是由于偏置方差分解的均方误差，这表明该算法的误差可以分解成其偏差和方差。这两个数量往往是相互对立的，合奏提供了一种有效的方法来管理它们，因为它们通过一组不同的基础学习者来减少差异，同时保持低偏差。尽管已经有许多关于分解其他损失函数的工作，但精确的数学联系很少被明确地用于集成，而仅仅被用作指导原则。在本文中，我们为任意二次可微损失函数制定了广义偏差方差分解，并在深度学习的背景下对其进行了研究。我们使用这种分解，以获得一个广义负相关学习（GNCL）算法，该算法提供了明确的控制集成的多样性和平滑插值之间的两个极端的独立训练和联合训练的集成。我们展示了GNCL如何封装许多以前的作品，并讨论了在何种情况下，神经网络的集成训练可能会失败，以及根据单个网络的选择，应该采用何种集成方法。我们在https://github.com/sbuschjaeger/gncl上公开发布我们的代码
英文摘要：
Ensemble algorithms offer state of the art performance in many machine learning applications. A common explanation for their excellent performance is due to the bias-variance decomposition of the mean squared error which shows that the algorithm's error can be decomposed into its bias and variance. Both quantities are often opposed to each other and ensembles offer an effective way to manage them as they reduce the variance through a diverse set of base learners while keeping the bias low at the same time. Even though there have been numerous works on decomposing other loss functions, the exact mathematical connection is rarely exploited explicitly for ensembling, but merely used as a guiding principle. In this paper, we formulate a generalized bias-variance decomposition for arbitrary twice differentiable loss functions and study it in the context of Deep Learning. We use this decomposition to derive a Generalized Negative Correlation Learning (GNCL) algorithm which offers explicit control over the ensemble's diversity and smoothly interpolates between the two extremes of independent training and the joint training of the ensemble. We show how GNCL encapsulates many previous works and discuss under which circumstances training of an ensemble of Neural Networks might fail and what ensembling method should be favored depending on the choice of the individual networks. We make our code publicly available under https://github.com/sbuschjaeger/gncl
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
