基于Bregman分歧的鲁棒双调和Logistic损失
Robust Bi-Tempered Logistic Loss Based on Bregman Divergences
摘要：
我们在指数函数中引入温度，并将神经网络的softmax输出层替换为高温泛化。类似地，我们用于训练的对数损失中的对数被低温对数取代。通过调节两个温度，我们创建了在单层情况下已经是非凸的损失函数。当用我们对逻辑损失的双温度概括替换神经网络的最后一层时，训练对噪声变得更加鲁棒。我们在一个简单的设置中可视化调整两个温度的效果，并显示我们的方法在大数据集上的有效性。我们的方法是基于Bregman分歧和优于相关的两个温度的方法使用Tsallis分歧。
英文摘要：
We introduce a temperature into the exponential function and replace the softmax output layer of neural nets by a high temperature generalization. Similarly, the logarithm in the log loss we use for training is replaced by a low temperature logarithm. By tuning the two temperatures we create loss functions that are non-convex already in the single layer case. When replacing the last layer of the neural nets by our bi-temperature generalization of logistic loss, the training becomes more robust to noise. We visualize the effect of tuning the two temperatures in a simple setting and show the efficacy of our method on large data sets. Our methodology is based on Bregman divergences and is superior to a related two-temperature method using the Tsallis divergence.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
