HEM:一种用于视觉分类任务的基于边界的损失函数
HEM: a margin-based loss for visual categorisation tasks
AI总结:
提出高错误边际(HEM)损失,一种改进的基于边界的损失函数,在多种视觉任务(如未知类拒绝、对抗鲁棒性、不平衡学习、持续学习和语义分割)上优于交叉熵损失,且与专门损失相比具有竞争力。
摘要:
在分类任务上训练深度神经网络(DNN)可以使用多种不同的损失函数,但交叉熵(CE)损失是事实上的标准。这里,我们提出一种替代损失——高错误边际(HEM),这是一种基于边界的损失,经过修改以改善神经网络的训练动态。HEM损失通过使用广泛的DNN架构和基准数据集进行了全面评估,所有实验设置和训练超参数均来自文献,因此针对CE损失进行了优化。结果发现,HEM在一系列基于图像的任务中比CE损失更有效:未知类拒绝、对抗鲁棒性、不平衡数据学习、持续学习和语义分割(一种像素级分类任务)。HEM仅在平衡训练数据下的干净和损坏图像分类方面劣于CE,且差异很小。我们还将HEM与先前提出的用于改善特定视觉任务性能的专门损失进行了比较。LogitNorm是一种在未知类拒绝上达到最先进性能的损失,在该任务上与HEM表现相似,但在持续学习和语义分割上表现差得多。针对不平衡数据设计的Logit-adjusted损失在该任务上优于HEM,但在未知类拒绝和语义分割上表现较差。DICE是一种流行的语义分割损失,在所有任务(包括语义分割)上均劣于HEM损失。总体而言,HEM在我们使用的所有任务中与最佳替代损失具有竞争力,并且在拒绝分布外样本、持续学习方面优于所有其他测试的损失,在语义分割上则显著优于它们。
英文摘要:
Training deep neural networks (DNNs) on classification tasks can be performed with a number of different losses, but cross-entropy (CE) loss is the de-facto standard. Here, we propose an alternative loss, high error margin (HEM), which is a margin based loss modified to improve the training dynamics of neural networks. HEM loss is evaluated extensively using a wide range of DNN architectures and benchmark datasets with all experimental settings and training hyper-parameters taken from the literature, and hence, optimised for CE loss. HEM is found to be more effective than CE loss across a range of image-based tasks: unknown class rejection, adversarial robustness, learning with imbalanced data, continual learning, and semantic segmentation (a pixel-wise classification task). HEM is inferior to CE only in terms of clean and corrupt image classification with balanced training data, and this difference is small. We also compare HEM to specialised losses that have previously been proposed to improve performance for specific vision tasks. LogitNorm, a loss achieving state-of-the-art performance on unknown class rejection, produces similar performance to HEM for this task, but is much poorer for continual learning and semantic segmentation. Logit-adjusted loss, designed for imbalanced data, has superior results to HEM for that task, but performs worse on unknown class rejection and semantic segmentation. DICE, a popular loss for semantic segmentation, is inferior to HEM loss on all tasks, including semantic segmentation. Overall, HEM is competitive with the best alternative loss for all the tasks we have used and performs better than all other tested losses in terms of rejecting out-of-distribution examples, for continual learning, and by a substantial margin for semantic segmentation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。