arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.13197cs.LGcs.CV
收藏

关系替代损失学习

Relational Surrogate Loss Learning

The University of Sydney(悉尼大学);SenseTime Research(商汤科技研究院);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Huazhong University of Science and Technology(华中科技大学);University of Science and Technology of China(中国科学技术大学)

Tao Huang, Zekang Li, Hua Lu, Yong Shan, Shusheng Yang, Yang Feng, Fei Wang, Shan You, Chang Xu

摘要:

机器学习中的评估度量通常很难被视为损失函数,因为它们可能是不可微和不可分解的,例如,平均精度和F1得分。本文旨在通过重新审视替代损失学习来解决这个问题,其中采用深度神经网络来近似评估指标。与其通过深度神经网络来追求评估度量的精确恢复,我们被提醒这些评估度量存在的目的,即区分一个模型比另一个模型更好还是更差。在本文中,我们表明,直接维护代理损失和指标之间的模型的关系就足够了,并提出了一种基于秩相关的优化方法,以最大限度地提高这种关系,并学习代理损失。与以前的作品相比,我们的方法更容易优化,并享有显着的效率和性能增益。大量的实验表明,我们的方法实现了各种任务,包括图像分类和神经机器翻译的改进,甚至优于人类姿势估计和机器阅读理解任务的最先进的方法。代码可在:https://github.com/hunto/ReLoss.

英文摘要:

Evaluation metrics in machine learning are often hardly taken as loss functions, as they could be non-differentiable and non-decomposable, e.g., average precision and F1 score. This paper aims to address this problem by revisiting the surrogate loss learning, where a deep neural network is employed to approximate the evaluation metrics. Instead of pursuing an exact recovery of the evaluation metric through a deep neural network, we are reminded of the purpose of the existence of these evaluation metrics, which is to distinguish whether one model is better or worse than another. In this paper, we show that directly maintaining the relation of models between surrogate losses and metrics suffices, and propose a rank correlation-based optimization method to maximize this relation and learn surrogate losses. Compared to previous works, our method is much easier to optimize and enjoys significant efficiency and performance gains. Extensive experiments show that our method achieves improvements on various tasks including image classification and neural machine translation, and even outperforms state-of-the-art methods on human pose estimation and machine reading comprehension tasks. Code is available at: https://github.com/hunto/ReLoss.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑