arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2207.08768cs.LG
收藏

机器学习中基于排序的可分解损失:综述

Rank-based Decomposable Losses in Machine Learning: A Survey

Indiana University-Purdue University Indianapolis(印第安纳大学与普渡大学印第安纳波利斯联合分校);Carnegie Mellon University(卡内基梅隆大学);University at Buffalo, SUNY(纽约州立大学布法罗分校)

Shu Hu, Xin Wang, Siwei Lyu

AI总结:

本文综述机器学习中基于排序的可分解损失,从个体损失与聚合损失视角提出八类分类体系,梳理公式、相关文献及未来方向。

摘要:

近期工作揭示了损失函数设计中的一个重要范式,即区分个体损失与聚合损失。个体损失衡量模型在单个样本上的质量,而聚合损失则组合每个训练样本上的个体损失/分数。二者具有一个共同过程,即将一组个体值聚合为单个数值。在损失设计中,排序顺序反映了个体值之间最基本的关系。此外,可分解性是指一个损失可以被分解为一组个体项的集合,这一性质已成为组织损失/分数的重要属性。本综述对机器学习中基于排序的可分解损失进行了系统而全面的回顾。具体而言,我们遵循聚合损失与个体损失的视角,提出了一种新的损失函数分类体系。我们识别出形成此类损失的聚合器,它们是集合函数的实例。我们将基于排序的可分解损失组织为八个类别。围绕这些类别,我们回顾了有关基于排序的聚合损失和基于排序的个体损失的文献。我们描述了这些损失的一般公式,并将它们与现有研究主题联系起来。我们还提出了未来研究方向,涵盖基于排序的可分解损失中尚未探索的问题、遗留问题以及新兴问题。

英文摘要:

Recent works have revealed an essential paradigm in designing loss functions that differentiate individual losses vs. aggregate losses. The individual loss measures the quality of the model on a sample, while the aggregate loss combines individual losses/scores over each training sample. Both have a common procedure that aggregates a set of individual values to a single numerical value. The ranking order reflects the most fundamental relation among individual values in designing losses. In addition, decomposability, in which a loss can be decomposed into an ensemble of individual terms, becomes a significant property of organizing losses/scores. This survey provides a systematic and comprehensive review of rank-based decomposable losses in machine learning. Specifically, we provide a new taxonomy of loss functions that follows the perspectives of aggregate loss and individual loss. We identify the aggregator to form such losses, which are examples of set functions. We organize the rank-based decomposable losses into eight categories. Following these categories, we review the literature on rank-based aggregate losses and rank-based individual losses. We describe general formulas for these losses and connect them with existing research topics. We also suggest future research directions spanning unexplored, remaining, and emerging issues in rank-based decomposable losses.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑