arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2106.11690cs.LG
收藏

多标签分类中基于属性的标签分箱

Gradient-based Label Binning in Multi-label Classification

TU Darmstadt(达姆施塔特工业大学);JKU Linz(约翰内斯·开普勒大学林茨分校);Paderborn University(帕德博恩大学)

Michael Rapp, Eneldo Loza Mencía, Johannes Fürnkranz, Eyke Hüllermeier

摘要:

在多标签分类中,单个示例可能同时与多个类别标签相关联,对标签之间的依赖关系进行建模的能力被认为是有效优化不可分解的评估措施(如子集0/1损失)的关键。梯度提升框架为专门针对这样的损失函数定制的学习模型提供了充分研究的基础,并且最近的研究证明了在多标记设置中实现高预测准确性的能力。利用二阶导数,如许多最近的提升方法所使用的,有助于指导不可分解损失的最小化,这是由于它将关于标签对的信息并入优化过程中。缺点是,即使标签的数量很小,这也会带来很高的计算成本。在这项工作中,我们解决了这种方法的计算瓶颈-需要解决一个线性方程组-通过将一种新的近似技术集成到升压过程。基于训练期间计算的导数,我们将标签动态地分组到预定义数量的bin中,以对线性系统的维度施加上限。我们使用现有的基于规则的算法进行的实验表明,这可能会提高训练速度,而不会对预测性能造成任何重大损失。

英文摘要:

In multi-label classification, where a single example may be associated with several class labels at the same time, the ability to model dependencies between labels is considered crucial to effectively optimize non-decomposable evaluation measures, such as the Subset 0/1 loss. The gradient boosting framework provides a well-studied foundation for learning models that are specifically tailored to such a loss function and recent research attests the ability to achieve high predictive accuracy in the multi-label setting. The utilization of second-order derivatives, as used by many recent boosting approaches, helps to guide the minimization of non-decomposable losses, due to the information about pairs of labels it incorporates into the optimization process. On the downside, this comes with high computational costs, even if the number of labels is small. In this work, we address the computational bottleneck of such approach -- the need to solve a system of linear equations -- by integrating a novel approximation technique into the boosting procedure. Based on the derivatives computed during training, we dynamically group the labels into a predefined number of bins to impose an upper bound on the dimensionality of the linear system. Our experiments, using an existing rule-based algorithm, suggest that this may boost the speed of training, without any significant loss in predictive performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑