arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.05786cs.CL
收藏

HiLight:一种具备分层局部对比学习的层级感知轻量全局模型

HiLight: A Hierarchy-aware Light Global Model with Hierarchical Local ConTrastive Learning

Huawei Technologies Ltd. Co.(华为技术有限公司)

Zhijian Chen, Zhonghua Li, Jianxin Yang, Ye Qi

AI总结:

针对分层文本分类中结构编码器参数随分类体系规模激增、递归正则化易坍缩的问题,提出仅含文本编码器和分类头的轻量模型HiLight,通过分层局部对比学习引入层级信息,在两个基准数据集上验证了有效性。

摘要:

分层文本分类(HTC)是多标签分类(MLC)的一个特殊子任务,其分类体系被构建为树状结构,每个样本至少被分配树中的一条路径。最新的HTC模型包含三个模块:文本编码器、结构编码器和多标签分类头。其中,结构编码器专门用于对分类体系的层级结构进行编码。然而,结构编码器存在规模问题:随着分类体系规模增大,近期HTC研究工作中的可学习参数会快速增长。递归正则化是另一种引入层级信息的常用方法,但它存在坍缩问题,通常通过分配较小权重(即1e-6)来缓解。本文提出了一种具备分层局部对比学习的层级感知轻量全局模型(HiLight),这是一种仅由文本编码器和多标签分类头组成的轻量高效全局模型。我们提出了一种用于引入层级信息的新型学习任务,名为分层局部对比学习(HiLCL)。我们在两个基准数据集上开展了大量实验,以验证所提模型的有效性。

英文摘要:

Hierarchical text classification (HTC) is a special sub-task of multi-label classification (MLC) whose taxonomy is constructed as a tree and each sample is assigned with at least one path in the tree. Latest HTC models contain three modules: a text encoder, a structure encoder and a multi-label classification head. Specially, the structure encoder is designed to encode the hierarchy of taxonomy. However, the structure encoder has scale problem. As the taxonomy size increases, the learnable parameters of recent HTC works grow rapidly. Recursive regularization is another widely-used method to introduce hierarchical information but it has collapse problem and generally relaxed by assigning with a small weight (ie. 1e-6). In this paper, we propose a Hierarchy-aware Light Global model with Hierarchical local conTrastive learning (HiLight), a lightweight and efficient global model only consisting of a text encoder and a multi-label classification head. We propose a new learning task to introduce the hierarchical information, called Hierarchical Local Contrastive Learning (HiLCL). Extensive experiments are conducted on two benchmark datasets to demonstrate the effectiveness of our model.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑