arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.09214cs.CL
收藏

通过多源信息挖掘与双层对比学习提升短文本分类性能

Boosting Short Text Classification with Multi-Source Information Exploration and Dual-Level Contrastive Learning

Jilin University(吉林大学);Heriot-Watt University(赫瑞瓦特大学);University of Trento(特伦托大学)

Yonghao Liu, Mengyu Li, Wei Pang, Fausto Giunchiglia, Lan Huang, Xiaoyue Feng, Renchu Guan

AI总结:

针对短文本分类的语义稀疏与标注不足问题,提出融合多源信息、图学习与双层对比学习的MI-DELIGHT模型,通过分层架构建模任务关联,性能超越现有模型及部分大语言模型。

摘要:

短文本分类作为自然语言处理的一个研究子方向,在实际场景中因语义稀疏性和标注样本不足而更具挑战性。本文提出了一种名为MI-DELIGHT的新型短文本分类模型。具体而言,该模型首先开展多源信息(即统计信息、语言信息和事实信息)挖掘以缓解稀疏性问题。随后,采用图学习方法学习以图形式呈现的短文本的表示。此外,我们引入了双层(即实例级和聚类级)对比学习辅助任务,以有效捕捉海量无标注数据中不同粒度的对比信息。同时,以往模型仅并行执行主任务和辅助任务,未考虑任务间的关联。因此,我们引入了一种分层架构来显式建模任务之间的相关性。我们在多个基准数据集上开展了大量实验,结果表明MI-DELIGHT显著优于以往的主流竞争模型,在部分数据集上甚至超过了流行的大语言模型。

英文摘要:

Short text classification, as a research subtopic in natural language processing, is more challenging due to its semantic sparsity and insufficient labeled samples in practical scenarios. We propose a novel model named MI-DELIGHT for short text classification in this work. Specifically, it first performs multi-source information (i.e., statistical information, linguistic information, and factual information) exploration to alleviate the sparsity issues. Then, the graph learning approach is adopted to learn the representation of short texts, which are presented in graph forms. Moreover, we introduce a dual-level (i.e., instance-level and cluster-level) contrastive learning auxiliary task to effectively capture different-grained contrastive information within massive unlabeled data. Meanwhile, previous models merely perform the main task and auxiliary tasks in parallel, without considering the relationship among tasks. Therefore, we introduce a hierarchical architecture to explicitly model the correlations between tasks. We conduct extensive experiments across various benchmark datasets, demonstrating that MI-DELIGHT significantly surpasses previous competitive models. It even outperforms popular large language models on several datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑