arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2305.08377cs.CL
收藏

基于大语言模型的文本分类

Text Classification via Large Language Models

Zhejiang University(浙江大学);Amazon(亚马逊公司);Nanyang Technological University(南洋理工大学);Chongqing University(重庆大学)

Xiaofei Sun, Xiaoya Li, Jiwei Li, Fei Wu, Shangwei Guo, Tianwei Zhang, Guoyin Wang

AI总结:

针对大语言模型文本分类性能弱、推理不足及上下文token受限问题,提出CARP渐进推理提示框架,结合kNN示例检索,在多个基准上取得SOTA,低资源与领域适配表现优异。

摘要:

尽管GPT-3等大规模语言模型(LLM)已取得显著成就,但在文本分类任务中,其性能仍远低于微调模型。原因在于:(1)在处理复杂语言现象(如语义强化、对比、反讽等)时缺乏推理能力;(2)上下文学习允许的token数量有限。\n 本文提出线索与推理提示框架(Clue And Reasoning Prompting,CARP)。CARP采用专为处理文本分类中复杂语言现象设计的渐进式推理策略:首先提示LLM寻找表层线索(如关键词、语气、语义关系、指代关系等),再基于这些线索启动诊断性推理过程以得出最终决策。为进一步解决token受限问题,CARP使用在监督数据集上微调的模型进行上下文学习中的k近邻(kNN)示例检索,使模型既能利用LLM的泛化能力,又能借助全标注数据集提供的任务特定证据。值得注意的是,在5个广泛使用的文本分类基准中,CARP在4个上取得了新的SOTA(最优)性能:SST-2上达97.39(提升1.24),AGNews上达96.40(提升0.72),R8上达98.78(提升0.25),R52上达96.95(提升0.6),在MR上的性能与SOTA相当(92.39对比93.3)。更重要的是,我们发现CARP在低资源和领域适配场景下表现出色:具体而言,每个类别仅使用16个示例时,CARP的性能就与每个类别使用1024个示例的监督模型相当。

英文摘要:

Despite the remarkable success of large-scale Language Models (LLMs) such as GPT-3, their performances still significantly underperform fine-tuned models in the task of text classification. This is due to (1) the lack of reasoning ability in addressing complex linguistic phenomena (e.g., intensification, contrast, irony etc); (2) limited number of tokens allowed in in-context learning. In this paper, we introduce Clue And Reasoning Prompting (CARP). CARP adopts a progressive reasoning strategy tailored to addressing the complex linguistic phenomena involved in text classification: CARP first prompts LLMs to find superficial clues (e.g., keywords, tones, semantic relations, references, etc), based on which a diagnostic reasoning process is induced for final decisions. To further address the limited-token issue, CARP uses a fine-tuned model on the supervised dataset for $k$NN demonstration search in the in-context learning, allowing the model to take the advantage of both LLM's generalization ability and the task-specific evidence provided by the full labeled dataset. Remarkably, CARP yields new SOTA performances on 4 out of 5 widely-used text-classification benchmarks, 97.39 (+1.24) on SST-2, 96.40 (+0.72) on AGNews, 98.78 (+0.25) on R8 and 96.95 (+0.6) on R52, and a performance comparable to SOTA on MR (92.39 v.s. 93.3). More importantly, we find that CARP delivers impressive abilities on low-resource and domain-adaptation setups. Specifically, using 16 examples per class, CARP achieves comparable performances to supervised models with 1,024 examples per class.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑