arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.12178cs.CLcs.AI
收藏

面向极端多标签分类的上下文学习

In-Context Learning for Extreme Multi-Label Classification

Ghent University – imec(根特大学 – imec);Stanford University(斯坦福大学)

Karel D'Oosterlinck, Omar Khattab, François Remy, Thomas Demeester, Chris Develder, Christopher Potts

AI总结:

提出 Infer--Retrieve--Rank 程序并用 DSPy 实现与优化,使语言模型结合检索器以数十个标注示例完成极端多标签分类,在多个基准上达到先进性能。

摘要:

包含数千个类别的多标签分类问题很难仅靠上下文学习解决,因为语言模型(LM)可能缺乏关于精确类别或如何分配这些类别的先验知识,而且通常不可能在提示中演示每个类别。我们提出了一个通用程序 Infer--Retrieve--Rank,它定义了 LM 与检索器之间的多步交互,以高效处理此类问题。我们使用 DSPy 编程模型实现该程序,该模型以声明方式指定上下文系统,并使用 DSPy 优化器通过仅自举数十个少样本示例来针对特定数据集进行调优。我们的主要极端分类程序针对每个任务单独优化,在三个基准(HOUSE、TECH、TECHWOLF)上取得了最先进结果。我们将同一程序应用于一个特性差异很大的基准,也取得了具有竞争力的性能(BioDEX)。与以往工作不同,我们提出的解决方案不需要微调,易于应用于新任务,减轻了提示工程负担,并且只需要数十个标注示例。我们的代码公开于 https://github.com/KarelDO/xmc.dspy。

英文摘要:

Multi-label classification problems with thousands of classes are hard to solve with in-context learning alone, as language models (LMs) might lack prior knowledge about the precise classes or how to assign them, and it is generally infeasible to demonstrate every class in a prompt. We propose a general program, $\texttt{Infer--Retrieve--Rank}$, that defines multi-step interactions between LMs and retrievers to efficiently tackle such problems. We implement this program using the $\texttt{DSPy}$ programming model, which specifies in-context systems in a declarative manner, and use $\texttt{DSPy}$ optimizers to tune it towards specific datasets by bootstrapping only tens of few-shot examples. Our primary extreme classification program, optimized separately for each task, attains state-of-the-art results across three benchmarks (HOUSE, TECH, TECHWOLF). We apply the same program to a benchmark with vastly different characteristics and attain competitive performance as well (BioDEX). Unlike prior work, our proposed solution requires no finetuning, is easily applicable to new tasks, alleviates prompt engineering, and requires only tens of labeled examples. Our code is public at https://github.com/KarelDO/xmc.dspy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑