arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.13065cs.CL
收藏

基于解耦表示的检索增强分类

Retrieval-Augmented Classification with Decoupled Representation

Beihang University(北京航空航天大学);ByteDance Inc.(字节跳动公司);Harbin Institute of Technology(哈尔滨工业大学)

Xinnian Liang, Shuangzhi Wu, Hui Huang, Jiaqi Bai, Chao Bian, Zhoujun Li

AI总结:

本文提出一种基于k近邻的检索增强分类方法,通过解耦分类与检索的表示来提升性能与训练稳定性,并在多个数据集上验证了其有效性和鲁棒性。

摘要:

检索增强方法已在多种分类任务中展现出有前景的结果。然而,现有方法侧重于检索额外上下文来丰富输入,这种方式对噪声敏感且不可扩展。本文沿此思路,提出一种基于k近邻(KNN)的检索增强分类方法,该方法将预测的标签分布与检索到的实例的标签分布进行插值。与标准KNN过程不同,我们提出一种解耦机制,因为我们发现用于分类和检索的共享表示会损害性能并导致训练不稳定。我们在广泛的分类数据集上评估了所提方法。实验结果表明了所提方法的有效性和鲁棒性。我们还进行了额外实验,以分析模型中不同组件的贡献。

英文摘要:

Retrieval augmented methods have shown promising results in various classification tasks. However, existing methods focus on retrieving extra context to enrich the input, which is noise sensitive and non-expandable. In this paper, following this line, we propose a $k$-nearest-neighbor (KNN) -based method for retrieval augmented classifications, which interpolates the predicted label distribution with retrieved instances' label distributions. Different from the standard KNN process, we propose a decoupling mechanism as we find that shared representation for classification and retrieval hurts performance and leads to training instability. We evaluate our method on a wide range of classification datasets. Experimental results demonstrate the effectiveness and robustness of our proposed method. We also conduct extra experiments to analyze the contributions of different components in our model.\footnote{\url{https://github.com/xnliang98/knn-cls-w-decoupling}}

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑