arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2010.03343cs.IRcs.CL
收藏

切片感知神经排序

Slice-Aware Neural Ranking

TU Delft(代尔夫特理工大学)

Gustavo Penha, Claudia Hauff

摘要:

通过错误分析了解神经排名模型何时以及为什么会在IR任务中失败是研究周期的重要组成部分。在这里,我们专注于以下挑战:(i)识别神经排名器无效的困难实例(一对问题和回答候选者)的类别,以及(ii)改进此类实例的神经排名。为了解决这两个挑战,我们求助于基于切片的学习,其目标是提高数据切片(子集)的神经模型的有效性。我们通过提出不同的切片函数(SF)来解决挑战(i),这些切片函数选择数据集的切片-基于先前的工作,我们以启发式的方式捕获神经排序器的不同故障。然后,对于挑战(ii),我们调整神经排名模型来学习切片感知表示,即调整后的模型学习基于模型对问题和响应所属切片的预测来不同地表示问题和响应。我们在三个不同的排名任务和四个语料库上的实验结果(源代码和数据可在https://github.com/Guzpenha/slice_based_learning上获得)表明,基于切片的学习比不知道切片的神经排名器平均提高了2%的有效性。

英文摘要:

Understanding when and why neural ranking models fail for an IR task via error analysis is an important part of the research cycle. Here we focus on the challenges of (i) identifying categories of difficult instances (a pair of question and response candidates) for which a neural ranker is ineffective and (ii) improving neural ranking for such instances. To address both challenges we resort to slice-based learning for which the goal is to improve effectiveness of neural models for slices (subsets) of data. We address challenge (i) by proposing different slicing functions (SFs) that select slices of the dataset---based on prior work we heuristically capture different failures of neural rankers. Then, for challenge (ii) we adapt a neural ranking model to learn slice-aware representations, i.e. the adapted model learns to represent the question and responses differently based on the model's prediction of which slices they belong to. Our experimental results (the source code and data are available at https://github.com/Guzpenha/slice_based_learning) across three different ranking tasks and four corpora show that slice-based learning improves the effectiveness by an average of 2% over a neural ranker that is not slice-aware.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑