神经排序模型中的灾难性遗忘研究
Studying Catastrophic Forgetting in Neural Ranking Models
摘要:
在最近的IR文献中已经提出了几种深度神经排序模型。虽然它们到由数据集持有的一个目标域的可转移性已经使用传统的领域自适应策略被广泛地解决,但是它们的跨域可转移性的问题仍然是研究不足的。我们在这里研究神经排名模型在获得新知识后,在多大程度上灾难性地忘记了从以前观察到的领域获得的旧知识,导致这些领域的性能下降。我们的实验表明,neuralIR排名模型的有效性是以灾难性遗忘为代价的,并且使用跨域正则化器的终身学习策略成功地缓解了这个问题。使用的解释方法建立在一个回归模型,我们还显示了领域特性的影响,灾难性遗忘的崛起。我们相信,所获得的结果可以在神经IR的理论和实践的未来工作是有用的。
英文摘要:
Several deep neural ranking models have been proposed in the recent IR literature. While their transferability to one target domain held by a dataset has been widely addressed using traditional domain adaptation strategies, the question of their cross-domain transferability is still under-studied. We study here in what extent neural ranking models catastrophically forget old knowledge acquired from previously observed domains after acquiring new knowledge, leading to performance decrease on those domains. Our experiments show that the effectiveness of neuralIR ranking models is achieved at the cost of catastrophic forgetting and that a lifelong learning strategy using a cross-domain regularizer success-fully mitigates the problem. Using an explanatory approach built on a regression model, we also show the effect of domain characteristics on the rise of catastrophic forgetting. We believe that the obtained results can be useful for both theoretical and practical future work in neural IR.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。