目标重要吗?代词解析训练目标比较
Does the Objective Matter? Comparing Training Objectives for Pronoun Resolution
摘要:
代词解析的困难情况一直被用作常识推理的长期基准。在最近的文献中,预训练的语言模型已被用于获得最先进的结果代词解析。总的来说,已经提出了四类培训和评价目标。这些工作中使用的训练数据集和预训练语言模型的多样性使得训练目标的选择是否至关重要变得不清楚。在这项工作中,我们做了一个公平的比较的性能和种子明智的稳定性的四个模型,代表四类目标。实验结果表明,序列排序的目标在域内表现最好,而候选词与代词之间的语义相似度的目标在域外表现最好。我们还观察到一个种子明智的不稳定性的模型使用序列排名,这是不是这样的情况下,使用其他目标。
英文摘要:
Hard cases of pronoun resolution have been used as a long-standing benchmark for commonsense reasoning. In the recent literature, pre-trained language models have been used to obtain state-of-the-art results on pronoun resolution. Overall, four categories of training and evaluation objectives have been introduced. The variety of training datasets and pre-trained language models used in these works makes it unclear whether the choice of training objective is critical. In this work, we make a fair comparison of the performance and seed-wise stability of four models that represent the four categories of objectives. Our experiments show that the objective of sequence ranking performs the best in-domain, while the objective of semantic similarity between candidates and pronoun performs the best out-of-domain. We also observe a seed-wise instability of the model using sequence ranking, which is not the case when the other objectives are used.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。