学习排序
Learning to Rank for Plausible Plausibility
摘要：
研究人员通过一组共享的自然语言理解（NLU）任务的结果表现来说明上下文编码策略的改进。这些任务中的许多都是分类预测类型：给定条件上下文（例如，NLI前提），基于相关联的提示提供标签（例如，NLI假说）。这些任务的分类性质导致在训练期间通常使用交叉熵对数损失目标。我们认为，这种损失是直观错误的，当应用到可扩展性任务，其中提示的设计既不是绝对必然的，也不矛盾的背景。对数损失自然会促使模型分配接近0.0或1.0的分数，这与我们建议使用的基于边际的损失相反。在讨论了我们的直觉之后，我们描述了一项基于MultiNLI的极端综合策划任务的确认研究。我们发现，基于保证金的损失导致一个更合理的模型的可扩展性。最后，我们通过这种损失的变化说明了合理选择（COPA）任务的改进。
英文摘要：
Researchers illustrate improvements in contextual encoding strategies via resultant performance on a battery of shared Natural Language Understanding (NLU) tasks. Many of these tasks are of a categorical prediction variety: given a conditioning context (e.g., an NLI premise), provide a label based on an associated prompt (e.g., an NLI hypothesis). The categorical nature of these tasks has led to common use of a cross entropy log-loss objective during training. We suggest this loss is intuitively wrong when applied to plausibility tasks, where the prompt by design is neither categorically entailed nor contradictory given the context. Log-loss naturally drives models to assign scores near 0.0 or 1.0, in contrast to our proposed use of a margin-based loss. Following a discussion of our intuition, we describe a confirmation study based on an extreme, synthetically curated task derived from MultiNLI. We find that a margin-based loss leads to a more plausible model of plausibility. Finally, we illustrate improvements on the Choice Of Plausible Alternative (COPA) task through this change in loss.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
