E-KAR：一个实现自然语言类比推理的基准
E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoning
摘要：
识别类比的能力是人类认知的基础。现有的测试词类比的基准并没有揭示神经模型类比推理的底层过程。我们认为，能够推理的模型应该是正确的，因为正确的原因，我们提出了一个第一个同类可解释的知识密集型类比推理基准（E-KAR）。我们的基准包括1，655道（中文）和1，251道（英文）来自公务员考试的问题，这些问题需要深入的背景知识来解决。更重要的是，我们设计了一个自由文本解释方案来解释是否应该进行类比，并为每个问题和候选答案手动注释它们。实证结果表明，这一基准是非常具有挑战性的一些国家的最先进的模型，解释生成和类比问答任务，邀请在这一领域的进一步研究。
英文摘要：
The ability to recognize analogies is fundamental to human cognition. Existing benchmarks to test word analogy do not reveal the underneath process of analogical reasoning of neural models. Holding the belief that models capable of reasoning should be right for the right reasons, we propose a first-of-its-kind Explainable Knowledge-intensive Analogical Reasoning benchmark (E-KAR). Our benchmark consists of 1,655 (in Chinese) and 1,251 (in English) problems sourced from the Civil Service Exams, which require intensive background knowledge to solve. More importantly, we design a free-text explanation scheme to explain whether an analogy should be drawn, and manually annotate them for each and every question and candidate answer. Empirical results suggest that this benchmark is very challenging for some state-of-the-art models for both explanation generation and analogical question answering tasks, which invites further research in this area.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
