Pento-DIARef:用于从示例中学习指称表达式生成增量算法的诊断数据集
Pento-DIARef: A Diagnostic Dataset for Learning the Incremental Algorithm for Referring Expression Generation from Examples
AI总结:
本文提出诊断数据集Pento-DIARef,用于探究神经模型能否从外延性数据示例中学习指称表达式生成的增量算法规律,实验表明结合视觉检测与定向数据生成的模型能近乎完美地完成任务。
摘要:
NLP任务通常外延性地通过包含示例实例化的数据集(例如,图像i和文本t的配对)来定义,但内涵性地通过任务口头描述中调用的能力(例如,“t是i的描述,为此需要识别和理解i的内容”)来获得动机。我们提出了Pento-DIARef,这是一个位于拼图块视觉领域的诊断数据集,其中的指称表达式由一个著名的符号算法(即“增量算法”)生成,该算法本身通过诉诸一种假设的能力(通过应用Gricean准则来消除干扰项)来获得动机。我们的问题随后是:在给定从视觉输入生成表达式的简单任务定义下,这种外延性描述(即数据集)是否足以让神经模型掌握潜在的规律性并展现出这一能力。我们发现,由视觉检测步骤和定向数据生成方案支持的模型实现了几乎完美的BLEU@1分数和句子准确率,而较简单的基线则未能做到。
英文摘要:
NLP tasks are typically defined extensionally through datasets containing example instantiations (e.g., pairs of image i and text t), but motivated intensionally through capabilities invoked in verbal descriptions of the task (e.g., "t is a description of i, for which the content of i needs to be recognised and understood"). We present Pento-DIARef, a diagnostic dataset in a visual domain of puzzle pieces where referring expressions are generated by a well-known symbolic algorithm (the "Incremental Algorithm"), which itself is motivated by appeal to a hypothesised capability (eliminating distractors through application of Gricean maxims). Our question then is whether the extensional description (the dataset) is sufficient for a neural model to pick up the underlying regularity and exhibit this capability given the simple task definition of producing expressions from visual inputs. We find that a model supported by a vision detection step and a targeted data generation scheme achieves an almost perfect BLEU@1 score and sentence accuracy, whereas simpler baselines do not.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。