DiNeR:一个用于评估组合泛化的大规模真实数据集
DiNeR: a Large Realistic Dataset for Evaluating Compositional Generalization
AI总结:
本文提出菜名识别任务DiNeR,构建包含3,811道菜和228,114条食谱的大规模真实中文数据集,以更丰富的语言现象和组合多样性评估组合泛化,并提供基于T5和大语言模型的基线方法。
摘要:
现有的大多数组合泛化数据集都是合成生成的,导致缺乏自然语言的变化。尽管最近有人尝试引入非合成数据集用于组合泛化,但它们要么数据规模有限,要么组合形式缺乏多样性。为了更好地研究具有更多语言现象和组合多样性的组合泛化问题,我们提出了菜名识别(DiNeR)任务,并创建了一个大规模真实中文数据集。给定一条食谱指令,模型需要识别由食物、动作和风味等多种组合构成的菜名。我们的数据集包含3,811道菜和228,114条食谱,涉及大量语言现象,如回指、省略和歧义。我们提供了两个基于T5和大语言模型(LLMs)的强基线。这项工作贡献了一个具有挑战性的任务、解决该任务的基线方法,以及对菜名识别背景下组合泛化的见解。代码和数据可在 https://github.com/Jumpy-pku/DiNeR 获取。
英文摘要:
Most of the existing compositional generalization datasets are synthetically-generated, resulting in a lack of natural language variation. While there have been recent attempts to introduce non-synthetic datasets for compositional generalization, they suffer from either limited data scale or a lack of diversity in the forms of combinations. To better investigate compositional generalization with more linguistic phenomena and compositional diversity, we propose the DIsh NamE Recognition (DiNeR) task and create a large realistic Chinese dataset. Given a recipe instruction, models are required to recognize the dish name composed of diverse combinations of food, actions, and flavors. Our dataset consists of 3,811 dishes and 228,114 recipes, and involves plenty of linguistic phenomena such as anaphora, omission and ambiguity. We provide two strong baselines based on T5 and large language models (LLMs). This work contributes a challenging task, baseline methods to tackle the task, and insights into compositional generalization in the context of dish name recognition. Code and data are available at https://github.com/Jumpy-pku/DiNeR.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。