HL数据集：场景、动作与理由的视觉基础描述
HL Dataset: Visually-grounded Description of Scenes, Actions and Rationales
AI总结：
本文提出HL Dataset，扩展COCO图像并标注场景、动作和理由三类高层次描述，同时提供置信度分数和合成叙事描述，并给出高层次图像描述任务的基线结果。
摘要：
当前的图像描述数据集主要关注以物体为中心的描述，描述图像中可见的物体，例如“人们在公园里吃东西”。尽管这些数据集有助于评估视觉与语言模型识别和描述视觉内容的能力，但它们无法支持涉及模型测试或微调的受控实验，以生成人类觉得容易且自然的高层次描述。例如，人们常常根据图像所描绘的场景类型（“人们在度假胜地”）以及他们正在执行的动作（“人们在野餐”）来描述图像。这类描述依赖于个人经验和常识性假设。我们提出了High-Level Dataset（HL Dataset），该数据集扩展了COCO数据集中的14997张图像，并配有一组新的134,973条人工标注的（高层次）描述，这些描述沿三个轴收集：场景、动作和理由。我们进一步用从一组独立读者收集的置信度分数扩展了该数据集，以及通过组合三个轴中的每一个轴合成生成的一组叙事性描述。我们对该数据集进行了描述并进行了广泛分析。我们还给出了High-Level Captioning任务的基线结果。
英文摘要：
Current captioning datasets focus on object-centric captions, describing the visible objects in the image, e.g. "people eating food in a park". Although these datasets are useful to evaluate the ability of Vision & Language models to recognize and describe visual content, they do not support controlled experiments involving model testing or fine-tuning, with more high-level captions, which humans find easy and natural to produce. For example, people often describe images based on the type of scene they depict ('people at a holiday resort') and the actions they perform ('people having a picnic'). Such descriptions draw on personal experience and commonsense assumptions. We present the High-Level Dataset a dataset extending 14997 images from the COCO dataset, aligned with a new set of 134,973 human-annotated (high-level) captions collected along three axes: scenes, actions, and rationales. We further extend this dataset with confidence scores collected from an independent set of readers, as well as a set of narrative captions generated synthetically, by combining each of the three axes. We describe this dataset and analyse it extensively. We also present baseline results for the High-Level Captioning task.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
