arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2103.11474cs.CVcs.CL

#PraCegoVer:一种用于葡萄牙语图像描述的大型数据集

#PraCegoVer: A Large Dataset for Image Captioning in Portuguese

Institute of Computing, University of Campinas(计算机学院,坎皮纳斯大学)

Gabriel Oliveira dos Santos, Esther Luna Colombini, Sandra Avila

AI总结:

本文提出#PraCegoVer数据集,基于Instagram帖子生成葡萄牙语图像描述,是首个大规模葡萄牙语图像描述数据集,具有单张图像仅一个描述及较长句子长度的挑战性特征。

摘要:

自动用自然句子描述图像对于支持视障人士上网具有重要意义。尽管已有许多数据集,但大多数仅包含英语描述,而其他语言描述的数据集稀缺。最近,网络上出现了一种名为PraCegoVer的运动,鼓励用户发布图像、添加#PraCegoVer标签并添加简短内容描述。受此启发,我们提出了#PraCegoVer数据集,基于Instagram帖子,包含葡萄牙语描述。这是首个大规模葡萄牙语图像描述数据集,具有自由标注的图像。此外,数据集中的描述带来了额外挑战:与MS COCO Captions等流行数据集相比,#PraCegoVer每张图像仅有一个参考描述,且参考句子的均值和方差显著更大。这些特性使数据集在语言方面和图像描述问题上具有吸引力。数据集已公开共享在https://github.com/gabrielsantosrv/PraCegoVer。

英文摘要:

Automatically describing images using natural sentences is an important task to support visually impaired people's inclusion onto the Internet. It is still a big challenge that requires understanding the relation of the objects present in the image and their attributes and actions they are involved in. Then, visual interpretation methods are needed, but linguistic models are also necessary to verbally describe the semantic relations. This problem is known as Image Captioning. Although many datasets were proposed in the literature, the majority contains only English captions, whereas datasets with captions described in other languages are scarce. Recently, a movement called PraCegoVer arose on the Internet, stimulating users from social media to publish images, tag #PraCegoVer and add a short description of their content. Thus, inspired by this movement, we have proposed the #PraCegoVer, a multi-modal dataset with Portuguese captions based on posts from Instagram. It is the first large dataset for image captioning in Portuguese with freely annotated images. Further, the captions in our dataset bring additional challenges to the problem: first, in contrast to popular datasets such as MS COCO Captions, #PraCegoVer has only one reference to each image; also, both mean and variance of our reference sentence length are significantly greater than those in the MS COCO Captions. These two characteristics contribute to making our dataset interesting due to the linguistic aspect and the challenges that it introduces to the image captioning problem. We publicly-share the dataset at https://github.com/gabrielsantosrv/PraCegoVer.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑