arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.09775cs.CV
收藏

图像来自哪里?通过分析描述词地理上剖析数据集

Where Do Images Come From? Analyzing Captions to Geographically Profile Datasets

Indian Institute of Science, Bangalore(印度科学研究院,班加罗尔);TNSQ AI(TNSQ人工智能);University of California, Irvine(加州大学尔湾分校)

Abhipsa Basu, Yugam Bahl, Kirti Bhagat, Preethi Seshadri, R. Venkatesh Babu, Danish Pruthi

AI总结:

研究通过分析图像描述词地理分布,发现训练数据严重偏向欧美国家,且代表性与GDP正相关,但生成图像的多样性不足。

摘要:

最近的研究表明,文本到图像模型往往无法生成具有地理代表性的图像,这引发了关于其训练数据代表性的担忧,并提出了问题:这些训练示例来自世界哪些地区?我们通过将图像-描述词对映射到国家来地理上剖析大规模多模态数据集,基于从描述词中提取的位置信息使用LLMs进行处理。研究英语描述词来自三个广泛使用的数据集(Re-LAION、DataComp1B和Conceptual Captions)的20个常见实体(例如房屋、旗帜),我们发现美国、英国和加拿大占样本的48.0%,而南美和非洲国家严重代表性不足,仅占1.8%和3.8%的图像。我们观察到一个国家的GDP与其在数据中的代表性之间存在强相关性(ρ=0.82)。检查Re-LAION数据集的4种非英语子集,我们发现代表性严重偏向于这些语言主要 spoken 的国家。此外,我们发现更高的代表性并不一定意味着更大的视觉或语义多样性。最后,分析由Stable Diffusion v1.3训练于Re-LAION的数据集生成的国家特定图像,我们显示尽管生成的图像看起来真实,但其覆盖范围相比真实世界图像严重受限。

英文摘要:

Recent studies show that text-to-image models often fail to generate geographically representative images, raising concerns about the representativeness of their training data and motivating the question: which parts of the world do these training examples come from? We geographically profile large-scale multimodal datasets by mapping image-caption pairs to countries based on location information extracted from captions using LLMs. Studying English captions from three widely used datasets (Re-LAION, DataComp1B, and Conceptual Captions) across $20$ common entities (e.g., house, flag), we find that the United States, the United Kingdom, and Canada account for $48.0\%$ of samples, while South American and African countries are severely under-represented with only $1.8\%$ and $3.8\%$ of images, respectively. We observe a strong correlation between a country's GDP and its representation in the data ($ρ= 0.82$). Examining non-English subsets for $4$ languages from the Re-LAION dataset, we find that representation skews heavily toward countries where these languages are predominantly spoken. Additionally, we find that higher representation does not necessarily translate to greater visual or semantic diversity. Finally, analyzing country-specific images generated by Stable Diffusion v1.3 trained on Re-LAION, we show that while generations appear realistic, they are severely limited in their coverage compared to real-world images.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑