Recent studies show that text-to-image models often fail to generate geographically representative images, raising concerns about the representativeness of their training data and motivating the question: which parts of the world do these training examples come from? We geographically profile large-scale multimodal datasets by mapping image-caption pairs to countries based on location information extracted from captions using LLMs. Studying English captions from three widely used datasets (Re-LAION, DataComp1B, and Conceptual Captions) across $20$ common entities (e.g., house, flag), we find that the United States, the United Kingdom, and Canada account for $48.0\%$ of samples, while South American and African countries are severely under-represented with only $1.8\%$ and $3.8\%$ of images, respectively. We observe a strong correlation between a country's GDP and its representation in the data ($ρ= 0.82$). Examining non-English subsets for $4$ languages from the Re-LAION dataset, we find that representation skews heavily toward countries where these languages are predominantly spoken. Additionally, we find that higher representation does not necessarily translate to greater visual or semantic diversity. Finally, analyzing country-specific images generated by Stable Diffusion v1.3 trained on Re-LAION, we show that while generations appear realistic, they are severely limited in their coverage compared to real-world images.
DIMCIM: A Quantitative Evaluation Framework for Default-mode Diversity and Generalization in Text-to-Image Generative Models
FAIR at Meta - New York and Montreal labs(Meta 新 York 和 Montreal 实验室);Courant Institute for Mathematical Sciences, NYU(纽约大学应用数学科学研究所);Mila - Quebec AI Institute(魁北克人工智能研究所);McGill University(麦吉尔大学);Canada CIFAR AI chair(加拿大 CIFAR 人工智能主席)
BigEarthNet.txt: A Large-Scale Multi-Sensor Image-Text Dataset and Benchmark for Earth Observation
BIFOLD(柏林智能数据与机器学习研究所);Technische Universität Berlin(柏林工业大学);University of Trento(特伦托大学);National Technical University(雅典国家技术大学);National Observatory of Athens(雅典国家天文台)
Adjudicated Captioning: Multi-Agent Alignment Scoring and Consensus-Distilled Beam Arbitration for Strict Zero-Shot Image Captioning
AI Platform OneNexus, OneMount(OneMount AI平台OneNexus);School of Electronic Engineering, Soongsil University(崇实大学电子工程学院);MoAdata(MoAdata公司);University of Economics Ho Chi Minh City (UEH)(胡志明市经济大学)