arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2210.08402cs.CVcs.AIcs.LG
收藏

LAION-5B:用于训练下一代图像-文本模型的开放式大规模数据集

LAION-5B: An open large-scale dataset for training next generation image-text models

LAION;UC Berkeley(加州大学伯克利分校);Gentec Data;TU Darmstadt(达姆施塔特工业大学);University of Washington(华盛顿大学);Technical University of Munich(慕尼黑工业大学);Stability AI;EleutherAI;Juelich Supercomputing Center (JSC)(于利希超级计算中心);Research Center Juelich (FZJ)(于利希研究中心)
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Pa… 展开作者

Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, Jenia Jitsev

摘要:

CLIP和DALL-E等开创性的语言视觉架构证明了在大量嘈杂的图像-文本数据上进行训练的实用性,而不依赖于标准视觉单峰监督学习中使用的昂贵的准确标签。由此产生的模型显示出强大的文本引导图像生成和传输到下游任务的能力,同时在zero-shot分类中表现出色,具有显著的分布鲁棒性。从那时起,ALIGN、BASIC、GLIDE、Flamingo和Imagen等大规模语言视觉模型进行了进一步的改进。研究这些模型的训练和能力需要包含数十亿图像-文本对的数据集。到目前为止,还没有这种规模的数据集公开提供给更广泛的研究社区。为了解决这个问题并使大规模多模态模型的研究民主化,我们提出了LAION-5 B-一个由58.5亿个CLIP过滤的图像-文本对组成的数据集,其中2.32B包含英语。我们展示了使用数据集成功复制和微调CLIP,GLIDE和稳定扩散等基础模型,并讨论了使用这种规模的公开数据集进行的进一步实验。此外,我们还提供了几个最近邻索引,一个改进的Web界面,用于数据集探索和子集生成,以及水印,NSFW和有毒内容检测的检测分数。公告页面https://laion.ai/laion-5b-a-new-era-of-open-large-scale-multi-modal-datasets/

英文摘要:

Groundbreaking language-vision architectures like CLIP and DALL-E proved the utility of training on large amounts of noisy image-text data, without relying on expensive accurate labels used in standard vision unimodal supervised learning. The resulting models showed capabilities of strong text-guided image generation and transfer to downstream tasks, while performing remarkably at zero-shot classification with noteworthy out-of-distribution robustness. Since then, large-scale language-vision models like ALIGN, BASIC, GLIDE, Flamingo and Imagen made further improvements. Studying the training and capabilities of such models requires datasets containing billions of image-text pairs. Until now, no datasets of this size have been made openly available for the broader research community. To address this problem and democratize research on large-scale multi-modal models, we present LAION-5B - a dataset consisting of 5.85 billion CLIP-filtered image-text pairs, of which 2.32B contain English language. We show successful replication and fine-tuning of foundational models like CLIP, GLIDE and Stable Diffusion using the dataset, and discuss further experiments enabled with an openly available dataset of this scale. Additionally we provide several nearest neighbor indices, an improved web-interface for dataset exploration and subset generation, and detection scores for watermark, NSFW, and toxic content detection. Announcement page https://laion.ai/laion-5b-a-new-era-of-open-large-scale-multi-modal-datasets/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑