LAION-5B: An open large-scale dataset for training next generation image-text models
LAION;UC Berkeley(加州大学伯克利分校);Gentec Data;TU Darmstadt(达姆施塔特工业大学);University of Washington(华盛顿大学);Technical University of Munich(慕尼黑工业大学);Stability AI;EleutherAI;Juelich Supercomputing Center (JSC)(于利希超级计算中心);Research Center Juelich (FZJ)(于利希研究中心)
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Pa… 展开作者
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, Jenia Jitsev
Groundbreaking language-vision architectures like CLIP and DALL-E proved the utility of training on large amounts of noisy image-text data, without relying on expensive accurate labels used in standard vision unimodal supervised learning. The resulting models showed capabilities of strong text-guided image generation and transfer to downstream tasks, while performing remarkably at zero-shot classification with noteworthy out-of-distribution robustness. Since then, large-scale language-vision models like ALIGN, BASIC, GLIDE, Flamingo and Imagen made further improvements. Studying the training and capabilities of such models requires datasets containing billions of image-text pairs. Until now, no datasets of this size have been made openly available for the broader research community. To address this problem and democratize research on large-scale multi-modal models, we present LAION-5B - a dataset consisting of 5.85 billion CLIP-filtered image-text pairs, of which 2.32B contain English language. We show successful replication and fine-tuning of foundational models like CLIP, GLIDE and Stable Diffusion using the dataset, and discuss further experiments enabled with an openly available dataset of this scale. Additionally we provide several nearest neighbor indices, an improved web-interface for dataset exploration and subset generation, and detection scores for watermark, NSFW, and toxic content detection. Announcement page https://laion.ai/laion-5b-a-new-era-of-open-large-scale-multi-modal-datasets/
LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs
LAION;Gentec Data;Technical University of Munich(慕尼黑工业大学);Juelich Supercomputing Center (JSC)(于利希超级计算中心);Research Center Juelich (FZJ)(于利希研究中心);Georgia Institute of Technology(佐治亚理工学院);EleutherAI
ESPLoRA: Enhanced Spatial Precision with Low-Rank Adaption in Text-to-Image Diffusion Models for High-Definition Synthesis
DISI, University of Trento(DISI,特伦托大学);Toretei S.r.l.(Toretei公司);Visual AI Lab, MIT-IBM Watson AI Lab(视觉人工智能实验室,MIT-IBM Watson人工智能实验室);Fondazione Bruno Kessler(布鲁诺·凯撒基金会)
FaceID-6M: A Large-Scale, Open-Source FaceID Customization Dataset
The University of Melbourne(墨尔本大学);University of Washington(华盛顿大学);Zhejiang University(浙江大学);Alibaba(阿里巴巴);Tsinghua University(清华大学);WeChat AI(微信AI);Tencent(腾讯);Nanyang Technological University(南洋理工大学)
Reproducible scaling laws for contrastive language-image learning
LAION;UC Berkeley(加州大学伯克利分校);HuggingFace;University of Washington(华盛顿大学);Juelich Supercomputing Center (JSC)(于利希超级计算中心);Research Center Juelich (FZJ)(于利希研究中心)
EDITOR: Effective and Interpretable Prompt Inversion for Text-to-Image Diffusion Models
University of Massachusetts, Amherst(马萨诸塞大学阿姆赫斯特分校);Georgia Institute of Technology(佐治亚理工学院);Rutgers University(罗格斯大学);University of Utah(犹他大学);Dolby Laboratories(杜比实验室)
Vision Model Pre-training on Interleaved Image-Text Data via Latent Compression Learning
Tsinghua University(清华大学);OpenGVLab(开放视觉实验室);Shanghai AI Laboratory(上海人工智能实验室);SenseTime Research(商汤科技研究院);The Chinese University of Hong Kong(香港中文大学);University of Science and Technology of China(中国科学技术大学);Xi’an Jiaotong University(西安交通大学);Beijing University of Posts and Telecommunications(北京邮电大学)