We introduce LlamaGen, a new family of image generation models that apply original ``next-token prediction'' paradigm of large language models to visual generation domain. It is an affirmative answer to whether vanilla autoregressive models, e.g., Llama, without inductive biases on visual signals can achieve state-of-the-art image generation performance if scaling properly. We reexamine design spaces of image tokenizers, scalability properties of image generation models, and their training data quality. The outcome of this exploration consists of: (1) An image tokenizer with downsample ratio of 16, reconstruction quality of 0.94 rFID and codebook usage of 97% on ImageNet benchmark. (2) A series of class-conditional image generation models ranging from 111M to 3.1B parameters, achieving 2.18 FID on ImageNet 256x256 benchmarks, outperforming the popular diffusion models such as LDM, DiT. (3) A text-conditional image generation model with 775M parameters, from two-stage training on LAION-COCO and high aesthetics quality images, demonstrating competitive performance of visual quality and text alignment. (4) We verify the effectiveness of LLM serving frameworks in optimizing the inference speed of image generation models and achieve 326% - 414% speedup. We release all models and codes to facilitate open-source community of visual generation and multimodal foundation models.
LlamaSeg: Image Segmentation via Autoregressive Mask Generation
Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)(广东人工智能与数字经济实验室);Tsinghua University(清华大学);Shenzhen University(深圳大学);Meituan(美团);Division of AMC and Department of ECE, HKUST(HKUST AMC 分部和电子工程系)
VCE: Safe Autoregressive Image Generation via Visual Contrast Exploitation
Fudan University(复旦大学);Shanghai Key Lab of Intell. Info. Processing, School of CS, Fudan University(复旦大学计算机科学技术学院上海市智能信息处理重点实验室);Shanghai Collaborative Innovation Center on Intelligent Visual Computing(上海智能视觉计算协同创新中心);International Computer Science Institute(国际计算机科学研究所);UC Berkeley(加州大学伯克利分校)
Resurrect Mask AutoRegressive Modeling for Efficient and Scalable Image Generation
Nanjing University(南京大学);Shanghai AI Laboratory(上海人工智能实验室);Shanghai Innovation Institute(上海创新研究院);The Chinese University of Hong Kong(香港中文大学);Shanghai Jiao Tong University(上海交通大学);University of Minnesota Twin Cities(明尼苏达大学 Twin Cities 分校)
Exploiting Discriminative Codebook Prior for Autoregressive Image Generation
Department of Computer Science and Engineering, The Hong Kong University of Science and Technology(计算机科学与工程系,香港科技大学);Tongyi Lab, Alibaba Group(阿里云实验室);Department of Biomedical Engineering, Duke University(生物医学工程系,杜克大学)
Not Like Transformers: Drop the Beat Representation for Dance Generation with Mamba-Based Diffusion Model
Ulsan National Institute of Science and Technology, South Korea(乌山国立科学技术研究院,韩国);Daegu Gyeongbuk Institute of Science and Technology, South Korea(大邱庆北科学技术研究院,韩国)
Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Models
Northeastern University(东北大学);Meta GenAI(Meta 生成人工智能);Meta FAIR;National University of Singapore(国立新加坡大学);The Chinese University of Hong Kong(香港中文大学);University of Washington(华盛顿大学)
LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuning
Gaoling School of AI, Renmin University of China(中国人民大学人工智能学院);Beijing Key Laboratory of Research on Large Models and Intelligent Governance(北京大型模型与智能治理研究重点实验室);Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE(下一代智能搜索与推荐工程技术研究中心,教育部);Ant Group(蚂蚁集团)