CAR: Controllable Autoregressive Modeling for Visual Generation
School of Electronic and Computer Engineering, Peking University(北京大学电子与计算机工程学院);Tencent Youtu Lab(腾讯优图实验室);Southern University of Science and Technology(南方科技大学);University of Washington(华盛顿大学)
Ziyu Yao, Jialin Li, Yifeng Zhou, Yong Liu, Xi Jiang, Chengjie Wang, Feng Zheng, Yuexian Zou, Lei Li
Controllable generation, which enables fine-grained control over generated outputs, has emerged as a critical focus in visual generative models. Currently, there are two primary technical approaches in visual generation: diffusion models and autoregressive models. Diffusion models, as exemplified by ControlNet and T2I-Adapter, offer advanced control mechanisms, whereas autoregressive models, despite showcasing impressive generative quality and scalability, remain underexplored in terms of controllability and flexibility. In this study, we introduce Controllable AutoRegressive Modeling (CAR), a novel, plug-and-play framework that integrates conditional control into multi-scale latent variable modeling, enabling efficient control generation within a pre-trained visual autoregressive model. CAR progressively refines and captures control representations, which are injected into each autoregressive step of the pre-trained model to guide the generation process. Our approach demonstrates excellent controllability across various types of conditions and delivers higher image quality compared to previous methods. Additionally, CAR achieves robust generalization with significantly fewer training resources compared to those required for pre-training the model. To the best of our knowledge, we are the first to propose a control framework for pre-trained autoregressive visual generation models.
DreamForge: Motion-Aware Autoregressive Video Generation for Multi-View Driving Scenes
Zhejiang University(浙江大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);University of Science and Technology of China(中国科学技术大学);Technical University of Munich(慕尼黑技术大学)
C2P-VAR: Continual and Compositional Personalization in Visual Autoregressive Models
Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Peng Cheng Laboratory(鹏城实验室);South China University of Technology(华南理工大学)
Self-control: A Better Conditional Mechanism for Masked Autoregressive Model
Nankai University(南开大学);School of Statistics and Data Science(统计与数据科学学院);Key Laboratory of Pure Mathematics and Combinatorics, Ministry of Education(教育部纯数学与组合数学重点实验室)
3D-WAG: Hierarchical Wavelet-Guided Autoregressive Generation for High-Fidelity 3D Shapes
Chair for Machine Learning University of Mannheim(机器学习主任办公室曼海姆大学);Autodesk AI Lab(Autodesk人工智能实验室);MPI for Informatics Saarland Informatics Campus(信息学研究所萨尔兰州信息学校区)
MosaicMem: Hybrid Spatial Memory for Controllable Video World Models
University of Toronto(多伦多大学);Vector Institute(向量研究所);The University of Osaka(大阪大学);Georgia Institute of Technology(佐治亚理工学院);Mujin Inc.(Mujin公司);University of Texas at Austin(德克萨斯大学奥斯汀分校);Taiyuan University of Technology(太原本科技大学)
Controllable Skin Synthesis via Lesion-Focused Vector Autoregression Model
School of Translational Medicine, Faculty of Medicine, Nursing and Health Sciences, Monash University(转化医学学院,医学、护理与健康科学学院,墨尔本大学);Melbourne Sexual Health Centre, Alfred Health(墨尔本性健康中心,阿尔弗雷德健康中心);AIM for Health Lab, Monash University(健康促进实验室,墨尔本大学);Faculty of IT, Monash University(信息技术学院,墨尔本大学);Faculty of Engineering, Monash University(工程学院,墨尔本大学);Faculty of Infectious and Tropical Diseases, London School of Hygiene and Tropical Medicine(传染病与热带医学学院,伦敦热带医学学院)
本文提出 LF-VAR 模型,通过病灶测量评分与类型标签引导,结合多尺度 VQVAE 和 VAR Transformer,实现基于语言提示的可控高保真皮肤图像合成,FID 平均达 0.74,较 SOTA 提升 6.3%。
Shanghai AI Laboratory(上海人工智能实验室);Shanghai Innovation Institute(上海创新研究院);Nanjing University(南京大学);The Chinese University of Hong Kong(香港中文大学);Shanghai Jiao Tong University(上海交通大学);Zhejiang University of Technology(浙江工业大学)
MegaParts: Scaling Part-Aware 3D Object Generation to 300 Parts via Token-Efficient Autoregressive Modeling
The University of Hong Kong(香港大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Fudan University(复旦大学);Tongji University(同济大学);University of Science and Technology of China(中国科学技术大学);Shanghai Jiao Tong University(上海交通大学)
EarthMapper: Visual Autoregressive Models for Controllable Bidirectional Satellite-Map Translation
School of Electronics and Information Engineering, Harbin Institute of Technology(电子与信息工程学院,哈尔滨工业大学);Faculty of Computing, Harbin Institute of Technology(计算机学院,哈尔滨工业大学);Peng Cheng Lab(鹏城实验室)