Recent progress in large models has led to significant advances in unified multimodal generation and understanding. However, the development of models that unify motion-language generation and understanding remains largely underexplored. Existing approaches often fine-tune large language models (LLMs) on paired motion-text data, which can result in catastrophic forgetting of linguistic capabilities due to the limited scale of available text-motion pairs. Furthermore, prior methods typically convert motion into discrete representations via quantization to integrate with language models, introducing substantial jitter artifacts from discrete tokenization. To address these challenges, we propose LLaMo, a unified framework that extends pretrained LLMs through a modality-specific Mixture-of-Transformers (MoT) architecture. This design inherently preserves the language understanding of the base model while enabling scalable multimodal adaptation. We encode human motion into a causal continuous latent space and maintain the next-token prediction paradigm in the decoder-only backbone through a lightweight flow-matching head, allowing for streaming motion generation in real-time (>30 FPS). Leveraging the comprehensive language understanding of pretrained LLMs and large-scale motion-text pretraining, our experiments demonstrate that LLaMo achieves high-fidelity text-to-motion generation and motion-to-text captioning in general settings, especially zero-shot motion generation, marking a significant step towards a general unified motion-language large model.
Physical Autoregressive Model for Robotic Manipulation without Action Pretraining
Sun Yat-sen University(中山大学);Guangdong Key Laboratory of Big Data Analysis and Processing(广东大数据分析与处理重点实验室);X-Era AI Lab(X-Era人工智能实验室);Guangdong University of Technology(广东工业大学)
FreeLoRA: Enabling Training-Free LoRA Fusion for Autoregressive Multi-Subject Personalization
School of Artificial Intelligence, Jilin University(吉林大学人工智能学院);Shanghai Innovation Institute(上海创新研究院);School of Computer Science, Fudan University(复旦大学计算机学院)
Edit-VAR: Taming Visual Autoregressive Model for Precise Video Editing
Sun Yat-sen University(中山大学);South China University of Technology(华南理工大学);Tsinghua University(清华大学);Shandong University(山东大学);Nankai University(南开大学);Hunan University(湖南大学)
Shanghai AI Laboratory(上海人工智能实验室);Shanghai Innovation Institute(上海创新研究院);Nanjing University(南京大学);The Chinese University of Hong Kong(香港中文大学);Shanghai Jiao Tong University(上海交通大学);Zhejiang University of Technology(浙江工业大学)
Streaming Autoregressive Video Generation via Diagonal Distillation
South China University of Technology(南方科技大学);Westlake University(西湖大学);Johns Hopkins University(约翰霍普金斯大学);University of California, Merced(加州大学默塞德分校);The Chinese University of Hong Kong(香港中文大学)
WorldCraft: From Camera Navigation to Object Manipulation in Interactive Video World Models
The Hong Kong University of Science and Technology(香港科学与技术大学);AI Technology Center, Tencent Video, Tencent(腾讯视频AI技术中心,腾讯);Wuhan University(武汉大学);Peking University(北京大学)
Glance Forcing: Making Bidirectional Diffusion Autoregressive at a Glance
Wuhan University(武汉大学);University of Electronic Science and Technology of China(电子科技大学);Sun Yat-sen University(中山大学);University of Washington(华盛顿大学);National University of Singapore(新加坡国立大学);Central South University(中南大学)
GALIP: Generative Adversarial CLIPs for Text-to-Image Synthesis
Nanjing University of Posts and Telecommunications(南京邮电大学);Peng Cheng Laboratory(鹏城实验室);ETH Zürich(苏黎世联邦理工学院);University of Chinese Academy of Sciences(中国科学院大学);Institute of Automation, CAS(中国科学院自动化研究所)
ActVAR: Activating Mixtures of Weights and Tokens for Efficient Visual Autoregressive Generation
School of Computer Science and Engineering, Central South University(中南大学计算机科学与工程学院);University of Electronic Science and Technology of China(电子科技大学);School of Computer Science, Peking University(北京大学计算机科学学院);SenseTime Research(商汤科技研究院);Shanghai Jiao Tong University(上海交通大学)
Spatiotemporal Satellite Image Downscaling with Transfer Encoders and Autoregressive Generative Models
University of Toronto(多伦多大学);Harvard T.H. Chan School of Public Health(哈佛大学T.H. Chan公共卫生学院);Harvard Medical School(哈佛医学院);VA Healthcare System Boston, U.S. Department of Veterans Affairs(美国退伍军人事务部波士顿医疗系统)
CheXanatomy: Anatomy-Aware Vision-Language Modeling for Chest Radiographs
Stanford Center for Artificial Intelligence in Medicine and Imaging, Stanford University(斯坦福大学医学与影像人工智能中心);Department of Radiology, Stanford University(斯坦福大学放射学系)
OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models
State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA School of Artificial Intelligence, University of Chinese Academy of Sciences AutoLab, School of Artificial Intelligence, Shanghai Jiao Tong University Voyager Research, Didi Chuxing School of Information Science(多模态人工智能系统国家重点实验室,中国科学院自动化所人工智能学院,中国科学院大学,AutoLab,上海交通大学人工智能学院,Voyager Research,滴滴出行,信息科学与技术学院)