Text-driven human motion generation based on diffusion strategies establishes a reliable foundation for multimodal applications in human-computer interactions. However, existing advances face significant efficiency challenges due to the substantial computational overhead of iteratively solving for nonlinear reverse diffusion trajectories during the inference phase. To this end, we propose the motion latent consistency training framework (MLCT), which precomputes reverse diffusion trajectories from raw data in the training phase and enables few-step or single-step inference via self-consistency constraints in the inference phase. Specifically, a motion autoencoder with quantization constraints is first proposed for constructing concise and bounded solution distributions for motion diffusion processes. Subsequently, a classifier-free guidance format is constructed via an additional unconditional loss function to accomplish the precomputation of conditional diffusion trajectories in the training phase. Finally, a clustering guidance module based on the K-nearest-neighbor algorithm is developed for the chain-conduction optimization mechanism of self-consistency constraints, which provides additional references of solution distributions at a small query cost. By combining these enhancements, we achieve stable and consistency training in non-pixel modality and latent representation spaces. Benchmark experiments demonstrate that our method significantly outperforms traditional consistency distillation methods with reduced training cost and enhances the consistency model to perform comparably to state-of-the-art models with lower inference costs.
TLCM: Training-efficient Latent Consistency Model for Image Generation with 2-8 Steps
AI Center, Guangdong OPPO Mobile Telecommunications Corp., Ltd(广东欧珀移动通信有限公司AI中心);Qing Yuan Research Institute, SEIEE, Shanghai Jiao Tong University(上海交通大学电子信息与电气工程学院清源研究院)
Multi-scale Latent Point Consistency Models for 3D Shape Generation
Wangxuan Institute of Computer Technology, Peking University(北京大学王轩计算机技术研究所);Department of Electrical and Computer Engineering, University of British Columbia(不列颠哥伦比亚大学电气与计算机工程系)
Trajectory Consistency Distillation: Improved Latent Consistency Distillation by Semi-Linear Consistency Function with Trajectory Mapping
South China University of Technology(华南理工大学);Nanyang Technological University(南洋理工大学);Beijing Institute of Technology(北京理工大学);The University of Sydney(悉尼大学)
LDC-VAE: A Latent Distribution Consistency Approach to Variational AutoEncoders
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
LatentEditor: Text Driven Local Editing of 3D Scenes
Center for Research in Computer Vision, University of Central Florida(中佛罗里达大学计算机视觉研究中心);Department of Computer Science, Wayne State University(韦恩州立大学计算机科学系)
FlowCoMotion: Text-to-Motion Generation via Token-Latent Flow Modeling
School of Artificial Intelligence and Data Science, University of Science and Technology of China(人工智能与数据科学学院,中国科学技术大学);Suzhou Institute for Advanced Research, University of Science and Technology of China(苏州先进研究院,中国科学技术大学)
InterLCM: Low-Quality Images as Intermediate States of Latent Consistency Models for Effective Blind Face Restoration
Nankai University(南开大学);Mohamed bin Zayed University of AI(穆罕默德·本·扎耶德人工智能大学);Linkoping University(林雪平大学);SB Intuitions, SoftBank(软银SB Intuitions);Nankai International Advanced Research Institute (Shenzhen Futian), Nankai University(南开大学深圳福田国际先进研究院)
LATINO-PRO: LAtent consisTency INverse sOlver with PRompt Optimization
CNRS(法国国家科学研究中心);Univ. Bordeaux(波尔多大学);INRIA(法国国家信息与自动化研究所);Bordeaux INP(波尔多国立高等工程技术学院);Heriot-Watt University(赫瑞瓦特大学);Maxwell Institute for Mathematical Sciences(麦克斯韦数学科学研究所)
DreamLCM: Towards High-Quality Text-to-3D Generation via Latent Consistency Model
Institute of Information Science, Beijing Jiaotong University(北京交通大学信息科学研究所);Visual Intelligence + X International Joint Laboratory of the Ministry of Education(教育部视觉智能与X国际联合实验室);College of Electrical Engineering and Automation, Shandong University of Science and Technology(山东科技大学电气与自动化工程学院);PengCheng Laboratory(鹏城实验室)
SurgicaL-CD: Generating Surgical Images via Unpaired Image Translation with Latent Consistency Diffusion Models
NCT/UCC Dresden(德累斯顿癌症中心/癌症联盟);DKFZ Heidelberg(海德堡德国癌症研究中心);Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology(德累斯顿工业大学卡尔·古斯塔夫·卡鲁斯医学院及大学医院);HZDR Dresden(德累斯顿亥姆霍兹中心);The Centre for Tactile Internet with Human-in-the-Loop (CeTI), TUD Dresden University of Technology(德累斯顿工业大学人机交互触觉互联网中心)
Zero-Shot Video Restoration and Enhancement with Text-to-Image Latent Diffusion Models and Multi-Modal References
School of Electrical and Information Engineering, Tianjin University(天津大学电气自动化与信息工程学院);Lappeenranta-Lahti University of Technology LUT(拉彭兰塔-拉赫蒂理工大学(LUT))
PIXART-δ: Fast and Controllable Image Generation with Latent Consistency Models
Huawei Noah’s Ark Lab(华为诺亚方舟实验室);Dalian University of Technology(大连理工大学);The University of Hong Kong(香港大学);IIIS, Tsinghua University(清华大学智能产业研究院);Hugging Face
MoLingo: Motion-Language Alignment for Text-to-Human Motion Generation
University of Tübingen(图宾根大学);Tübingen AI Center(图宾根人工智能中心);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);Imperial College London(伦敦帝国理工学院);Zuse School ELIZA(Zuse ELIZA 学院)
LatentColorization: Latent Diffusion-Based Speaker Video Colorization
SFI Centre for Research Training in Artificial Intelligence(SFI人工智能研究培训中心);University of Galway(戈尔韦大学);School of Computer Science(计算机科学学院);School of Engineering(工程学院)