In this work, we investigate a simple and must-known conditional generative framework based on Vector Quantised-Variational AutoEncoder (VQ-VAE) and Generative Pre-trained Transformer (GPT) for human motion generation from textural descriptions. We show that a simple CNN-based VQ-VAE with commonly used training recipes (EMA and Code Reset) allows us to obtain high-quality discrete representations. For GPT, we incorporate a simple corruption strategy during the training to alleviate training-testing discrepancy. Despite its simplicity, our T2M-GPT shows better performance than competitive approaches, including recent diffusion-based approaches. For example, on HumanML3D, which is currently the largest dataset, we achieve comparable performance on the consistency between text and generated motion (R-Precision), but with FID 0.116 largely outperforming MotionDiffuse of 0.630. Additionally, we conduct analyses on HumanML3D and observe that the dataset size is a limitation of our approach. Our work suggests that VQ-VAE still remains a competitive approach for human motion generation.
GPT4Motion: Scripting Physical Motions in Text-to-Video Generation via Blender-Oriented GPT Planning
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University of Chinese Academy of Sciences(中国科学院大学);VIVO AI Lab(VIVO人工智能实验室)
LangXAI: Integrating Large Vision Models for Generating Textual Explanations to Enhance Explainability in Visual Perception Tasks
University of New Brunswick(新不伦瑞克大学);Quy Nhon AI, FPT Software(FPT软件公司归仁人工智能部门);Friedrich-Alexander-University Erlangen-Nürnberg(埃尔朗根-纽伦堡弗里德里希-亚历山大大学);Software AG(软件AG公司)
LangXAI 框架将 XAI 与先进视觉模型结合,为分类、检测和分割任务生成文本解释,以高 BERTScore 验证其合理性,从而提升终端用户对视觉 AI 的理解与信任。
3D-CT-GPT: Generating 3D Radiology Reports through Integration of Large Vision-Language Models
Northwestern Polytechnical University(西北工业大学);The Second Xiangya Hospital, Central South University(中南大学湘雅二医院);Clinical Research Center for Medical Imaging in Hunan Province(湖南省医学影像临床研究中心);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University of Georgia(佐治亚大学)
Generating Findings for Jaw Cysts in Dental Panoramic Radiographs Using a GPT-Based VLM: A Preliminary Study on Building a Two-Stage Self-Correction Loop with Structured Output (SLSO) Framework
Gifu University(岐阜大学);EyeTech Co., Ltd(EyeTech有限公司);Asahi University(朝日大学);Shiga University(滋贺大学);The University of Osaka Dental Hospital(大阪大学齿科医院)