CoMA: Compositional Human Motion Generation with Multi-modal Agents
University of California, Irvine(加利福尼亚大学欧文分校);Stony Brook University(石溪大学);Southeast University(东南大学);Chongqing University(重庆大学);Huazhong University of Science and Technology(华中科技大学);Northeastern University(东北大学)
Shanlin Sun, Gabriel De Araujo, Jiaqi Xu, Shenghan Zhou, Hanwen Zhang, Ziheng Huang, Chenyu You, Xiaohui Xie
3D human motion generation has seen substantial advancement in recent years. While state-of-the-art approaches have improved performance significantly, they still struggle with complex and detailed motions unseen in training data, largely due to the scarcity of motion datasets and the prohibitive cost of generating new training examples. To address these challenges, we introduce CoMA, an agent-based solution for complex human motion generation, editing, and comprehension. CoMA leverages multiple collaborative agents powered by large language and vision models, alongside a mask transformer-based motion generator featuring body part-specific encoders and codebooks for fine-grained control. Our framework enables generation of both short and long motion sequences with detailed instructions, text-guided motion editing, and self-correction for improved quality. Evaluations on the HumanML3D dataset demonstrate competitive performance against state-of-the-art methods. Additionally, we create a set of context-rich, compositional, and long text prompts, where user studies show our method significantly outperforms existing approaches.
Compositional Cross-Modality Translation via Whole-Volume Multitask Latent Flow Matching
Università Campus Bio-Medico di Roma(罗马 Campus Bio-Medico 大学);Umeå University(于默奥大学);UniCamillus – Saint Camillus International University of Health Sciences(UniCamillus – 圣卡米勒斯国际健康科学大学)
ORCA: Hunting Compositional Failures in Text-to-Image Diffusion
Wellcome Sanger Institute(威康桑格研究所);Cambridge Stem Cell Institute, University of Cambridge(剑桥大学剑桥干细胞研究所);Cambridge Centre for AI in Medicine, University of Cambridge(剑桥大学剑桥人工智能医学中心);University of Oxford(牛津大学);Apple(苹果公司);University of Helsinki(赫尔辛基大学);Institute for Molecular Medicine Finland (FIMM), University of Helsinki(赫尔辛基大学芬兰分子医学研究所);School of Computing and Information Systems, University of Melbourne(墨尔本大学计算与信息系统学院);Department of Medicine, University of Cambridge(剑桥大学医学系)
TTOM: Test-Time Optimization and Memorization for Compositional Video Generation
National University of Singapore(国立新加坡大学);University of Science and Technology of China(中国科学技术大学);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳))
Cross-Modal Masked Compositional Concept Modeling for Enhancing Visio-Linguistic Compositionality
MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China(中国科学技术大学,教育部脑启发智能感知与认知重点实验室);Independent Researcher(独立研究员)
CCSD: Cross-Modal Compositional Self-Distillation for Robust Brain Tumor Segmentation with Missing Modalities
Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University(上海智能自主系统研究院,同济大学);College of Biomedical Engineering, Fudan University(生物医学工程学院,复旦大学);School of Medicine, Tongji University(医学院,同济大学);College of Information Engineering, China Jiliang University(信息工程学院,中国嘉兴大学)
PaCE: Unified Multi-modal Dialogue Pre-training with Progressive and Compositional Experts
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University of Chinese Academy of Sciences(中国科学院大学);DAMO Academy, Alibaba Group(阿里巴巴集团达摩院);University of Science and Technology of China(中国科学技术大学)
Multi-Sourced Compositional Generalization in Visual Question Answering
Beijing Key Laboratory of Intelligent Information Technology, School of Computer Science & Technology, Beijing Institute of Technology, China(北京智能信息科技重点实验室,计算机科学与技术学院,北京理工大学,中国);Guangdong Laboratory of Machine Perception and Intelligent Computing, Shenzhen MSU-BIT University, China(广东机器感知与智能计算实验室,深圳MSU-BIT大学,中国)
Compositional Segmentation of Cardiac Images Leveraging Metadata
School of Electronic Engineering and Computer Science, Queen Mary University of London(伦敦玛丽女王大学电子工程与计算机科学学院);Queen Mary’s Digital Environment Research Institute (DERI)(玛丽女王数字环境研究所);Department of Computer Science, University College London(伦敦大学学院计算机科学系);School of Engineering and Materials Science, Queen Mary University of London(伦敦玛丽女王大学工程与材料科学学院)
ViLPAct: A Benchmark for Compositional Generalization on Multimodal Human Activities
Monash University(莫纳什大学);University of Electronic Science and Technology of China(电子科技大学);University of Technology Sydney(悉尼科技大学);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Peng Cheng Lab(鹏城实验室)
MUSE: Benchmarking Large Vision-Language Models on Multi-Modal Understanding in Situated Education
AI Singapore, National University of Singapore(新加坡国立大学新加坡人工智能);School of Computing, National University of Singapore(新加坡国立大学计算学院);Institute of Advanced Intelligence and Computing, A*STAR(新加坡科技研究局先进智能与计算研究所)
Compressing then Matching: An Efficient Pre-training Paradigm for Multimodal Embedding
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences(人工智能安全国家重点实验室,计算技术研究所,中国科学院);University of Chinese Academy of Sciences(中国科学院大学);Kuaishou Technology(快手科技)