多模态大模型 - arXivDaily 专题

2506.11571 2025-07-21 cs.CV 74%

VFaith: Do Large Multimodal Models Really Reason on Seen Images Rather than Previous Memories?

Jiachen Yu, Yufei Zhan, Ziheng Wu, Yousong Zhu, Jinqiao Wang, Minghui Qiu

机构 * Tsinghua University（清华大学）； Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所基础模型研究中心）； School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）； ByteDance China（字节跳动中国）； Peng Cheng Laboratory（鹏城实验室）； Wuhan AI Research（武汉人工智能研究）

专题命中多模态生成：multimodal(title)；分类 cs.CV

2507.05568 2025-07-09 cs.CV cs.LG 74%

ReLayout: Integrating Relation Reasoning for Content-aware Layout Generation with Multi-modal Large Language Models

Jiaxu Tian, Xuehui Yu, Yaoxing Wang, Pan Wang, Guangqian Guo, Shan Gao

专题命中多模态生成：multi-modal(title)；分类 cs.CV

2506.11182 2025-06-16 q-bio.GN cs.AI 74%

Multimodal Modeling of CRISPR-Cas12 Activity Using Foundation Models and Chromatin Accessibility Data

Azim Dehghani Amirabad, Yanfei Zhang, Artem Moskalev, Sowmya Rajesh, Tommaso Mansi, Shuwei Li, Mangal Prakash, Rui Liao

专题命中多模态生成：multimodal(title)；分类 cs.AI

Comments This manuscript has been accepted by ICML workshop 2025

2412.08468 2025-06-10 cs.RO cs.CV 74%

Multi-GraspLLM: A Multimodal LLM for Multi-Hand Semantic Guided Grasp Generation

Haosheng Li, Weixin Mao, Weipeng Deng, Chenyu Meng, Haoqiang Fan, Tiancai Wang, Yoshie Osamu, Ping Tan, Hongan Wang, Xiaoming Deng

机构 * Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）； Waseda University（早稻田大学）； University of Hong Kong（香港大学）； MEGVII Technology（美格智能科技）； Hong Kong University of Science and Technology（香港科技大学）

专题命中多模态生成：multimodal(title)；分类 cs.CV

Comments 16 pages, 10 figures

2505.10495 2025-05-26 cs.LG cs.CL 74%

RouteNator: A Router-Based Multi-Modal Architecture for Generating Synthetic Training Data for Function Calling LLMs

Vibha Belavadi, Tushar Vatsa, Dewang Sultania, Suhas Suresha, Ishita Verma, Cheng Chen, Tracy Holloway King, Michael Friedrich

机构 * Adobe Inc.（Adobe公司）

专题命中多模态生成：multi-modal(title)；分类 cs.CL

Comments Proceedings of the 4th International Workshop on Knowledge-Augmented Methods for Natural Language Processing

Journal ref https://aclanthology.org/2025.knowledgenlp-1.10/ KnowledgeNLP 2025

2412.01487 2025-05-07 cs.AI 74%

FastRM: An efficient and automatic explainability framework for multimodal generative models

Gabriela Ben-Melech Stan, Estelle Aflalo, Man Luo, Shachar Rosenman, Tiep Le, Sayak Paul, Shao-Yen Tseng, Vasudev Lal

机构 * Intel Labs（英特尔实验室）； Hugging Face

专题命中多模态生成：multimodal(title)；分类 cs.AI

2504.02542 2025-04-08 cs.CV 74%

Audio-visual Controlled Video Diffusion with Masked Selective State Spaces Modeling for Natural Talking Head Generation

Fa-Ting Hong, Zunnan Xu, Zixiang Zhou, Jun Zhou, Xiu Li, Qin Lin, Qinglin Lu, Dan Xu

专题命中多模态生成：audio-visual(title)；分类 cs.CV

2503.10639 2025-03-14 cs.CV 74%

GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editing

Rongyao Fang, Chengqi Duan, Kun Wang, Linjiang Huang, Hao Li, Shilin Yan, Hao Tian, Xingyu Zeng, Rui Zhao, Jifeng Dai, Xihui Liu, Hongsheng Li

专题命中多模态生成：multimodal(title)；分类 cs.CV

Comments Dataset and models are released in https://github.com/rongyaofang/GoT

2503.08165 2025-03-12 cs.CV 74%

Multimodal Generation of Animatable 3D Human Models with AvatarForge

Xinhang Liu, Yu-Wing Tai, Chi-Keung Tang

专题命中多模态生成：multimodal(title)；分类 cs.CV

2503.00619 2025-03-04 cs.IR cs.AI cs.LG 74%

PinLanding: Content-First Keyword Landing Page Generation via Multi-Modal AI for Web-Scale Discovery

Faye Zhang, Jasmine Wan, Qianyu Cheng, Jinfeng Rao

专题命中多模态生成：multi-modal(title)；分类 cs.AI

2502.04378 2025-02-10 cs.CV cs.GR cs.LG cs.SE 74%

DILLEMA: Diffusion and Large Language Models for Multi-Modal Augmentation

Luciano Baresi, Davide Yi Xian Hu, Muhammad Irfan Mas'udi, Giovanni Quattrocchi

专题命中多模态生成：multi-modal(title)；分类 cs.CV

2412.07320 2025-01-09 cs.CV 74%

CoMA: Compositional Human Motion Generation with Multi-modal Agents

Shanlin Sun, Gabriel De Araujo, Jiaqi Xu, Shenghan Zhou, Hanwen Zhang, Ziheng Huang, Chenyu You, Xiaohui Xie

专题命中多模态生成：multi-modal(title)；分类 cs.CV

Comments Project Page: https://gabrie-l.github.io/coma-page/

2410.06997 2024-12-30 eess.IV cs.CV 74%

Feasibility Study of a Diffusion-Based Model for Cross-Modal Generation of Knee MRI from X-ray: Integrating Radiographic Feature Information

Zhe Wang, Yung Hsin Chen, Aladine Chetouani, Fabian Bauer, Yuhua Ru, Fang Chen, Liping Zhang, Rachid Jennane, Mohamed Jarraya

专题命中多模态生成：cross-modal(title)；分类 cs.CV

2408.14368 2024-12-24 cs.RO cs.AI 74%

GR-MG: Leveraging Partially Annotated Data via Multi-Modal Goal-Conditioned Policy

Peiyan Li, Hongtao Wu, Yan Huang, Chilam Cheang, Liang Wang, Tao Kong

专题命中多模态生成：multi-modal(title)；分类 cs.AI

Comments 8 pages, 5 figures, RA-L

2406.10740 2024-11-12 cs.CV 74%

FreeMotion: MoCap-Free Human Motion Synthesis with Multimodal Large Language Models

Zhikai Zhang, Yitang Li, Haofeng Huang, Mingxian Lin, Li Yi

专题命中多模态生成：multimodal(title)；分类 cs.CV

2411.03982 2024-11-07 cs.CV 74%

ReEdit: Multimodal Exemplar-Based Image Editing with Diffusion Models

Ashutosh Srivastava, Tarun Ram Menta, Abhinav Java, Avadhoot Jadhav, Silky Singh, Surgan Jandial, Balaji Krishnamurthy

专题命中多模态生成：multimodal(title)；分类 cs.CV

Comments First three authors contributed equally to this work

2207.09185 2024-10-23 cs.LG cs.AI 74%

Multimodal hierarchical Variational AutoEncoders with Factor Analysis latent space

Alejandro Guerrero-López, Carlos Sevilla-Salcedo, Vanessa Gómez-Verdejo, Pablo M. Olmos

专题命中多模态生成：multimodal(title)；分类 cs.AI

Comments 21 pages main work, 2 pages supplementary, 14 figures

2406.13305 2024-10-15 cs.AI 74%

Multimodal MRI Accurately Identifies Amyloid Status in Unbalanced Cohorts in Alzheimer's Disease Continuum

Giorgio Dolci, Charles A. Ellis, Federica Cruciani, Lorenza Brusini, Anees Abrol, Ilaria Boscolo Galazzo, Gloria Menegaz, Vince D. Calhoun

专题命中多模态生成：multimodal(title)；分类 cs.AI

Comments 16 pages, 3 figures, provisionally accepted to a journal

2410.08100 2024-10-15 cs.CV 74%

CrackSegDiff: Diffusion Probability Model-based Multi-modal Crack Segmentation

Xiaoyan Jiang, Licheng Jiang, Anjie Wang, Kaiying Zhu, Yongbin Gao

专题命中多模态生成：multi-modal(title)；分类 cs.CV

2406.02936 2024-10-02 eess.IV cs.CV 74%

Radiomics-guided Multimodal Self-attention Network for Predicting Pathological Complete Response in Breast MRI

Jonghun Kim, Hyunjin Park

专题命中多模态生成：multimodal(title)；分类 cs.CV

Comments 5 pages, 5 figures, IEEE ISBI 2024 proceedings

2409.02919 2024-09-10 cs.CV 74%

HiPrompt: Tuning-free Higher-Resolution Generation with Hierarchical MLLM Prompts

Xinyu Liu, Yingqing He, Lanqing Guo, Xiang Li, Bu Jin, Peng Li, Yan Li, Chi-Min Chan, Qifeng Chen, Wei Xue, Wenhan Luo, Qifeng Liu, Yike Guo

专题命中多模态生成：MLLM(title)；分类 cs.CV

Comments https://liuxinyv.github.io/HiPrompt/

2406.02610 2024-06-06 q-bio.QM cs.AI cs.LG 74%

MoFormer: Multi-objective Antimicrobial Peptide Generation Based on Conditional Transformer Joint Multi-modal Fusion Descriptor

Li Wang, Xiangzheng Fu, Jiahao Yang, Xinyi Zhang, Xiucai Ye, Yiping Liu, Tetsuya Sakurai, Xiangxiang Zeng

专题命中多模态生成：multi-modal(title)；分类 cs.AI

2405.07164 2024-05-14 cs.CV 74%

Modeling Pedestrian Intrinsic Uncertainty for Multimodal Stochastic Trajectory Prediction via Energy Plan Denoising

Yao Liu, Quan Z. Sheng, Lina Yao

专题命中多模态生成：multimodal(title)；分类 cs.CV

2404.09977 2024-04-16 cs.CV 74%

MaxFusion: Plug&Play Multi-Modal Generation in Text-to-Image Diffusion Models

Nithin Gopalakrishnan Nair, Jeya Maria Jose Valanarasu, Vishal M Patel

专题命中多模态生成：multi-modal(title)；分类 cs.CV

2311.17516 2024-04-02 cs.CR cs.CV 74%

MMA-Diffusion: MultiModal Attack on Diffusion Models

Yijun Yang, Ruiyuan Gao, Xiaosen Wang, Tsung-Yi Ho, Nan Xu, Qiang Xu

专题命中多模态生成：multimodal(title)；分类 cs.CV

Comments CVPR 2024. Our codes and benchmarks are available at https://github.com/cure-lab/MMA-Diffusion

2403.08764 2024-03-14 cs.CV 74%

VLOGGER: Multimodal Diffusion for Embodied Avatar Synthesis

Enric Corona, Andrei Zanfir, Eduard Gabriel Bazavan, Nikos Kolotouros, Thiemo Alldieck, Cristian Sminchisescu

专题命中多模态生成：multimodal(title)；分类 cs.CV

Comments Project web: https://enriccorona.github.io/vlogger/

2311.04552 2024-02-27 eess.IV cs.CV 74%

A 3D generative model of pathological multi-modal MR images and segmentations

Virginia Fernandez, Walter Hugo Lopez Pinaya, Pedro Borges, Mark S. Graham, Tom Vercauteren, M. Jorge Cardoso

专题命中多模态生成：multi-modal(title)；分类 cs.CV

Comments Accepted for publication at the 2023 Deep Generative Models (DGM4MICCAI) MICCAI workshop (Vancouver, Canada)

2312.17353 2024-01-03 eess.SY cs.AI cs.LG cs.SY 74%

Towards Auto-Modeling of Formal Verification for NextG Protocols: A Multimodal cross- and self-attention Large Language Model Approach

Jingda Yang, Ying Wang

专题命中多模态生成：multimodal(title)；分类 cs.AI

2310.07287 2023-10-12 cs.MM 74%

Interactive Interior Design Recommendation via Coarse-to-fine Multimodal Reinforcement Learning

He Zhang, Ying Sun, Weiyu Guo, Yafei Liu, Haonan Lu, Xiaodong Lin, Hui Xiong

专题命中多模态生成：multimodal(title)；分类 cs.MM

Comments Accepted by ACM International Conference on Multimedia'23. 9 pages, 7 figures

2302.08706 2023-02-21 cs.CV 74%

Fine-grained Cross-modal Fusion based Refinement for Text-to-Image Synthesis

Haoran Sun, Yang Wang, Haipeng Liu, Biao Qian

专题命中多模态生成：cross-modal(title)；分类 cs.CV

Comments 13 pages, 8 figures, accepted by Chinese Journal of Electronics

1. 多模态生成 4979 篇

VFaith: Do Large Multimodal Models Really Reason on Seen Images Rather than Previous Memories?

ReLayout: Integrating Relation Reasoning for Content-aware Layout Generation with Multi-modal Large Language Models

Multimodal Modeling of CRISPR-Cas12 Activity Using Foundation Models and Chromatin Accessibility Data

Multi-GraspLLM: A Multimodal LLM for Multi-Hand Semantic Guided Grasp Generation

RouteNator: A Router-Based Multi-Modal Architecture for Generating Synthetic Training Data for Function Calling LLMs

FastRM: An efficient and automatic explainability framework for multimodal generative models

Audio-visual Controlled Video Diffusion with Masked Selective State Spaces Modeling for Natural Talking Head Generation

GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editing

Multimodal Generation of Animatable 3D Human Models with AvatarForge

PinLanding: Content-First Keyword Landing Page Generation via Multi-Modal AI for Web-Scale Discovery

DILLEMA: Diffusion and Large Language Models for Multi-Modal Augmentation

CoMA: Compositional Human Motion Generation with Multi-modal Agents

Feasibility Study of a Diffusion-Based Model for Cross-Modal Generation of Knee MRI from X-ray: Integrating Radiographic Feature Information

GR-MG: Leveraging Partially Annotated Data via Multi-Modal Goal-Conditioned Policy

FreeMotion: MoCap-Free Human Motion Synthesis with Multimodal Large Language Models

ReEdit: Multimodal Exemplar-Based Image Editing with Diffusion Models

Multimodal hierarchical Variational AutoEncoders with Factor Analysis latent space

Multimodal MRI Accurately Identifies Amyloid Status in Unbalanced Cohorts in Alzheimer's Disease Continuum

CrackSegDiff: Diffusion Probability Model-based Multi-modal Crack Segmentation

Radiomics-guided Multimodal Self-attention Network for Predicting Pathological Complete Response in Breast MRI

HiPrompt: Tuning-free Higher-Resolution Generation with Hierarchical MLLM Prompts

MoFormer: Multi-objective Antimicrobial Peptide Generation Based on Conditional Transformer Joint Multi-modal Fusion Descriptor

Modeling Pedestrian Intrinsic Uncertainty for Multimodal Stochastic Trajectory Prediction via Energy Plan Denoising

MaxFusion: Plug&Play Multi-Modal Generation in Text-to-Image Diffusion Models

MMA-Diffusion: MultiModal Attack on Diffusion Models

VLOGGER: Multimodal Diffusion for Embodied Avatar Synthesis

A 3D generative model of pathological multi-modal MR images and segmentations

Towards Auto-Modeling of Formal Verification for NextG Protocols: A Multimodal cross- and self-attention Large Language Model Approach

Interactive Interior Design Recommendation via Coarse-to-fine Multimodal Reinforcement Learning

Fine-grained Cross-modal Fusion based Refinement for Text-to-Image Synthesis