多模态大模型 - arXivDaily 专题

2506.10395 2025-07-15 cs.CV cs.AI 73%

Pisces: An Auto-regressive Foundation Model for Image Understanding and Generation

Zhiyang Xu, Jiuhai Chen, Zhaojiang Lin, Xichen Pan, Lifu Huang, Tianyi Zhou, Madian Khabsa, Qifan Wang, Di Jin, Michihiro Yasunaga, Lili Yu, Xi Victoria Lin, Shaoliang Nie

机构 * Meta ； University of Maryland（马里兰大学）； New York University（纽约大学）

专题命中多模态生成：multimodal(abstract)；multimodal foundation model(abstract)；分类 cs.CV、cs.AI

Comments Unified image understanding and generation model

2506.18658 2025-06-24 cs.CV cs.AI 73%

Historical Report Guided Bi-modal Concurrent Learning for Pathology Report Generation

Ling Zhang, Boxiang Yun, Qingli Li, Yan Wang

机构 * Shanghai Key Laboratory of Multidimensional Information Processing, \ China Normal University, Shanghai, China

专题命中多模态生成：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

2503.16563 2025-06-12 cs.LG cs.AI cs.CL q-bio.BM 73%

Chem42: a Family of chemical Language Models for Target-aware Ligand Generation

Aahan Singh, Engin Tekin, Maryam Nadeem, Nancy A. ElNaker, Mohammad Amaan Sayeed, Natalia Vassilieva, Boulbaba Ben Amor

机构 * Inception Institute of Artificial Intelligence（Inception人工智能研究院）； Cerebras Systems（Cerebras系统）

专题命中多模态生成：multimodal(abstract)；cross-modal(abstract)；分类 cs.CL、cs.AI

2505.14151 2025-06-05 cs.CV cs.MM 73%

ReactDiff: Latent Diffusion for Facial Reaction Generation

Jiaming Li, Sheng Wang, Xin Wang, Yitao Zhu, Honglin Xiong, Zixu Zhuang, Qian Wang

机构 * School of Biomedical Engineering & State Key Laboratory of Advanced Medical Materials and Devices, ShanghaiTech University（生物医学工程学院及先进医学材料与器件国家重点实验室，上海科技大学）； School of Biomedical Engineering, Shanghai Jiao Tong University（生物医学工程学院，上海交通大学）； Shanghai Clinical Research and Trial Center（上海临床研究与试验中心）

专题命中多模态生成：multi-modal(abstract)；audio-visual(abstract)；分类 cs.CV、cs.MM

Comments Accepted by Neural Networks

2410.17885 2025-06-02 cs.AI cs.CV 73%

Theorem-Validated Reverse Chain-of-Thought Problem Generation for Geometric Reasoning

Linger Deng, Linghao Zhu, Yuliang Liu, Yu Wang, Qunyi Xie, Jingjing Wu, Gang Zhang, Yingying Zhu, Xiang Bai

机构 * Huazhong University of Science and Technology（华中科技大学）； Department of Computer Vision Technology, Baidu Inc（百度公司计算机视觉技术部门）

专题命中多模态生成：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

2505.20271 2025-05-27 cs.CV cs.AI cs.GR 73%

In-Context Brush: Zero-shot Customized Subject Insertion with Context-Aware Latent Space Manipulation

Yu Xu, Fan Tang, You Wu, Lin Gao, Oliver Deussen, Hongbin Yan, Jintao Li, Juan Cao, Tong-Yee Lee

专题命中多模态生成：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

2505.13082 2025-05-20 cs.SD cs.AI eess.AS 73%

MultiActor-Audiobook: Zero-Shot Audiobook Generation with Faces and Voices of Multiple Speakers

Kyeongman Park, Seongho Joo, Kyomin Jung

机构 * Seoul National University（首尔国立大学）

专题命中多模态生成：multimodal(abstract)；MLLM(abstract)；分类 cs.AI、eess.AS

2410.02240 2025-05-14 cs.CV cs.AI 73%

SCA: Improve Semantic Consistent in Unrestricted Adversarial Attacks via DDPM Inversion

Zihao Pan, Lifeng Chen, Weibin Wu, Yuhang Cao, Zibin Zheng

机构 * School of Software Engineering, Sun Yat-sen University（中山大学软件工程学院）； Computer Science, Beijing Jiaotong University（北京交通大学计算机科学）

专题命中多模态生成：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

2503.22145 2025-03-31 cs.LG cs.CL cs.CV cs.HC 73%

Tokenization of Gaze Data

Tim Rolff, Jurik Karimian, Niklas Hypki, Susanne Schmidt, Markus Lappe, Frank Steinicke

专题命中多模态生成：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

2503.22138 2025-03-31 cs.SD cs.CV eess.AS 73%

Enhancing Dance-to-Music Generation via Negative Conditioning Latent Diffusion Model

Changchang Sun, Gaowen Liu, Charles Fleming, Yan Yan

专题命中多模态生成：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、eess.AS

2503.14908 2025-03-20 cs.GR cs.AI cs.CV 73%

POSTA: A Go-to Framework for Customized Artistic Poster Generation

Haoyu Chen, Xiaojie Xu, Wenbo Li, Jingjing Ren, Tian Ye, Songhua Liu, Ying-Cong Chen, Lei Zhu, Xinchao Wang

专题命中多模态生成：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Accepted to CVPR 2025

2308.09599 2025-02-26 cs.CV cs.MM 73%

Language-Guided Diffusion Model for Visual Grounding

Sijia Chen, Baochun Li

专题命中多模态生成：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.MM

Comments 20 pages, 16 figures

2412.15220 2024-12-23 cs.MM cs.SD eess.AS 73%

SyncFlow: Toward Temporally Aligned Joint Audio-Video Generation from Text

Haohe Liu, Gael Le Lan, Xinhao Mei, Zhaoheng Ni, Anurag Kumar, Varun Nagaraja, Wenwu Wang, Mark D. Plumbley, Yangyang Shi, Vikas Chandra

专题命中多模态生成：multi-modal(abstract)；audio-visual(abstract)；分类 cs.MM、eess.AS

2411.19220 2024-12-02 cs.CV cs.MM 73%

Automatic Prompt Generation and Grounding Object Detection for Zero-Shot Image Anomaly Detection

Tsun-Hin Cheung, Ka-Chun Fung, Songjiang Lai, Kwan-Ho Lin, Vincent Ng, Kin-Man Lam

专题命中多模态生成：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.MM

Comments Accepted to APSIPA ASC 2024

2411.17066 2024-11-27 cs.CV cs.CL cs.SC 73%

Relations, Negations, and Numbers: Looking for Logic in Generative Text-to-Image Models

Colin Conwell, Rupert Tawiah-Quashie, Tomer Ullman

专题命中多模态生成：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV、cs.CL

2411.15453 2024-11-26 cs.CV cs.AI 73%

Enhancing Instruction-Following Capability of Visual-Language Models by Reducing Image Redundancy

Te Yang, Jian Jia, Xiangyu Zhu, Weisong Zhao, Bo Wang, Yanhua Cheng, Yan Li, Shengyuan Liu, Quan Chen, Peng Jiang, Kun Gai, Zhen Lei

专题命中多模态生成：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

2406.04485 2024-11-12 cs.AI cs.CV 73%

GenAI Arena: An Open Evaluation Platform for Generative Models

Dongfu Jiang, Max Ku, Tianle Li, Yuansheng Ni, Shizhuo Sun, Rongqi Fan, Wenhu Chen

专题命中多模态生成：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 9 pages,7 figures

Journal ref NeurIPS 2024

2411.01805 2024-11-05 cs.SD cs.MM eess.AS 73%

MoMu-Diffusion: On Learning Long-Term Motion-Music Synchronization and Correspondence

Fuming You, Minghui Fang, Li Tang, Rongjie Huang, Yongqi Wang, Zhou Zhao

专题命中多模态生成：multi-modal(abstract)；cross-modal(abstract)；分类 cs.MM、eess.AS

Comments NeurIPS 2024

2409.17508 2024-11-04 cs.CV cs.AI cs.LG 73%

Uni-Med: A Unified Medical Generalist Foundation Model For Multi-Task Learning Via Connector-MoE

Xun Zhu, Ying Hu, Fanbin Mo, Miao Li, Ji Wu

专题命中多模态生成：multi-modal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

2410.03334 2024-10-07 cs.CV cs.AI 73%

An X-Ray Is Worth 15 Features: Sparse Autoencoders for Interpretable Radiology Report Generation

Ahmed Abdulaal, Hugo Fry, Nina Montaña-Brown, Ayodeji Ijishakin, Jack Gao, Stephanie Hyland, Daniel C. Alexander, Daniel C. Castro

专题命中多模态生成：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV、cs.AI

2409.06945 2024-09-12 cs.CV cs.AI 73%

FSMDet: Vision-guided feature diffusion for fully sparse 3D detector

Tianran Liu, Morteza Mousa Pasandi, Robert Laganiere

专题命中多模态生成：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by European Conference on Computer Vision (ECCV) 2024 workshop on VCAD

2404.14715 2024-07-23 cs.CV cs.CL 73%

FINEMATCH: Aspect-based Fine-grained Image and Text Mismatch Detection and Correction

Hang Hua, Jing Shi, Kushal Kafle, Simon Jenni, Daoan Zhang, John Collomosse, Scott Cohen, Jiebo Luo

专题命中多模态生成：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Comments ECCV 2024

2406.08478 2024-06-19 cs.CV cs.CL 73%

What If We Recaption Billions of Web Images with LLaMA-3?

Xianhang Li, Haoqin Tu, Mude Hui, Zeyu Wang, Bingchen Zhao, Junfei Xiao, Sucheng Ren, Jieru Mei, Qing Liu, Huangjie Zheng, Yuyin Zhou, Cihang Xie

专题命中多模态生成：cross-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Comments First five authors contributed equally

2406.07502 2024-06-12 cs.CV cs.CL 73%

Image Textualization: An Automatic Framework for Creating Accurate and Detailed Image Descriptions

Renjie Pi, Jianshu Zhang, Jipeng Zhang, Rui Pan, Zhekai Chen, Tong Zhang

专题命中多模态生成：multi-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

2403.07750 2024-06-10 cs.CV cs.AI 73%

Synth$^2$: Boosting Visual-Language Models with Synthetic Captions and Image Embeddings

Sahand Sharifzadeh, Christos Kaplanis, Shreya Pathak, Dharshan Kumaran, Anastasija Ilic, Jovana Mitrovic, Charles Blundell, Andrea Banino

专题命中多模态生成：multi-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments 9 pages, 6 figures

2405.15439 2024-05-27 cs.CV cs.AI 73%

Text-guided 3D Human Motion Generation with Keyframe-based Parallel Skip Transformer

Zichen Geng, Caren Han, Zeeshan Hayder, Jian Liu, Mubarak Shah, Ajmal Mian

专题命中多模态生成：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

2402.05195 2024-04-11 cs.CV cs.CL 73%

$λ$-ECLIPSE: Multi-Concept Personalized Text-to-Image Diffusion Models by Leveraging CLIP Latent Space

Maitreya Patel, Sangmin Jung, Chitta Baral, Yezhou Yang

专题命中多模态生成：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Comments Project page: https://eclipse-t2i.github.io/Lambda-ECLIPSE/

2404.03590 2024-04-05 cs.CV cs.AI 73%

SemGrasp: Semantic Grasp Generation via Language Aligned Discretization

Kailin Li, Jingbo Wang, Lixin Yang, Cewu Lu, Bo Dai

专题命中多模态生成：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

2304.14178 2024-04-01 cs.CL cs.CV cs.LG 73%

mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality

Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, Chenliang Li, Yuanhong Xu, Hehong Chen, Junfeng Tian, Qi Qian, Ji Zhang, Fei Huang, Jingren Zhou

专题命中多模态生成：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Working in Process

2308.09911 2024-03-29 cs.CV cs.MM 73%

Noisy-Correspondence Learning for Text-to-Image Person Re-identification

Yang Qin, Yingke Chen, Dezhong Peng, Xi Peng, Joey Tianyi Zhou, Peng Hu

专题命中多模态生成：cross-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.MM

1. 多模态生成 4979 篇

Pisces: An Auto-regressive Foundation Model for Image Understanding and Generation

Historical Report Guided Bi-modal Concurrent Learning for Pathology Report Generation

Chem42: a Family of chemical Language Models for Target-aware Ligand Generation

ReactDiff: Latent Diffusion for Facial Reaction Generation

Theorem-Validated Reverse Chain-of-Thought Problem Generation for Geometric Reasoning

In-Context Brush: Zero-shot Customized Subject Insertion with Context-Aware Latent Space Manipulation

MultiActor-Audiobook: Zero-Shot Audiobook Generation with Faces and Voices of Multiple Speakers

SCA: Improve Semantic Consistent in Unrestricted Adversarial Attacks via DDPM Inversion

Tokenization of Gaze Data

Enhancing Dance-to-Music Generation via Negative Conditioning Latent Diffusion Model

POSTA: A Go-to Framework for Customized Artistic Poster Generation

Language-Guided Diffusion Model for Visual Grounding

SyncFlow: Toward Temporally Aligned Joint Audio-Video Generation from Text

Automatic Prompt Generation and Grounding Object Detection for Zero-Shot Image Anomaly Detection

Relations, Negations, and Numbers: Looking for Logic in Generative Text-to-Image Models

Enhancing Instruction-Following Capability of Visual-Language Models by Reducing Image Redundancy

GenAI Arena: An Open Evaluation Platform for Generative Models

MoMu-Diffusion: On Learning Long-Term Motion-Music Synchronization and Correspondence

Uni-Med: A Unified Medical Generalist Foundation Model For Multi-Task Learning Via Connector-MoE

An X-Ray Is Worth 15 Features: Sparse Autoencoders for Interpretable Radiology Report Generation

FSMDet: Vision-guided feature diffusion for fully sparse 3D detector

FINEMATCH: Aspect-based Fine-grained Image and Text Mismatch Detection and Correction

What If We Recaption Billions of Web Images with LLaMA-3?

Image Textualization: An Automatic Framework for Creating Accurate and Detailed Image Descriptions

Synth$^2$: Boosting Visual-Language Models with Synthetic Captions and Image Embeddings

Text-guided 3D Human Motion Generation with Keyframe-based Parallel Skip Transformer

$λ$-ECLIPSE: Multi-Concept Personalized Text-to-Image Diffusion Models by Leveraging CLIP Latent Space

SemGrasp: Semantic Grasp Generation via Language Aligned Discretization

mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality

Noisy-Correspondence Learning for Text-to-Image Person Re-identification