多模态大模型 - arXivDaily 专题

2505.22525 2025-05-29 cs.CV cs.AI cs.CL 67%

Thinking with Generated Images

Ethan Chern, Zhulin Hu, Steffi Chern, Siqi Kou, Jiadi Su, Yan Ma, Zhijie Deng, Pengfei Liu

机构 * Shanghai Jiao Tong University（上海交通大学）； SII ； Fudan University（复旦大学）； Generative AI Research Lab (GAIR)（生成式人工智能研究实验室）

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

2412.10419 2025-05-29 cs.CV cs.AI cs.CL cs.LG cs.SY eess.SY 67%

Preference Adaptive and Sequential Text-to-Image Generation

Ofir Nabati, Guy Tennenholtz, ChihWei Hsu, Moonkyung Ryu, Deepak Ramachandran, Yinlam Chow, Xiang Li, Craig Boutilier

机构 * International Conference on Machine Learning (ICML 2025)（国际机器学习会议）

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted to ICML 2025 Link to PASTA dataset: https://www.kaggle.com/datasets/googleai/pasta-data

2505.11178 2025-05-19 cs.CV cs.AI cs.CL 67%

CompAlign: Improving Compositional Text-to-Image Generation with a Complex Benchmark and Fine-Grained Feedback

Yixin Wan, Kai-Wei Chang

机构 * Department of Computer Science University of California, Los Angeles（计算机科学系，加州大学洛杉矶分校）

专题命中多模态生成：MLLM(abstract)；分类 cs.CV、cs.CL、cs.AI

2404.18624 2025-05-05 cs.CL cs.AI cs.CV cs.LG 67%

Do Vision & Language Decoders use Images and Text equally? How Self-consistent are their Explanations?

Letitia Parcalabescu, Anette Frank

机构 * Computational Linguistics Department（计算语言学系）； Heidelberg University（海德堡大学）

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 30 pages, 8 figures, 11 tables

2503.24354 2025-04-10 cs.LG cs.AI cs.CL cs.CV 67%

ORAL: Prompting Your Large-Scale LoRAs via Conditional Recurrent Diffusion

Rana Muhammad Shahroz Khan, Dongwen Tang, Pingzhi Li, Kai Wang, Tianlong Chen

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

2411.14062 2025-03-11 cs.CV cs.AI cs.CL cs.LG 67%

MMGenBench: Fully Automatically Evaluating LMMs from the Text-to-Image Generation Perspective

Hailang Huang, Yong Wang, Zixuan Huang, Huaqiu Li, Tongwen Huang, Xiangxiang Chu, Richong Zhang

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments This project is available at: https://github.com/lerogo/MMGenBench

2502.08556 2025-02-13 cs.CV cs.AI cs.LG cs.MM 67%

Human-Centric Foundation Models: Perception, Generation and Agentic Modeling

Shixiang Tang, Yizhou Wang, Lu Chen, Yuan Wang, Sida Peng, Dan Xu, Wanli Ouyang

专题命中多模态生成：multi-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments 9 pages

2502.02610 2025-02-06 cs.AI cs.CV cs.HC cs.MM 67%

Secure & Personalized Music-to-Video Generation via CHARCHA

Mehul Agarwal, Gauri Agarwal, Santiago Benoit, Andrew Lippman, Jean Oh

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments NeurIPS 2024 Creative AI Track

2406.10857 2025-01-03 cs.SE 67%

LMM-enhanced Safety-Critical Scenario Generation for Autonomous Driving System Testing From Non-Accident Traffic Videos

Haoxiang Tian, Xingshuo Han, Yuan Zhou, Guoquan Wu, An Guo, Mingfei Cheng, Shuo Li, Jun Wei, Tianwei Zhang

专题命中多模态生成：multimodal(abstract)；multi-modal(abstract)

Comments 17 pages

2412.18390 2024-12-30 cs.CV cs.AI cs.LG cs.MM 67%

RDPM: Solve Diffusion Probabilistic Models via Recurrent Token Prediction

Xiaoping Wu, Jie Hu, Xiaoming Wei

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments 8 pages

2412.12121 2024-12-18 cs.DL cs.AI cs.CL cs.CV cs.LG 67%

NLLG Quarterly arXiv Report 09/24: What are the most influential current AI Papers?

Christoph Leiter, Jonas Belouadi, Yanran Chen, Ran Zhang, Daniil Larionov, Aida Kostikova, Steffen Eger

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

2412.07801 2024-12-12 cs.CV cs.AI cs.CL 67%

Learning to Correction: Explainable Feedback Generation for Visual Commonsense Reasoning Distractor

Jiali Chen, Xusen Hei, Yuqi Xue, Yuancheng Wei, Jiayuan Xie, Yi Cai, Qing Li

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted by ACM MM 2024

2403.13802 2024-11-26 cs.CV cs.AI cs.CL cs.LG 67%

ZigMa: A DiT-style Zigzag Mamba Diffusion Model

Vincent Tao Hu, Stefan Andreas Baumann, Ming Gui, Olga Grebenkova, Pingchuan Ma, Johannes Schusterbauer, Björn Ommer

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments ECCV 2024 Project Page: https://taohu.me/zigma/

2411.15113 2024-11-25 cs.CV cs.AI cs.CL cs.LG 67%

Efficient Pruning of Text-to-Image Models: Insights from Pruning Stable Diffusion

Samarth N Ramesh, Zhixue Zhao

专题命中多模态生成：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

2411.09955 2024-11-22 cs.CV cs.AI cs.HC cs.LG cs.MM 67%

Instruction-Guided Editing Controls for Images and Multimedia: A Survey in LLM era

Thanh Tam Nguyen, Zhao Ren, Trinh Pham, Thanh Trung Huynh, Phi Le Nguyen, Hongzhi Yin, Quoc Viet Hung Nguyen

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Fixed a serious error in author information

2411.09933 2024-11-18 cs.CV cs.AI cs.CL cs.NE 67%

JRadiEvo: A Japanese Radiology Report Generation Model Enhanced by Evolutionary Optimization of Model Merging

Kaito Baba, Ryota Yagi, Junichiro Takahashi, Risa Kishikawa, Satoshi Kodera

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted by NeurIPS'24 Workshop on AIM-FM: Advancements In Medical Foundation Models: Explainability, Robustness, Security, and Beyond

2410.20359 2024-11-04 cs.SD cs.AI cs.CV cs.GR eess.AS 67%

Conditional GAN for Enhancing Diffusion Models in Efficient and Authentic Global Gesture Generation from Audios

Yongkang Cheng, Mingjiang Liang, Shaoli Huang, Gaoge Han, Jifeng Ning, Wei Liu

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.AI、eess.AS

Comments Accepted by WACV 2025 (Round 1)

2404.04251 2024-11-01 cs.CV cs.AI cs.CL 67%

Who Evaluates the Evaluations? Objectively Scoring Text-to-Image Prompt Coherence Metrics with T2IScoreScore (TS2)

Michael Saxon, Fatima Jahara, Mahsa Khoshnoodi, Yujie Lu, Aditya Sharma, William Yang Wang

专题命中多模态生成：cross-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments NeurIPS 2024 Spotlight

2401.10712 2024-10-15 cs.CV cs.AI cs.CL 67%

Q&A Prompts: Discovering Rich Visual Clues through Mining Question-Answer Prompts for VQA requiring Diverse World Knowledge

Haibo Wang, Weifeng Ge

专题命中多模态生成：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted by ECCV'24

2406.00093 2024-10-04 cs.CV cs.AI cs.GR cs.LG cs.MM 67%

Bootstrap3D: Improving Multi-view Diffusion Model with Synthetic Data

Zeyi Sun, Tong Wu, Pan Zhang, Yuhang Zang, Xiaoyi Dong, Yuanjun Xiong, Dahua Lin, Jiaqi Wang

专题命中多模态生成：image-text(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Project Page: https://sunzey.github.io/Bootstrap3D/

2311.16201 2024-09-26 cs.CV cs.AI cs.CL cs.LG 67%

Pre-trained Language Models Do Not Help Auto-regressive Text-to-Image Generation

Yuhui Zhang, Brandon McKinzie, Zhe Gan, Vaishaal Shankar, Alexander Toshev

专题命中多模态生成：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Published at EMNLP 2024 Main Conference

2310.12128 2024-07-16 cs.CV cs.AI cs.CL cs.LG 67%

DiagrammerGPT: Generating Open-Domain, Open-Platform Diagrams via LLM Planning

Abhay Zala, Han Lin, Jaemin Cho, Mohit Bansal

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments COLM 2024; Project page: https://diagrammerGPT.github.io/

2407.03188 2024-07-12 cs.SD cs.AI cs.MM eess.AS 67%

MuDiT & MuSiT: Alignment with Colloquial Expression in Description-to-Song Generation

Zihao Wang, Haoxuan Liu, Jiaxing Yu, Tao Zhang, Yan Liu, Kejun Zhang

专题命中多模态生成：cross-modal(abstract)；分类 cs.AI、cs.MM、eess.AS

Comments 19 pages, 5 figures

2407.07771 2024-07-11 cs.CL cs.CV cs.MM 67%

Multi-task Prompt Words Learning for Social Media Content Generation

Haochen Xue, Chong Zhang, Chengzhi Liu, Fangyu Wu, Xiaobo Jin

专题命中多模态生成：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.MM

Comments 8 pages, 5 figures

Journal ref International Joint Conference on Neural Networks 2024

2402.13254 2024-06-13 cs.CV cs.AI cs.CL cs.LG 67%

CounterCurate: Enhancing Physical and Semantic Visio-Linguistic Compositional Reasoning via Counterfactual Examples

Jianrui Zhang, Mu Cai, Tengyang Xie, Yong Jae Lee

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 15 pages, 6 figures, 12 tables, Project Page: https://countercurate.github.io/

2405.16277 2024-06-04 cs.CL cs.AI cs.CV cs.LG 67%

Picturing Ambiguity: A Visual Twist on the Winograd Schema Challenge

Brendan Park, Madeline Janecek, Naser Ezzati-Jivan, Yifeng Li, Ali Emami

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 9 pages (excluding references), accepted to ACL 2024 Main Conference

2305.12328 2024-05-30 cs.CV cs.AI cs.MM 67%

InstructVid2Vid: Controllable Video Editing with Natural Language Instructions

Bosheng Qin, Juncheng Li, Siliang Tang, Tat-Seng Chua, Yueting Zhuang

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted by ICME 2024

2404.18928 2024-04-30 cs.CV cs.AI cs.CL cs.GR cs.LG 67%

Stylus: Automatic Adapter Selection for Diffusion Models

Michael Luo, Justin Wong, Brandon Trabucco, Yanping Huang, Joseph E. Gonzalez, Zhifeng Chen, Ruslan Salakhutdinov, Ion Stoica

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Project Website: https://stylus-diffusion.github.io

2404.13899 2024-04-23 cs.CL cs.AI cs.MM 67%

Towards Better Text-to-Image Generation Alignment via Attention Modulation

Yihang Wu, Xiao Cao, Kaixin Li, Zitan Chen, Haonan Wang, Lei Meng, Zhiyong Huang

专题命中多模态生成：image-text(abstract)；分类 cs.CL、cs.AI、cs.MM

2312.02439 2024-04-23 cs.AI cs.CL cs.CV 67%

Let's Think Outside the Box: Exploring Leap-of-Thought in Large Language Models with Creative Humor Generation

Shanshan Zhong, Zhongzhan Huang, Shanghua Gao, Wushao Wen, Liang Lin, Marinka Zitnik, Pan Zhou

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Technical report

1. 多模态生成 4975 篇

Thinking with Generated Images

Preference Adaptive and Sequential Text-to-Image Generation

CompAlign: Improving Compositional Text-to-Image Generation with a Complex Benchmark and Fine-Grained Feedback

Do Vision & Language Decoders use Images and Text equally? How Self-consistent are their Explanations?

ORAL: Prompting Your Large-Scale LoRAs via Conditional Recurrent Diffusion

MMGenBench: Fully Automatically Evaluating LMMs from the Text-to-Image Generation Perspective

Human-Centric Foundation Models: Perception, Generation and Agentic Modeling

Secure & Personalized Music-to-Video Generation via CHARCHA

LMM-enhanced Safety-Critical Scenario Generation for Autonomous Driving System Testing From Non-Accident Traffic Videos

RDPM: Solve Diffusion Probabilistic Models via Recurrent Token Prediction

NLLG Quarterly arXiv Report 09/24: What are the most influential current AI Papers?

Learning to Correction: Explainable Feedback Generation for Visual Commonsense Reasoning Distractor

ZigMa: A DiT-style Zigzag Mamba Diffusion Model

Efficient Pruning of Text-to-Image Models: Insights from Pruning Stable Diffusion

Instruction-Guided Editing Controls for Images and Multimedia: A Survey in LLM era

JRadiEvo: A Japanese Radiology Report Generation Model Enhanced by Evolutionary Optimization of Model Merging

Conditional GAN for Enhancing Diffusion Models in Efficient and Authentic Global Gesture Generation from Audios

Who Evaluates the Evaluations? Objectively Scoring Text-to-Image Prompt Coherence Metrics with T2IScoreScore (TS2)

Q&A Prompts: Discovering Rich Visual Clues through Mining Question-Answer Prompts for VQA requiring Diverse World Knowledge

Bootstrap3D: Improving Multi-view Diffusion Model with Synthetic Data

Pre-trained Language Models Do Not Help Auto-regressive Text-to-Image Generation

DiagrammerGPT: Generating Open-Domain, Open-Platform Diagrams via LLM Planning

MuDiT & MuSiT: Alignment with Colloquial Expression in Description-to-Song Generation

Multi-task Prompt Words Learning for Social Media Content Generation

CounterCurate: Enhancing Physical and Semantic Visio-Linguistic Compositional Reasoning via Counterfactual Examples

Picturing Ambiguity: A Visual Twist on the Winograd Schema Challenge

InstructVid2Vid: Controllable Video Editing with Natural Language Instructions

Stylus: Automatic Adapter Selection for Diffusion Models

Towards Better Text-to-Image Generation Alignment via Attention Modulation

Let's Think Outside the Box: Exploring Leap-of-Thought in Large Language Models with Creative Humor Generation