多模态大模型 - arXivDaily 专题

2310.07944 2023-12-05 cs.AI 79%

AutoRepo: A general framework for multi-modal LLM-based automated construction reporting

Hongxu Pu, Xincong Yang, Jing Li, Runhao Guo, Heng Li

专题命中多模态生成：multi-modal(title)；multimodal(abstract)；分类 cs.AI

Comments We believe that keeping this version of the paper publicly available may lead to confusion or misinterpretation regarding our current research direction and findings

2303.10868 2023-12-04 cs.CL 79%

Retrieving Multimodal Information for Augmented Generation: A Survey

Ruochen Zhao, Hailin Chen, Weishi Wang, Fangkai Jiao, Xuan Long Do, Chengwei Qin, Bosheng Ding, Xiaobao Guo, Minzhi Li, Xingxuan Li, Shafiq Joty

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CL

2311.05199 2023-11-10 cs.CV 79%

BrainNetDiff: Generative AI Empowers Brain Network Generation via Multimodal Diffusion Model

Yongcheng Zong, Shuqiang Wang

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

2310.20251 2023-11-01 cs.MM 79%

An Implementation of Multimodal Fusion System for Intelligent Digital Human Generation

Yingjie Zhou, Yaodong Chen, Kaiyue Bi, Lian Xiong, Hui Liu

专题命中多模态生成：multimodal(title,abstract)；分类 cs.MM

2310.16131 2023-10-26 cs.CL 79%

GenKIE: Robust Generative Multimodal Document Key Information Extraction

Panfeng Cao, Ye Wang, Qiang Zhang, Zaiqiao Meng

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CL

Comments Accepted by EMNLP 2023, Findings paper

2310.13787 2023-10-24 cs.LG cs.AI 79%

Enhancing Illicit Activity Detection using XAI: A Multimodal Graph-LLM Framework

Jack Nicholls, Aditya Kuppa, Nhien-An Le-Khac

专题命中多模态生成：multimodal(title,abstract)；分类 cs.AI

Comments 6 pages, 3 figures

2310.05355 2023-10-10 cs.CV 79%

C^2M-DoT: Cross-modal consistent multi-view medical report generation with domain transfer network

Ruizhi Wang, Xiangtao Wang, Jie Zhou, Thomas Lukasiewicz, Zhenghua Xu

专题命中多模态生成：cross-modal(title,abstract)；分类 cs.CV

2309.03364 2023-09-08 cs.SD eess.AS 79%

Highly Controllable Diffusion-based Any-to-Any Voice Conversion Model with Frame-level Prosody Feature

Kyungguen Byun, Sunkuk Moon, Erik Visser

专题命中多模态生成：any-to-any(title,abstract)；分类 eess.AS

Comments 5 pages, 3 figures, submitted to ICASSP 2024

2309.01981 2023-09-06 cs.RO cs.AI cs.GR 79%

Graph-Based Interaction-Aware Multimodal 2D Vehicle Trajectory Prediction using Diffusion Graph Convolutional Networks

Keshu Wu, Yang Zhou, Haotian Shi, Xiaopeng Li, Bin Ran

专题命中多模态生成：multimodal(title)；multi-modal(abstract)；分类 cs.AI

2308.14748 2023-08-29 cs.GR cs.CV 79%

MagicAvatar: Multimodal Avatar Generation and Animation

Jianfeng Zhang, Hanshu Yan, Zhongcong Xu, Jiashi Feng, Jun Hao Liew

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

Comments Project page: https://magic-avatar.github.io/

2112.13592 2023-08-25 cs.CV 79%

Multimodal Image Synthesis and Editing: The Generative AI Era

Fangneng Zhan, Yingchen Yu, Rongliang Wu, Jiahui Zhang, Shijian Lu, Lingjie Liu, Adam Kortylewski, Christian Theobalt, Eric Xing

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

Comments TPAMI 2023

2308.11206 2023-08-23 cs.CV 79%

DiffCloth: Diffusion Based Garment Synthesis and Manipulation via Structural Cross-modal Semantic Alignment

Xujie Zhang, Binbin Yang, Michael C. Kampffmeyer, Wenqing Zhang, Shiyue Zhang, Guansong Lu, Liang Lin, Hang Xu, Xiaodan Liang

专题命中多模态生成：cross-modal(title,abstract)；分类 cs.CV

Comments accepted by ICCV2023

2308.08849 2023-08-21 cs.CV 79%

A Survey on Deep Multi-modal Learning for Body Language Recognition and Generation

Li Liu, Lufei Gao, Wentao Lei, Fengji Ma, Xiaotian Lin, Jinting Wang

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CV

2308.04733 2023-08-15 cs.CV 79%

TextPainter: Multimodal Text Image Generation with Visual-harmony and Text-comprehension for Poster Design

Yifan Gao, Jinpeng Lin, Min Zhou, Chuanbin Liu, Hongtao Xie, Tiezheng Ge, Yuning Jiang

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

Comments Accepted to ACM MM 2023. Dataset Link: https://tianchi.aliyun.com/dataset/160034

2307.01969 2023-07-06 cs.CV 79%

Multimodal Prompt Learning for Product Title Generation with Extremely Limited Labels

Bang Yang, Fenglin Liu, Zheng Li, Qingyu Yin, Chenyu You, Bing Yin, Yuexian Zou

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

Comments accepted by ACL Findings 2023

2307.01425 2023-07-06 cs.CV 79%

Consistent Multimodal Generation via A Unified GAN Framework

Zhen Zhu, Yijun Li, Weijie Lyu, Krishna Kumar Singh, Zhixin Shu, Soeren Pirk, Derek Hoiem

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

Comments In review

2306.16649 2023-06-30 cs.CL 79%

ZeroGen: Zero-shot Multimodal Controllable Text Generation with Multiple Oracles

Haoqin Tu, Bowen Yang, Xianfeng Zhao

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CL

Comments 17 pages, preprint

2306.09385 2023-06-19 cs.LG cs.AI eess.SP 79%

Employing Multimodal Machine Learning for Stress Detection

Rahee Walambe, Pranav Nayak, Ashmit Bhardwaj, Ketan Kotecha

专题命中多模态生成：multimodal(title,abstract)；分类 cs.AI

2306.01322 2023-06-05 cs.LG cs.CR cs.CV 79%

Privacy Distillation: Reducing Re-identification Risk of Multimodal Diffusion Models

Virginia Fernandez, Pedro Sanchez, Walter Hugo Lopez Pinaya, Grzegorz Jacenków, Sotirios A. Tsaftaris, Jorge Cardoso

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

2306.01144 2023-06-05 cs.LG cs.CL 79%

Evaluating the Capabilities of Multi-modal Reasoning Models with Synthetic Task Data

Nathan Vaska, Victoria Helus

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CL

2305.02572 2023-06-01 cs.CV 79%

High-fidelity Generalized Emotional Talking Face Generation with Multi-modal Emotion Space Learning

Chao Xu, Junwei Zhu, Jiangning Zhang, Yue Han, Wenqing Chu, Ying Tai, Chengjie Wang, Zhifeng Xie, Yong Liu

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CV

2302.12764 2023-05-22 cs.CV 79%

Modulating Pretrained Diffusion Models for Multimodal Image Synthesis

Cusuh Ham, James Hays, Jingwan Lu, Krishna Kumar Singh, Zhifei Zhang, Tobias Hinz

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

Comments SIGGRAPH Conference Proceedings 2023. Project page at https://mcm-diffusion.github.io

2305.02594 2023-05-10 cs.CV 79%

Multimodal-driven Talking Face Generation via a Unified Diffusion-based Generator

Chao Xu, Shaoting Zhu, Junwei Zhu, Tianxin Huang, Jiangning Zhang, Ying Tai, Yong Liu

专题命中多模态生成：multimodal(title,abstract)；分类 cs.CV

2304.11697 2023-04-25 cs.CV eess.IV 79%

Informative Data Selection with Uncertainty for Multi-modal Object Detection

Xinyu Zhang, Zhiwei Li, Zhenhong Zou, Xin Gao, Yijin Xiong, Dafeng Jin, Jun Li, Huaping Liu

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CV

2304.10530 2023-04-21 cs.CV 79%

Collaborative Diffusion for Multi-Modal Face Generation and Editing

Ziqi Huang, Kelvin C. K. Chan, Yuming Jiang, Ziwei Liu

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CV

Comments CVPR 2023. Project page: https://ziqihuangg.github.io/projects/collaborative-diffusion.html Code: https://github.com/ziqihuangg/Collaborative-Diffusion

2304.09728 2023-04-21 cs.CV eess.IV 79%

Any-to-Any Style Transfer: Making Picasso and Da Vinci Collaborate

Songhua Liu, Jingwen Ye, Xinchao Wang

专题命中多模态生成：any-to-any(title,abstract)；分类 cs.CV

Comments Work in progress

2212.00793 2023-04-21 cs.CV 79%

Unite and Conquer: Plug & Play Multi-Modal Synthesis using Diffusion Models

Nithin Gopalakrishnan Nair, Wele Gedara Chaminda Bandara, Vishal M. Patel

专题命中多模态生成：multi-modal(title)；multimodal(abstract)；分类 cs.CV

Comments Accepted at CVPR 2023

2303.14081 2023-03-27 eess.IV cs.CV 79%

CoLa-Diff: Conditional Latent Diffusion Model for Multi-Modal MRI Synthesis

Lan Jiang, Ye Mao, Xi Chen, Xiangfeng Wang, Chao Li

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CV

Comments 8 pages

2212.04493 2023-03-23 cs.CV cs.LG 79%

SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generation

Yen-Chi Cheng, Hsin-Ying Lee, Sergey Tulyakov, Alexander Schwing, Liangyan Gui

专题命中多模态生成：multimodal(title)；multi-modal(abstract)；分类 cs.CV

Comments In CVPR 2023. Project page and code is available at: https://yccyenchicheng.github.io/SDFusion/. Fix some typos

2303.09319 2023-03-17 cs.CV 79%

Unified Multi-Modal Latent Diffusion for Joint Subject and Text Conditional Image Generation

Yiyang Ma, Huan Yang, Wenjing Wang, Jianlong Fu, Jiaying Liu

专题命中多模态生成：multi-modal(title,abstract)；分类 cs.CV

1. 多模态生成 4979 篇

AutoRepo: A general framework for multi-modal LLM-based automated construction reporting

Retrieving Multimodal Information for Augmented Generation: A Survey

BrainNetDiff: Generative AI Empowers Brain Network Generation via Multimodal Diffusion Model

An Implementation of Multimodal Fusion System for Intelligent Digital Human Generation

GenKIE: Robust Generative Multimodal Document Key Information Extraction

Enhancing Illicit Activity Detection using XAI: A Multimodal Graph-LLM Framework

C^2M-DoT: Cross-modal consistent multi-view medical report generation with domain transfer network

Highly Controllable Diffusion-based Any-to-Any Voice Conversion Model with Frame-level Prosody Feature

Graph-Based Interaction-Aware Multimodal 2D Vehicle Trajectory Prediction using Diffusion Graph Convolutional Networks

MagicAvatar: Multimodal Avatar Generation and Animation

Multimodal Image Synthesis and Editing: The Generative AI Era

DiffCloth: Diffusion Based Garment Synthesis and Manipulation via Structural Cross-modal Semantic Alignment

A Survey on Deep Multi-modal Learning for Body Language Recognition and Generation

TextPainter: Multimodal Text Image Generation with Visual-harmony and Text-comprehension for Poster Design

Multimodal Prompt Learning for Product Title Generation with Extremely Limited Labels

Consistent Multimodal Generation via A Unified GAN Framework

ZeroGen: Zero-shot Multimodal Controllable Text Generation with Multiple Oracles

Employing Multimodal Machine Learning for Stress Detection

Privacy Distillation: Reducing Re-identification Risk of Multimodal Diffusion Models

Evaluating the Capabilities of Multi-modal Reasoning Models with Synthetic Task Data

High-fidelity Generalized Emotional Talking Face Generation with Multi-modal Emotion Space Learning

Modulating Pretrained Diffusion Models for Multimodal Image Synthesis

Multimodal-driven Talking Face Generation via a Unified Diffusion-based Generator

Informative Data Selection with Uncertainty for Multi-modal Object Detection

Collaborative Diffusion for Multi-Modal Face Generation and Editing

Any-to-Any Style Transfer: Making Picasso and Da Vinci Collaborate

Unite and Conquer: Plug & Play Multi-Modal Synthesis using Diffusion Models

CoLa-Diff: Conditional Latent Diffusion Model for Multi-Modal MRI Synthesis

SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generation

Unified Multi-Modal Latent Diffusion for Joint Subject and Text Conditional Image Generation