多模态大模型

Towards Next-Generation Urban Decision Support Systems through AI-Powered Construction of Scientific Ontology using Large Language Models -- A Case in Optimizing Intermodal Freight Transportation

Jose Tupayachi, Haowen Xu, Olufemi A. Omitaomu, Mustafa Can Camur, Aliza Sharmin, Xueping Li

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.AI

Journal ref Smart Cities, 2024, 7(5), 2392-2421

A Survey for Foundation Models in Autonomous Driving

Haoxiang Gao, Zhongruo Wang, Yaqian Li, Kaiwen Long, Ming Yang, Yiqing Shen

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV

GoT-CQA: Graph-of-Thought Guided Compositional Reasoning for Chart Question Answering

Lingling Zhang, Muye Huang, QianYing Wang, Yaxian Wang, Wenjun Wu, Jun Liu

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV

DAP: Diffusion-based Affordance Prediction for Multi-modality Storage

Haonan Chang, Kowndinya Boyalakuntla, Yuhan Liu, Xinyu Zhang, Liam Schramm, Abdeslam Boularias

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV

Comments Paper Accepted by IROS2024. Arxiv version is 8 pages

CC-GPX: Extracting High-Quality Annotated Geospatial Data from Common Crawl

Ilya Ilyankou, Meihui Wang, Stefano Cavazzi, James Haworth

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CL

Comments Accepted as a poster to ACM SIGSPATIAL 2024

LLM4GEN: Leveraging Semantic Representation of LLMs for Text-to-Image Generation

Mushui Liu, Yuhang Ma, Yang Zhen, Jun Dan, Yunlong Yu, Zeng Zhao, Zhipeng Hu, Bai Liu, Changjie Fan

专题命中 多模态生成 ：image-text(abstract)；分类 cs.CV

Comments 11 pages, 13 figures

FaceCat: Enhancing Face Recognition Security with a Unified Diffusion Model

Jiawei Chen, Xiao Yang, Yinpeng Dong, Hang Su, Zhaoxia Yin

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV

Comments Under review

Latent Space Disentanglement in Diffusion Transformers Enables Zero-shot Fine-grained Semantic Editing

Zitao Shuai, Chenwei Wu, Zhengxu Tang, Bowen Song, Liyue Shen

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV

Collaborative Control for Geometry-Conditioned PBR Image Generation

Shimon Vainer, Mark Boss, Mathias Parger, Konstantin Kutsy, Dante De Nigris, Ciara Rowles, Nicolas Perony, Simon Donné

专题命中 多模态生成 ：cross-modal(abstract)；分类 cs.CV

Comments 19 pages, 10 figures; Project page: https://unity-research.github.io/holo-gen/

ZipGait: Bridging Skeleton and Silhouette with Diffusion Model for Advancing Gait Recognition

Fanxu Min, Qing Cai, Shaoxiang Guo, Yang Yu, Hao Fan, Junyu Dong

专题命中 多模态生成 ：cross-modal(abstract)；分类 cs.CV

Cross-View Meets Diffusion: Aerial Image Synthesis with Geometry and Text Guidance

Ahmad Arrabi, Xiaohan Zhang, Waqas Sultani, Chen Chen, Safwan Wshah

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV

The Tug-of-War Between Deepfake Generation and Detection

Hannah Lee, Changyeon Lee, Kevin Farhat, Lin Qiu, Steve Geluso, Aerin Kim, Oren Etzioni

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV

MetaEnzyme: Meta Pan-Enzyme Learning for Task-Adaptive Redesign

Jiangbin Zheng, Han Zhang, Qianqing Xu, An-Ping Zeng, Stan Z. Li

专题命中 多模态生成 ：cross-modal(abstract)；分类 cs.AI

Comments Accepted to ACM Multimedia 2024

Scene123: One Prompt to 3D Scene Generation via Video-Assisted and Consistency-Enhanced MAE

Yiying Yang, Fukun Yin, Jiayuan Fan, Xin Chen, Wanzhang Li, Gang Yu

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV

Comments arXiv admin note: text overlap with arXiv:2305.11588 by other authors

SeLoRA: Self-Expanding Low-Rank Adaptation of Latent Diffusion Model for Medical Image Synthesis

Yuchen Mao, Hongwei Li, Wei Pang, Giorgos Papanastasiou, Guang Yang, Chengjia Wang

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV

Comments Project Page: https://yuchen20.github.io/SeLoRA.github.io/

Building an Ethical and Trustworthy Biomedical AI Ecosystem for the Translational and Clinical Integration of Foundational Models

Simha Sankar Baradwaj, Destiny Gilliland, Jack Rincon, Henning Hermjakob, Yu Yan, Irsyad Adam, Gwyneth Lemaster, Dean Wang, Karol Watson, Alex Bui, Wei Wang, Peipei Ping

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.AI

Comments 3 figures, 3 tables

HAIFIT: Human-to-AI Fashion Image Translation

Jianan Jiang, Xinglin Li, Weiren Yu, Di Wu

专题命中 多模态生成 ：cross-modal(abstract)；分类 cs.CV

Comments 10 pages,8 figures

Context-Aware Indoor Point Cloud Object Generation through User Instructions

Yiyang Luo, Ke Lin, Chao Gu

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV

Comments Accepted by ACMMM'24

Attacks and Defenses for Generative Diffusion Models: A Comprehensive Survey

Vu Tuan Truong, Luan Ba Dang, Long Bao Le

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.AI

S^2Former-OR: Single-Stage Bi-Modal Transformer for Scene Graph Generation in OR

Jialun Pei, Diandian Guo, Jingyang Zhang, Manxi Lin, Yueming Jin, Pheng-Ann Heng

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV

Comments This work has been accepted by TMI2024

MDT-A2G: Exploring Masked Diffusion Transformers for Co-Speech Gesture Generation

Xiaofeng Mao, Zhengkai Jiang, Qilin Wang, Chencan Fu, Jiangning Zhang, Jiafu Wu, Yabiao Wang, Chengjie Wang, Wei Li, Mingmin Chi

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV

VAR-CLIP: Text-to-Image Generator with Visual Auto-Regressive Modeling

Qian Zhang, Xiangzi Dai, Ninghua Yang, Xiang An, Ziyong Feng, Xingyu Ren

专题命中 多模态生成 ：image-text(abstract)；分类 cs.CV

Comments total 10 pages, code:https://github.com/daixiangzi/VAR-CLIP

Illustrating Classic Brazilian Books using a Text-To-Image Diffusion Model

Felipe Mahlow, André Felipe Zanella, William Alberto Cruz Castañeda, Regilene Aparecida Sarzi-Ribeiro

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.AI

Comments 7 pages, 2 figures

Towards Reliable Advertising Image Generation Using Human Feedback

Zhenbang Du, Wei Feng, Haohan Wang, Yaoyu Li, Jingsen Wang, Jian Li, Zheng Zhang, Jingjing Lv, Xin Zhu, Junsheng Jin, Junjie Shen, Zhangang Lin, Jingping Shao

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV

Comments ECCV2024

Autonomous LLM-Enhanced Adversarial Attack for Text-to-Motion

Honglei Miao, Fan Ma, Ruijie Quan, Kun Zhan, Yi Yang

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV

Unveiling the Potential of AI for Nanomaterial Morphology Prediction

Ivan Dubrovsky, Andrei Dmitrenko, Aleksei Dmitrenko, Nikita Serov, Vladimir Vinogradov

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.AI

Journal ref Proceedings of the 41 st International Conference on Machine Learning. PMLR 235, 2024, 11957--11978

Robust Facial Reactions Generation: An Emotion-Aware Framework with Modality Compensation

Guanyu Hu, Jie Wei, Siyang Song, Dimitrios Kollias, Xinyu Yang, Zhonglin Sun, Odysseus Kaloidas

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV

Navigation Instruction Generation with BEV Perception and Large Language Models

Sheng Fan, Rui Liu, Wenguan Wang, Yi Yang

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV

Comments ECCV 2024; Project Page: https://github.com/FanScy/BEVInstructor

Text-based Talking Video Editing with Cascaded Conditional Diffusion

Bo Han, Heqing Zou, Haoyang Li, Guangcong Wang, Chng Eng Siong

专题命中 多模态生成 ：audio-visual(abstract)；分类 cs.CV

MedRAT: Unpaired Medical Report Generation via Auxiliary Tasks

Elad Hirsch, Gefen Dawidowicz, Ayellet Tal

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV