多模态大模型 - arXivDaily 专题

2412.18826 2024-12-30 cs.CL 83%

RapGuard: Safeguarding Multimodal Large Language Models via Rationale-aware Defensive Prompting

Yilei Jiang, Yingshui Tan, Xiangyu Yue

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL

2412.16869 2024-12-24 cs.CV 83%

CoF: Coarse to Fine-Grained Image Understanding for Multi-modal Large Language Models

Yeyuan Wang, Dehong Gao, Bin Li, Rujiao Long, Lei Yi, Xiaoyan Cai, Libin Yang, Jinxia Zhang, Shanqing Yu, Qi Xuan

专题命中图文多模态：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments 5 pages, Accepted by ICASSP2025, full paper

2412.11124 2024-12-24 cs.CV 83%

Combating Multimodal LLM Hallucination via Bottom-Up Holistic Reasoning

Shengqiong Wu, Hao Fei, Liangming Pan, William Yang Wang, Shuicheng Yan, Tat-Seng Chua

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments 16 pages, 10 figures, accepted by AAAI 25

2412.15813 2024-12-23 cs.CV 83%

Cross-Modal Few-Shot Learning with Second-Order Neural Ordinary Differential Equations

Yi Zhang, Chun-Wun Cheng, Junyi He, Zhihai He, Carola-Bibiane Schönlieb, Yuyan Chen, Angelica I Aviles-Rivero

专题命中图文多模态：cross-modal(title,abstract)；image-text(abstract)；分类 cs.CV

2411.01756 2024-12-17 cs.CV 83%

ChatTracker: Enhancing Visual Tracking Performance via Chatting with Multimodal Large Language Model

Yiming Sun, Fan Yu, Shaoxiang Chen, Yu Zhang, Junwei Huang, Chenhui Li, Yang Li, Changbo Wang

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

2412.10372 2024-12-16 cs.CV 83%

UniMed-CLIP: Towards a Unified Image-Text Pretraining Paradigm for Diverse Medical Imaging Modalities

Muhammad Uzair Khattak, Shahina Kunhimon, Muzammal Naseer, Salman Khan, Fahad Shahbaz Khan

专题命中图文多模态：image-text(title,abstract)；multi-modal(abstract)；分类 cs.CV

Comments Code, models and demo available at https://github.com/mbzuai-oryx/UniMed-CLIP

2412.09616 2024-12-16 cs.CV 83%

V2PE: Improving Multimodal Long-Context Capability of Vision-Language Models with Variable Visual Position Encoding

Junqi Ge, Ziyi Chen, Jintao Lin, Jinguo Zhu, Xihui Liu, Jifeng Dai, Xizhou Zhu

专题命中图文多模态：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV

Comments The code and models will be available at https://github.com/OpenGVLab/V2PE

2412.07119 2024-12-11 cs.CV 83%

DiffCLIP: Few-shot Language-driven Multimodal Classifier

Jiaqing Zhang, Mingxiang Cao, Xue Yang, Kai Jiang, Yunsong Li

专题命中图文多模态：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV

2412.03665 2024-12-06 cs.CV cs.AI cs.CL cs.MM 83%

Personalizing Multimodal Large Language Models for Image Captioning: An Experimental Analysis

Davide Bucciarelli, Nicholas Moratelli, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments ECCV 2024 Workshop on Green Foundation Models

2406.02884 2024-11-27 cs.CV 83%

PosterLLaVa: Constructing a Unified Multi-modal Layout Generator with LLM

Tao Yang, Yingmin Luo, Zhongang Qi, Yang Wu, Ying Shan, Chang Wen Chen

专题命中图文多模态：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments 13 pages; with PosterGen as extension; IEEE template

2411.09301 2024-11-15 cs.CV 83%

LHRS-Bot-Nova: Improved Multimodal Large Language Model for Remote Sensing Vision-Language Interpretation

Zhenshi Li, Dilxat Muhtar, Feng Gu, Xueliang Zhang, Pengfeng Xiao, Guangjun He, Xiaoxiang Zhu

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

2411.05023 2024-11-11 cs.CL cs.LG quant-ph 83%

Multimodal Quantum Natural Language Processing: A Novel Framework for using Quantum Methods to Analyse Real Data

Hala Hawashin

专题命中图文多模态：multimodal(title,abstract)；image-text(abstract)；分类 cs.CL

Comments This thesis, awarded a distinction by the Department of Computer Science at University College London, was successfully defended by the author in September 2024 in partial fulfillment of the requirements for an MSc in Emerging Digital Technologies

2404.13046 2024-11-01 cs.CV 83%

MoVA: Adapting Mixture of Vision Experts to Multimodal Context

Zhuofan Zong, Bingqi Ma, Dazhong Shen, Guanglu Song, Hao Shao, Dongzhi Jiang, Hongsheng Li, Yu Liu

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments NeurIPS 2024

2410.22736 2024-10-31 cs.CL 83%

Constructing Multimodal Datasets from Scratch for Rapid Development of a Japanese Visual Language Model

Keito Sasagawa, Koki Maeda, Issa Sugiura, Shuhei Kurita, Naoaki Okazaki, Daisuke Kawahara

专题命中图文多模态：multimodal(title,abstract)；image-text(abstract)；分类 cs.CL

Comments 15 pages, 7 figures

2403.08773 2024-10-29 cs.CV cs.AI cs.CL cs.MM 83%

Veagle: Advancements in Multimodal Representation Learning

Rajat Chawla, Arkajit Datta, Tushar Verma, Adarsh Jha, Anmol Gautam, Ayush Vatsal, Sukrit Chaterjee, Mukunda NS, Ishaan Bhola

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

2310.13120 2024-10-17 cs.CV cs.LG 83%

RSAdapter: Adapting Multimodal Models for Remote Sensing Visual Question Answering

Yuduo Wang, Pedram Ghamisi

专题命中图文多模态：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV

Journal ref IEEE Trans. Geosci. Remote Sens., vol. 62, pp. 1-13, 2024

2410.11829 2024-10-16 cs.CV 83%

MMFuser: Multimodal Multi-Layer Feature Fuser for Fine-Grained Vision-Language Understanding

Yue Cao, Yangzhou Liu, Zhe Chen, Guangchen Shi, Wenhai Wang, Danhuai Zhao, Tong Lu

专题命中图文多模态：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments 11 pages, 6 figures, technical report

2302.06891 2024-10-01 cs.CV 83%

UKnow: A Unified Knowledge Protocol with Multimodal Knowledge Graph Datasets for Reasoning and Vision-Language Pre-Training

Biao Gong, Shuai Tan, Yutong Feng, Xiaoying Xie, Yuyuan Li, Chaochao Chen, Kecheng Zheng, Yujun Shen, Deli Zhao

专题命中图文多模态：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV

2409.15310 2024-09-25 cs.LG cs.CV 83%

Visual Prompting in Multimodal Large Language Models: A Survey

Junda Wu, Zhehao Zhang, Yu Xia, Xintong Li, Zhaoyang Xia, Aaron Chang, Tong Yu, Sungchul Kim, Ryan A. Rossi, Ruiyi Zhang, Subrata Mitra, Dimitris N. Metaxas, Lina Yao, Jingbo Shang, Julian McAuley

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments 10 pages

2409.03412 2024-09-06 cs.CV physics.med-ph 83%

TG-LMM: Enhancing Medical Image Segmentation Accuracy through Text-Guided Large Multi-Modal Model

Yihao Zhao, Enhao Zhong, Cuiyun Yuan, Yang Li, Man Zhao, Chunxia Li, Jun Hu, Chenbin Liu

专题命中图文多模态：multi-modal(title,abstract)；image-text(abstract)；分类 cs.CV

Comments 11 pages, 2 figures

2009.08395 2024-09-05 cs.CV cs.AI cs.CL cs.LG cs.MM 83%

A Multimodal Memes Classification: A Survey and Open Research Issues

Tariq Habib Afridi, Aftab Alam, Muhammad Numan Khan, Jawad Khan, Young-Koo Lee

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments This is a survey paper on recent state of the art VL models that can be used for memes classification. it has 15 pages and 2 figures

Journal ref SCA 2020

2408.15556 2024-08-29 cs.CV 83%

Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models

Wenbin Wang, Liang Ding, Minyan Zeng, Xiabin Zhou, Li Shen, Yong Luo, Dacheng Tao

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

2408.14594 2024-08-28 cs.CV 83%

MMR: Evaluating Reading Ability of Large Multimodal Models

Jian Chen, Ruiyi Zhang, Yufan Zhou, Ryan Rossi, Jiuxiang Gu, Changyou Chen

专题命中图文多模态：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CV

2407.18626 2024-07-29 cs.CL cs.AI cs.CV cs.DL cs.MM 83%

Every Part Matters: Integrity Verification of Scientific Figures Based on Multimodal Large Language Models

Xiang Shi, Jiawei Liu, Yinpeng Liu, Qikai Cheng, Wei Lu

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 28 pages, 11 figures, under review

2407.12709 2024-07-18 cs.CV 83%

MoME: Mixture of Multimodal Experts for Generalist Multimodal Large Language Models

Leyang Shen, Gongwei Chen, Rui Shao, Weili Guan, Liqiang Nie

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments Github: https://github.com/JiuTian-VL/MoME

2403.00219 2024-07-12 cs.CV 83%

Multi-modal Attribute Prompting for Vision-Language Models

Xin Liu, Jiamin Wu, and Wenfei Yang, Xu Zhou, Tianzhu Zhang

专题命中图文多模态：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted for Publication in IEEE Transactions on Circuits and Systems for Video Technology (TCSVT)

2407.01884 2024-07-03 cs.CV cs.HC 83%

EIT-1M: One Million EEG-Image-Text Pairs for Human Visual-textual Recognition and More

Xu Zheng, Ling Wang, Kanghao Chen, Yuanhuiyi Lyu, Jiazhou Zhou, Lin Wang

专题命中图文多模态：image-text(title,abstract)；multi-modal(abstract)；分类 cs.CV

2407.00203 2024-07-02 cs.CV 83%

PathGen-1.6M: 1.6 Million Pathology Image-text Pairs Generation through Multi-agent Collaboration

Yuxuan Sun, Yunlong Zhang, Yixuan Si, Chenglu Zhu, Zhongyi Shui, Kai Zhang, Jingxiong Li, Xingheng Lyu, Tao Lin, Lin Yang

专题命中图文多模态：image-text(title,abstract)；multimodal(abstract)；分类 cs.CV

Comments 13 pages, 3 figures

2406.18579 2024-06-28 cs.CV cs.IR 83%

Hire: Hybrid-modal Interaction with Multiple Relational Enhancements for Image-Text Matching

Xuri Ge, Fuhai Chen, Songpei Xu, Fuxiang Tao, Jie Wang, Joemon M. Jose

专题命中图文多模态：image-text(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments 22pages, 5 Figures, 6 tables, the extension of CMSEI in WACV23, and submitted to ACM TIST. arXiv admin note: text overlap with arXiv:2210.08908

2405.10954 2024-05-21 cs.CV 83%

Multimodal CLIP Inference for Meta-Few-Shot Image Classification

Constance Ferragu, Philomene Chagniot, Vincent Coyette

专题命中图文多模态：multimodal(title,abstract)；multimodal foundation model(abstract)；分类 cs.CV

1. 图文多模态 4682 篇

RapGuard: Safeguarding Multimodal Large Language Models via Rationale-aware Defensive Prompting

CoF: Coarse to Fine-Grained Image Understanding for Multi-modal Large Language Models

Combating Multimodal LLM Hallucination via Bottom-Up Holistic Reasoning

Cross-Modal Few-Shot Learning with Second-Order Neural Ordinary Differential Equations

ChatTracker: Enhancing Visual Tracking Performance via Chatting with Multimodal Large Language Model

UniMed-CLIP: Towards a Unified Image-Text Pretraining Paradigm for Diverse Medical Imaging Modalities

V2PE: Improving Multimodal Long-Context Capability of Vision-Language Models with Variable Visual Position Encoding

DiffCLIP: Few-shot Language-driven Multimodal Classifier

Personalizing Multimodal Large Language Models for Image Captioning: An Experimental Analysis

PosterLLaVa: Constructing a Unified Multi-modal Layout Generator with LLM

LHRS-Bot-Nova: Improved Multimodal Large Language Model for Remote Sensing Vision-Language Interpretation

Multimodal Quantum Natural Language Processing: A Novel Framework for using Quantum Methods to Analyse Real Data

MoVA: Adapting Mixture of Vision Experts to Multimodal Context

Constructing Multimodal Datasets from Scratch for Rapid Development of a Japanese Visual Language Model

Veagle: Advancements in Multimodal Representation Learning

RSAdapter: Adapting Multimodal Models for Remote Sensing Visual Question Answering

MMFuser: Multimodal Multi-Layer Feature Fuser for Fine-Grained Vision-Language Understanding

UKnow: A Unified Knowledge Protocol with Multimodal Knowledge Graph Datasets for Reasoning and Vision-Language Pre-Training

Visual Prompting in Multimodal Large Language Models: A Survey

TG-LMM: Enhancing Medical Image Segmentation Accuracy through Text-Guided Large Multi-Modal Model

A Multimodal Memes Classification: A Survey and Open Research Issues

Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models

MMR: Evaluating Reading Ability of Large Multimodal Models

Every Part Matters: Integrity Verification of Scientific Figures Based on Multimodal Large Language Models

MoME: Mixture of Multimodal Experts for Generalist Multimodal Large Language Models

Multi-modal Attribute Prompting for Vision-Language Models

EIT-1M: One Million EEG-Image-Text Pairs for Human Visual-textual Recognition and More

PathGen-1.6M: 1.6 Million Pathology Image-text Pairs Generation through Multi-agent Collaboration

Hire: Hybrid-modal Interaction with Multiple Relational Enhancements for Image-Text Matching

Multimodal CLIP Inference for Meta-Few-Shot Image Classification