多模态大模型 - arXivDaily 专题

2005.13061 2024-12-09 eess.IV cs.CV 79%

Prediction of Thrombectomy Functional Outcomes using Multimodal Data

Zeynel A. Samak, Philip Clatworthy, Majid Mirmehdi

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

Comments Accepted at Medical Image Understanding and Analysis (MIUA) 2020

2411.15459 2024-11-26 cs.CV 79%

MambaVLT: Time-Evolving Multimodal State Space Model for Vision-Language Tracking

Xinqi Liu, Li Zhou, Zikun Zhou, Jianqiu Chen, Zhenyu He

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

2411.14402 2024-11-22 cs.CV cs.LG 79%

Multimodal Autoregressive Pre-training of Large Vision Encoders

Enrico Fini, Mustafa Shukor, Xiujun Li, Philipp Dufter, Michal Klein, David Haldimann, Sai Aitharaju, Victor Guilherme Turrisi da Costa, Louis Béthune, Zhe Gan, Alexander T Toshev, Marcin Eichner, Moin Nabi, Yinfei Yang, Joshua M. Susskind, Alaaeldin El-Nouby

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

Comments https://github.com/apple/ml-aim

2411.13033 2024-11-21 eess.IV cs.CV 79%

LMM-driven Semantic Image-Text Coding for Ultra Low-bitrate Learned Image Compression

Shimon Murai, Heming Sun, Jiro Katto

专题命中图文多模态：image-text(title)；multi-modal(abstract)；分类 cs.CV

Comments IEEE VCIP 2024 poster

2404.19287 2024-11-13 cs.CV 79%

Revisiting the Adversarial Robustness of Vision Language Models: a Multimodal Perspective

Wanqi Zhou, Shuanghao Bai, Danilo P. Mandic, Qibin Zhao, Badong Chen

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

Comments 17 pages, 13 figures

2410.16261 2024-11-08 cs.CV 79%

Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performance

Zhangwei Gao, Zhe Chen, Erfei Cui, Yiming Ren, Weiyun Wang, Jinguo Zhu, Hao Tian, Shenglong Ye, Junjun He, Xizhou Zhu, Lewei Lu, Tong Lu, Yu Qiao, Jifeng Dai, Wenhai Wang

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

Comments Technical report

2405.17815 2024-11-05 cs.CV 79%

Visual Anchors Are Strong Information Aggregators For Multimodal Large Language Model

Haogeng Liu, Quanzeng You, Xiaotian Han, Yongfei Liu, Huaibo Huang, Ran He, Hongxia Yang

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

2410.23144 2024-10-31 cs.AI 79%

Public Domain 12M: A Highly Aesthetic Image-Text Dataset with Novel Governance Mechanisms

Jordan Meyer, Nick Padgett, Cullen Miller, Laura Exline

专题命中图文多模态：image-text(title,abstract)；分类 cs.AI

Comments Project Page: https://source.plus/pd12m

2309.12657 2024-10-28 cs.CV 79%

Exploiting Modality-Specific Features For Multi-Modal Manipulation Detection And Grounding

Jiazhen Wang, Bin Liu, Changtao Miao, Zhiwei Zhao, Wanyi Zhuang, Qi Chu, Nenghai Yu

专题命中图文多模态：multi-modal(title,abstract)；分类 cs.CV

Comments This work has been submitted to the IEEE for possible publication. Camera-ready version and supplementary material

2410.16163 2024-10-22 cs.CV 79%

Griffon-G: Bridging Vision-Language and Vision-Centric Tasks via Large Multimodal Models

Yufei Zhan, Hongyin Zhao, Yousong Zhu, Fan Yang, Ming Tang, Jinqiao Wang

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

Comments This work has been submitted to the IEEE for possible publication. Codes and data will be later released at https://github.com/jefferyZhan/Griffon

2406.10701 2024-10-15 cs.CL 79%

MIND: Multimodal Shopping Intention Distillation from Large Vision-language Models for E-commerce Purchase Understanding

Baixuan Xu, Weiqi Wang, Haochen Shi, Wenxuan Ding, Huihao Jing, Tianqing Fang, Jiaxin Bai, Xin Liu, Changlong Yu, Zheng Li, Chen Luo, Qingyu Yin, Bing Yin, Long Chen, Yangqiu Song

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CL

Comments EMNLP 2024 main conference

2212.07050 2024-10-14 cs.LG cs.CV eess.IV 79%

Significantly improving zero-shot X-ray pathology classification via fine-tuning pre-trained image-text encoders

Jongseong Jang, Daeun Kyung, Seung Hwan Kim, Honglak Lee, Kyunghoon Bae, Edward Choi

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV

Journal ref Sci Rep 14, 23199 (2024)

2410.07783 2024-10-11 cs.CV 79%

CLIP Multi-modal Hashing for Multimedia Retrieval

Jian Zhu, Mingkai Sheng, Zhangmin Huang, Jingfei Chang, Jinling Jiang, Jian Long, Cheng Luo, Lei Liu

专题命中图文多模态：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by 31st International Conference on MultiMedia Modeling (MMM2025)

2410.04426 2024-10-08 cs.CV 79%

CoVLM: Leveraging Consensus from Vision-Language Models for Semi-supervised Multi-modal Fake News Detection

Devank, Jayateja Kalla, Soma Biswas

专题命中图文多模态：multi-modal(title)；image-text(abstract)；分类 cs.CV

Comments Accepted in ACCV 2024

2410.00905 2024-10-02 cs.CV 79%

Removing Distributional Discrepancies in Captions Improves Image-Text Alignment

Yuheng Li, Haotian Liu, Mu Cai, Yijun Li, Eli Shechtman, Zhe Lin, Yong Jae Lee, Krishna Kumar Singh

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV

2409.18753 2024-09-30 cs.CV 79%

Enhancing Explainability in Multimodal Large Language Models Using Ontological Context

Jihen Amara, Birgitta König-Ries, Sheeba Samuel

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

2409.16261 2024-09-25 cs.CV 79%

CDChat: A Large Multimodal Model for Remote Sensing Change Description

Mubashir Noman, Noor Ahsan, Muzammal Naseer, Hisham Cholakkal, Rao Muhammad Anwer, Salman Khan, Fahad Shahbaz Khan

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

2409.13407 2024-09-23 cs.CV 79%

Instruction-guided Multi-Granularity Segmentation and Captioning with Large Multimodal Model

Li Zhou, Xu Yuan, Zenghui Sun, Zikun Zhou, Jingsong Lan

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

Comments Code and dataset will be released at https://github.com/lizhou-cs/mglmm. 7 pages, 4 figures with Supplementary Material

2409.07463 2024-09-13 cs.CV cs.LG 79%

Multi-Modal Instruction-Tuning Small-Scale Language-and-Vision Assistant for Semiconductor Electron Micrograph Analysis

Sakhinana Sagar Srinivas, Geethan Sannidhi, Venkataramana Runkana

专题命中图文多模态：multi-modal(title)；multimodal(abstract)；分类 cs.CV

Comments Paper published at AAAI 2024 Spring Symposium Series

2312.06726 2024-09-05 cs.CV 79%

Filter & Align: Leveraging Human Knowledge to Curate Image-Text Data

Lei Zhang, Fangxun Shu, Tianyang Liu, Sucheng Ren, Hao Jiang, Cihang Xie

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV

2408.05097 2024-08-12 cs.LG cs.AI 79%

Hyperbolic Learning with Multimodal Large Language Models

Paolo Mandica, Luca Franco, Konstantinos Kallidromitis, Suzanne Petryk, Fabio Galasso

专题命中图文多模态：multimodal(title)；multi-modal(abstract)；分类 cs.AI

Comments ECCV 2024 - Beyond Euclidean Workshop

2403.01988 2024-08-07 cs.CL 79%

FKA-Owl: Advancing Multimodal Fake News Detection through Knowledge-Augmented LVLMs

Xuannan Liu, Peipei Li, Huaibo Huang, Zekun Li, Xing Cui, Jiahao Liang, Lixiong Qin, Weihong Deng, Zhaofeng He

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CL

Comments Accepted by ACM MM 2024.Project page: https://liuxuannan.github.io/FKA_Owl.github.io/

2408.02718 2024-08-07 cs.CV 79%

MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models

Fanqing Meng, Jin Wang, Chuanhao Li, Quanfeng Lu, Hao Tian, Jiaqi Liao, Xizhou Zhu, Jifeng Dai, Yu Qiao, Ping Luo, Kaipeng Zhang, Wenqi Shao

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

Comments Project Page: https://mmiu-bench.github.io/

2408.02484 2024-08-06 cs.CV 79%

Exploring Conditional Multi-Modal Prompts for Zero-shot HOI Detection

Ting Lei, Shaofeng Yin, Yuxin Peng, Yang Liu

专题命中图文多模态：multi-modal(title,abstract)；分类 cs.CV

2406.14343 2024-07-23 cs.AI 79%

IWISDM: Assessing instruction following in multimodal models at scale

Xiaoxuan Lei, Lucas Gomez, Hao Yuan Bai, Pouya Bashivan

专题命中图文多模态：multimodal(title,abstract)；分类 cs.AI

2404.12467 2024-07-18 cs.CV cs.LG 79%

Towards Multi-modal Transformers in Federated Learning

Guangyu Sun, Matias Mendieta, Aritra Dutta, Xin Li, Chen Chen

专题命中图文多模态：multi-modal(title,abstract)；分类 cs.CV

Comments 2024 European Conference on Computer Vision (ECCV)

2407.08196 2024-07-12 cs.AI 79%

SoupLM: Model Integration in Large Language and Multi-Modal Models

Yue Bai, Zichen Zhang, Jiasen Lu, Yun Fu

专题命中图文多模态：multi-modal(title)；multimodal(abstract)；分类 cs.AI

2403.06407 2024-07-09 cs.CV 79%

Can LLMs' Tuning Methods Work in Medical Multimodal Domain?

Jiawei Chen, Yue Jiang, Dingkang Yang, Mingcheng Li, Jinjie Wei, Ziyun Qian, Lihua Zhang

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

Comments Accepted by MICCAI 2024

2407.02586 2024-07-04 cs.CV 79%

Improving Visual Storytelling with Multimodal Large Language Models

Xiaochuan Lin, Xiangyong Chen

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

Comments 10 pages

2406.17057 2024-06-26 astro-ph.IM astro-ph.GA cs.AI 79%

At First Sight: Zero-Shot Classification of Astronomical Images with Large Multimodal Models

Dimitrios Tanoglidis, Bhuvnesh Jain

专题命中图文多模态：multimodal(title,abstract)；分类 cs.AI

Comments 5 pages, 3 images. Prepared for submission to RNAAS

1. 图文多模态 4676 篇

Prediction of Thrombectomy Functional Outcomes using Multimodal Data

MambaVLT: Time-Evolving Multimodal State Space Model for Vision-Language Tracking

Multimodal Autoregressive Pre-training of Large Vision Encoders

LMM-driven Semantic Image-Text Coding for Ultra Low-bitrate Learned Image Compression

Revisiting the Adversarial Robustness of Vision Language Models: a Multimodal Perspective

Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performance

Visual Anchors Are Strong Information Aggregators For Multimodal Large Language Model

Public Domain 12M: A Highly Aesthetic Image-Text Dataset with Novel Governance Mechanisms

Exploiting Modality-Specific Features For Multi-Modal Manipulation Detection And Grounding

Griffon-G: Bridging Vision-Language and Vision-Centric Tasks via Large Multimodal Models

MIND: Multimodal Shopping Intention Distillation from Large Vision-language Models for E-commerce Purchase Understanding

Significantly improving zero-shot X-ray pathology classification via fine-tuning pre-trained image-text encoders

CLIP Multi-modal Hashing for Multimedia Retrieval

CoVLM: Leveraging Consensus from Vision-Language Models for Semi-supervised Multi-modal Fake News Detection

Removing Distributional Discrepancies in Captions Improves Image-Text Alignment

Enhancing Explainability in Multimodal Large Language Models Using Ontological Context

CDChat: A Large Multimodal Model for Remote Sensing Change Description

Instruction-guided Multi-Granularity Segmentation and Captioning with Large Multimodal Model

Multi-Modal Instruction-Tuning Small-Scale Language-and-Vision Assistant for Semiconductor Electron Micrograph Analysis

Filter & Align: Leveraging Human Knowledge to Curate Image-Text Data

Hyperbolic Learning with Multimodal Large Language Models

FKA-Owl: Advancing Multimodal Fake News Detection through Knowledge-Augmented LVLMs

MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models

Exploring Conditional Multi-Modal Prompts for Zero-shot HOI Detection

IWISDM: Assessing instruction following in multimodal models at scale

Towards Multi-modal Transformers in Federated Learning

SoupLM: Model Integration in Large Language and Multi-Modal Models

Can LLMs' Tuning Methods Work in Medical Multimodal Domain?

Improving Visual Storytelling with Multimodal Large Language Models

At First Sight: Zero-Shot Classification of Astronomical Images with Large Multimodal Models