多模态大模型 - arXivDaily 专题

2504.14200 2025-07-31 cs.CV cs.AI 76%

Enhancing Multimodal In-Context Learning for Image Classification through Coreset Optimization

Huiyi Chen, Jiawei Peng, Kaihua Tang, Xin Geng, Xu Yang

机构 * Southeast University（东南大学）； Huawei Singapore Research Center（华为新加坡研究中心）

专题命中图文多模态：multimodal(title)；分类 cs.CV、cs.AI

Comments 11 pages, 5 figures

2506.17372 2025-06-24 cs.CY cs.AI cs.CV 76%

Multimodal Political Bias Identification and Neutralization

Cedric Bernard, Xavier Pleimling, Amun Kharel, Chase Vickery

专题命中图文多模态：multimodal(title)；分类 cs.CV、cs.AI

2406.09295 2025-06-05 cs.CL cs.CV 76%

AlignMMBench: Evaluating Chinese Multimodal Alignment in Large Vision-Language Models

Yuhang Wu, Wenmeng Yu, Yean Cheng, Yan Wang, Xiaohan Zhang, Jiazheng Xu, Ming Ding, Yuxiao Dong

机构 * Tsinghua University（清华大学）； Zhipu AI（智谱AI）； Peking University（北京大学）

专题命中图文多模态：multimodal(title)；分类 cs.CV、cs.CL

2506.02412 2025-06-04 cs.CL cs.AI 76%

SingaKids: A Multilingual Multimodal Dialogic Tutor for Language Learning

Zhengyuan Liu, Geyu Lin, Hui Li Tan, Huayun Zhang, Yanfeng Lu, Xiaoxue Gao, Stella Xin Yin, He Sun, Hock Huan Goh, Lung Hsiang Wong, Nancy F. Chen

机构 * Nanyang Technological University, Singapore（新加坡南洋理工大学）； National Institute of Education (NIE), Singapore（新加坡国家教育研究所）； Institute for Infocomm Research (I 2 R), A*STAR, Singapore（新加坡资讯与通信研究院（I2R））

专题命中图文多模态：multimodal(title)；分类 cs.CL、cs.AI

Comments ACL 2025 Industry Track

2506.02295 2025-06-04 cs.CV cs.AI 76%

QARI-OCR: High-Fidelity Arabic Text Recognition through Multimodal Large Language Model Adaptation

Ahmed Wasfy, Omer Nacar, Abdelakreem Elkhateb, Mahmoud Reda, Omar Elshehy, Adel Ammar, Wadii Boulila

机构 * NAMAA ； KAND CA Corp. ； Prince Sultan University

专题命中图文多模态：multimodal(title)；分类 cs.CV、cs.AI

2505.17529 2025-05-26 cs.CV cs.AI 76%

Do You Keep an Eye on What I Ask? Mitigating Multimodal Hallucination via Attention-Guided Ensemble Decoding

Yeongjae Cho, Keonwoo Kim, Taebaek Hwang, Sungzoon Cho

机构 * Seoul National University（首尔国立大学）； Kim & Chang AI&IT System Center（金·昌AI&IT系统中心）； Waddle Corporation（Waddle公司）

专题命中图文多模态：multimodal(title)；分类 cs.CV、cs.AI

2504.20220 2025-04-30 cs.CL cs.CV 76%

A Multimodal Pipeline for Clinical Data Extraction: Applying Vision-Language Models to Scans of Transfusion Reaction Reports

Henning Schäfer, Cynthia S. Schmidt, Johannes Wutzkowsky, Kamil Lorek, Lea Reinartz, Johannes Rückert, Christian Temme, Britta Böckmann, Peter A. Horn, Christoph M. Friedrich

机构 * Institute for Transfusion Medicine, University Hospital Essen, Hufelandstraße 55, Essen, Germany（1 输血医学研究所，埃森大学医院，Hufelandstraße 55，德国）； Department of Computer Science, University of Applied Sciences and Arts Dortmund (FHDO), Emil-Figge Str. 42, Dortmund, Germany（2 计算机科学系，多特蒙德应用科学大学（FHDO），Emil-Figge Str. 42，德国）； Institute for Medical Informatics, Biometry and Epidemiology (IMIBE), University Hospital Essen, Hufelandstraße 55, Essen, Germany（3 医学信息学、生物统计学和流行病学研究所（IMIBE），埃森大学医院，Hufelandstraße 55，德国）； Institute for AI in Medicine (IKIM), University Hospital Essen, Girardetstraße 2, Essen, Germany（4 医学人工智能研究所（IKIM），埃森大学医院，Girardetstraße 2，德国）； Institute of Interventional and Diagnostic Radiology and Neuroradiology, University Hospital Essen, Hufelandstraße 55, Essen, Germany（5 干预性和诊断放射学及神经放射学研究所，埃森大学医院，Hufelandstraße 55，德国）； Department of Dermatology, University Hospital Essen, Hufelandstraße 55, Essen, Germany（6 皮肤科系，埃森大学医院，Hufelandstraße 55，德国）

专题命中图文多模态：multimodal(title)；分类 cs.CV、cs.CL

2503.12490 2025-03-18 cs.CV cs.AI 76%

GeoRSMLLM: A Multimodal Large Language Model for Vision-Language Tasks in Geoscience and Remote Sensing

Zilun Zhang, Haozhan Shen, Tiancheng Zhao, Bin Chen, Zian Guan, Yuhao Wang, Xu Jia, Yuxiang Cai, Yongheng Shang, Jianwei Yin

专题命中图文多模态：multimodal(title)；分类 cs.CV、cs.AI

2502.06873 2025-02-12 cs.CL cs.AI 76%

Multimodal Cognitive Reframing Therapy via Multi-hop Psychotherapeutic Reasoning

Subin Kim, Hoonrae Kim, Heejin Do, Gary Geunbae Lee

专题命中图文多模态：multimodal(title)；分类 cs.CL、cs.AI

Comments NAACL 2025 Main

2407.03340 2025-02-03 cs.AI cs.CL cs.HC cs.LG cs.RO eess.IV 76%

A Multi-Modal Explainability Approach for Human-Aware Robots in Multi-Party Conversation

Iveta Bečková, Štefan Pócoš, Giulia Belgiovine, Marco Matarese, Omar Eldardeer, Alessandra Sciutti, Carlo Mazzola

专题命中图文多模态：multi-modal(title)；分类 cs.CL、cs.AI

Comments 32pp (+6pp sup.mat.) Accepted in Computer Vision and Image Understanding Journal on January 23, 2025. This research received funding Horizon-Europe TERAIS project (G.A. 101079338) and Slovak Research and Development Agency, project no. APVV-21-0105

2501.04155 2025-01-09 cs.CV cs.CL cs.LG 76%

MM-GEN: Enhancing Task Performance Through Targeted Multimodal Data Curation

Siddharth Joshi, Besmira Nushi, Vidhisha Balachandran, Varun Chandrasekaran, Vibhav Vineet, Neel Joshi, Baharan Mirzasoleiman

专题命中图文多模态：multimodal(title)；分类 cs.CV、cs.CL

2310.02692 2024-12-25 cs.CV cs.AI 76%

Clustering-based Image-Text Graph Matching for Domain Generalization

Nokyung Park, Daewon Chae, Jeongyong Shim, Sangpil Kim, Eun-Sol Kim, Jinkyu Kim

专题命中图文多模态：image-text(title)；分类 cs.CV、cs.AI

2304.10727 2024-12-23 cs.CV cs.AI 76%

RoCOCO: Robustness Benchmark of MS-COCO to Stress-test Image-Text Matching Models

Seulki Park, Daeho Um, Hajung Yoon, Sanghyuk Chun, Sangdoo Yun, Jin Young Choi

专题命中图文多模态：image-text(title)；分类 cs.CV、cs.AI

Comments Accepted to ECCV Synthetic Data for Computer Vision Workshop (Oral)

2410.21220 2024-10-29 cs.CV cs.AI cs.IR cs.LG 76%

Vision Search Assistant: Empower Vision-Language Models as Multimodal Search Engines

Zhixin Zhang, Yiyuan Zhang, Xiaohan Ding, Xiangyu Yue

专题命中图文多模态：multimodal(title)；分类 cs.CV、cs.AI

Comments Code is available at https://github.com/cnzzx/VSA

2410.07854 2024-10-11 cs.CV cs.MM 76%

HeGraphAdapter: Tuning Multi-Modal Vision-Language Models with Heterogeneous Graph Adapter

Yumiao Zhao, Bo Jiang, Xiao Wang, Qin Xu, Jin Tang

专题命中图文多模态：multi-modal(title)；分类 cs.CV、cs.MM

2409.19474 2024-10-08 cs.CV cs.AI 76%

FairPIVARA: Reducing and Assessing Biases in CLIP-Based Multimodal Models

Diego A. B. Moreira, Alef Iury Ferreira, Jhessica Silva, Gabriel Oliveira dos Santos, Luiz Pereira, João Medrado Gondim, Gustavo Bonil, Helena Maia, Nádia da Silva, Simone Tiemi Hashiguti, Jefersson A. dos Santos, Helio Pedrini, Sandra Avila

专题命中图文多模态：multimodal(title)；分类 cs.CV、cs.AI

Comments 14 pages, 10 figures. Accepted to 35th British Machine Vision Conference (BMVC 2024), Workshop on Privacy, Fairness, Accountability and Transparency in Computer Vision

2404.03022 2024-06-13 cs.CL cs.CV cs.IT cs.LG math.IT 76%

BCAmirs at SemEval-2024 Task 4: Beyond Words: A Multimodal and Multilingual Exploration of Persuasion in Memes

Amirhossein Abaskohi, Amirhossein Dabiriaghdam, Lele Wang, Giuseppe Carenini

专题命中图文多模态：multimodal(title)；分类 cs.CV、cs.CL

Comments 12 pages, 5 tables, 2 figures, Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024) @ NAACL 2024

2310.20225 2024-04-30 cs.CV cs.AI 76%

A Multi-Modal Foundation Model to Assist People with Blindness and Low Vision in Environmental Interaction

Yu Hao, Fan Yang, Hao Huang, Shuaihang Yuan, Sundeep Rangan, John-Ross Rizzo, Yao Wang, Yi Fang

专题命中图文多模态：multi-modal(title)；分类 cs.CV、cs.AI

2401.04575 2024-03-07 cs.CV cs.AI 76%

Let's Go Shopping (LGS) -- Web-Scale Image-Text Dataset for Visual Concept Understanding

Yatong Bai, Utsav Garg, Apaar Shanker, Haoming Zhang, Samyak Parajuli, Erhan Bas, Isidora Filipovic, Amelia N. Chu, Eugenia D Fomitcheva, Elliot Branson, Aerin Kim, Somayeh Sojoudi, Kyunghyun Cho

专题命中图文多模态：image-text(title)；分类 cs.CV、cs.AI

2401.08025 2024-02-23 cs.AI cs.CL cs.LG 76%

Self-Imagine: Effective Unimodal Reasoning with Multimodal Models using Self-Imagination

Syeda Nahida Akter, Aman Madaan, Sangwu Lee, Yiming Yang, Eric Nyberg

专题命中图文多模态：multimodal(title)；分类 cs.CL、cs.AI

Comments 18 pages, 9 figures, 12 tables

2401.06167 2024-01-15 cs.CV cs.AI 76%

Enhancing Multimodal Understanding with CLIP-Based Image-to-Text Transformation

Chang Che, Qunwei Lin, Xinyu Zhao, Jiaxin Huang, Liqiang Yu

专题命中图文多模态：multimodal(title)；分类 cs.CV、cs.AI

2307.15460 2023-07-31 cs.CV cs.CL 76%

Cross-Modal Concept Learning and Inference for Vision-Language Models

Yi Zhang, Ce Zhang, Yushun Tang, Zhihai He

专题命中图文多模态：cross-modal(title)；分类 cs.CV、cs.CL

2208.05140 2023-04-13 eess.IV cs.CL cs.CV cs.LG 76%

Self-supervised Multi-modal Training from Uncurated Image and Reports Enables Zero-shot Oversight Artificial Intelligence in Radiology

Sangjoon Park, Eun Sun Lee, Kyung Sook Shin, Jeong Eun Lee, Jong Chul Ye

专题命中图文多模态：multi-modal(title)；分类 cs.CV、cs.CL

2212.09621 2022-12-20 cs.CL cs.CV 76%

Wukong-Reader: Multi-modal Pre-training for Fine-grained Visual Document Understanding

Haoli Bai, Zhiguang Liu, Xiaojun Meng, Wentao Li, Shuang Liu, Nian Xie, Rongfu Zheng, Liangwei Wang, Lu Hou, Jiansheng Wei, Xin Jiang, Qun Liu

专题命中图文多模态：multi-modal(title)；分类 cs.CV、cs.CL

2211.04971 2022-11-11 cs.CL cs.CV 76%

Understanding Cross-modal Interactions in V&L Models that Generate Scene Descriptions

Michele Cafagna, Kees van Deemter, Albert Gatt

专题命中图文多模态：cross-modal(title)；分类 cs.CV、cs.CL

2112.03562 2021-12-10 cs.CV cs.CL cs.LG 76%

CMA-CLIP: Cross-Modality Attention CLIP for Image-Text Classification

Huidong Liu, Shaoyuan Xu, Jinmiao Fu, Yang Liu, Ning Xie, Chien-Chih Wang, Bryan Wang, Yi Sun

专题命中图文多模态：image-text(title)；分类 cs.CV、cs.CL

Comments 9 pages, 2 figures, 6 tables, 1 algorithm

1711.05535 2021-07-28 cs.CV cs.MM 76%

Dual-Path Convolutional Image-Text Embeddings with Instance Loss

Zhedong Zheng, Liang Zheng, Michael Garrett, Yi Yang, Mingliang Xu, Yi-Dong Shen

专题命中图文多模态：image-text(title)；分类 cs.CV、cs.MM

Comments 15pages, 15 figures, 8 tables

1907.07804 2020-07-06 cs.LG cs.CL cs.CV cs.NE stat.ML 76%

OmniNet: A unified architecture for multi-modal multi-task learning

Subhojeet Pramanik, Priyanka Agrawal, Aman Hussain

专题命中图文多模态：multi-modal(title)；分类 cs.CV、cs.CL

Comments Source code available at: https://github.com/subho406/OmniNet

1806.02934 2018-06-11 stat.ML cs.CL cs.CV cs.LG 76%

Learn from Your Neighbor: Learning Multi-modal Mappings from Sparse Annotations

Ashwin Kalyan, Stefan Lee, Anitha Kannan, Dhruv Batra

专题命中图文多模态：multi-modal(title)；分类 cs.CV、cs.CL

Comments To be presented at ICML 2018; 10 pages 5 figures

1611.08481 2017-02-08 cs.AI cs.CV 76%

GuessWhat?! Visual object discovery through multi-modal dialogue

Harm de Vries, Florian Strub, Sarath Chandar, Olivier Pietquin, Hugo Larochelle, Aaron Courville

专题命中图文多模态：multi-modal(title)；分类 cs.CV、cs.AI

Comments 23 pages; CVPR 2017 submission; see https://guesswhat.ai

1. 图文多模态 4657 篇

Enhancing Multimodal In-Context Learning for Image Classification through Coreset Optimization

Multimodal Political Bias Identification and Neutralization

AlignMMBench: Evaluating Chinese Multimodal Alignment in Large Vision-Language Models

SingaKids: A Multilingual Multimodal Dialogic Tutor for Language Learning

QARI-OCR: High-Fidelity Arabic Text Recognition through Multimodal Large Language Model Adaptation

Do You Keep an Eye on What I Ask? Mitigating Multimodal Hallucination via Attention-Guided Ensemble Decoding

A Multimodal Pipeline for Clinical Data Extraction: Applying Vision-Language Models to Scans of Transfusion Reaction Reports

GeoRSMLLM: A Multimodal Large Language Model for Vision-Language Tasks in Geoscience and Remote Sensing

Multimodal Cognitive Reframing Therapy via Multi-hop Psychotherapeutic Reasoning

A Multi-Modal Explainability Approach for Human-Aware Robots in Multi-Party Conversation

MM-GEN: Enhancing Task Performance Through Targeted Multimodal Data Curation

Clustering-based Image-Text Graph Matching for Domain Generalization

RoCOCO: Robustness Benchmark of MS-COCO to Stress-test Image-Text Matching Models

Vision Search Assistant: Empower Vision-Language Models as Multimodal Search Engines

HeGraphAdapter: Tuning Multi-Modal Vision-Language Models with Heterogeneous Graph Adapter

FairPIVARA: Reducing and Assessing Biases in CLIP-Based Multimodal Models

BCAmirs at SemEval-2024 Task 4: Beyond Words: A Multimodal and Multilingual Exploration of Persuasion in Memes

A Multi-Modal Foundation Model to Assist People with Blindness and Low Vision in Environmental Interaction

Let's Go Shopping (LGS) -- Web-Scale Image-Text Dataset for Visual Concept Understanding

Self-Imagine: Effective Unimodal Reasoning with Multimodal Models using Self-Imagination

Enhancing Multimodal Understanding with CLIP-Based Image-to-Text Transformation

Cross-Modal Concept Learning and Inference for Vision-Language Models

Self-supervised Multi-modal Training from Uncurated Image and Reports Enables Zero-shot Oversight Artificial Intelligence in Radiology

Wukong-Reader: Multi-modal Pre-training for Fine-grained Visual Document Understanding

Understanding Cross-modal Interactions in V&L Models that Generate Scene Descriptions

CMA-CLIP: Cross-Modality Attention CLIP for Image-Text Classification

Dual-Path Convolutional Image-Text Embeddings with Instance Loss

OmniNet: A unified architecture for multi-modal multi-task learning

Learn from Your Neighbor: Learning Multi-modal Mappings from Sparse Annotations

GuessWhat?! Visual object discovery through multi-modal dialogue