多模态大模型

Understand, Think, and Answer: Advancing Visual Reasoning with Large Multimodal Models

Yufei Zhan, Hongyin Zhao, Yousong Zhu, Shurong Zheng, Fan Yang, Ming Tang, Jinqiao Wang

机构 * Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所基础模型研究中心） ； School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院） ； Peng Cheng Laboratory, Shenzhen, China（鹏城实验室） ； Wuhan AI Research, Wuhan, China（武汉人工智能研究所） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Tech report

Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generation

Yue Yang, Ajay Patel, Matt Deitke, Tanmay Gupta, Luca Weihs, Andrew Head, Mark Yatskar, Chris Callison-Burch, Ranjay Krishna, Aniruddha Kembhavi, Christopher Clark

机构 * University of Pennsylvania（宾夕法尼亚大学） ； Allen Institute for Artificial Intelligence（人工智能研究所） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Published in ACL 2025, project page: https://yueyang1996.github.io/cosyn/

Debating for Better Reasoning: An Unsupervised Multimodal Approach

Ashutosh Adhikari, Mirella Lapata

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

MASSV: Multimodal Adaptation and Self-Data Distillation for Speculative Decoding of Vision-Language Models

Mugilan Ganesan, Shane Segal, Ankur Aggarwal, Nish Sinnadurai, Sean Lie, Vithursan Thangarasa

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Main paper: 11 pages, 4 figures, 3 tables. Supplementary: 1 page

Vision-Language Models Create Cross-Modal Task Representations

Grace Luo, Trevor Darrell, Amir Bar

机构 * University of California, Berkeley, USA（加州大学伯克利分校） 纠错

专题命中 图文多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.CL

Comments ICML 2025

CLIP-KOA: Enhancing Knee Osteoarthritis Diagnosis with Multi-Modal Learning and Symmetry-Aware Loss Functions

Yejin Jeong, Donghun Lee

机构 * Department of Mathematics, Korea University（数学系，韩国大学） 纠错

专题命中 图文多模态 ：multi-modal(title)；multimodal(abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 2 figures

DeepSPG: Exploring Deep Semantic Prior Guidance for Low-light Image Enhancement with Multimodal Learning

Jialang Lu, Huayu Zhao, Huiyu Zhai, Xingxing Yang, Shini Han

机构 * School of Cyber Science and Technology, Hubei University（湖北大学计算机科学与技术学院） ； Department of Electrical Automation Design, Beijing Shougang International Engineering Technology（北京首钢国际工程技术部） ； School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院） ； Department of Computer Science, Hong Kong Baptist University（香港 Baptist 大学计算机科学部） ； School of Computer Science and Technology, Harbin University of Science and Technology（哈尔滨理工大学计算机科学与技术学院） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

Comments Accepted by ICMR 2025 Main track. Code is available at https://github.com/Wenyuzhy/DeepSPG

FashionM3: Multimodal, Multitask, and Multiround Fashion Assistant based on Unified Vision-Language Model

Kaicheng Pang, Xingxing Zou, Waikeung Wong

机构 * Laboratory for Artificial Intelligence in Design（人工智能设计实验室） ； School of Fashion and Textiles（时尚与纺织学院） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning

Di Zhang, Junxian Li, Jingdi Lei, Xunzhi Wang, Yujie Liu, Zonglin Yang, Jiatong Li, Weida Wang, Suorong Yang, Jianbo Wu, Peng Ye, Wanli Ouyang, Dongzhan Zhou

机构 * Fudan University（复旦大学） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； Shanghai Jiaotong University（上海交通大学） ； Nankai University（南开大学） ； Shanghai University（上海大学） ； Nanyang Technological University（南洋理工大学） ； Hong Kong Polytechnic University（香港理工大学） ； Tongji University（同济大学） ； Nanjing University（南京大学） ； University of California, Merced（加州大学梅德福分校） ； Chinese University of Hong Kong（香港中文大学） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments 16 pages, 11 figures

Chain-of-Modality: Learning Manipulation Programs from Multimodal Human Videos with Vision-Language-Models

Chen Wang, Fei Xia, Wenhao Yu, Tingnan Zhang, Ruohan Zhang, C. Karen Liu, Li Fei-Fei, Jie Tan, Jacky Liang

机构 * Google DeepMind（谷歌DeepMind） ； Stanford University（斯坦福大学） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.AI、cs.MM

Comments ICRA 2025

Why We Feel: Breaking Boundaries in Emotional Reasoning with Multimodal Large Language Models

Yuxiang Lin, Jingdong Sun, Zhi-Qi Cheng, Jue Wang, Haomin Liang, Zebang Cheng, Yifei Dong, Jun-Yan He, Xiaojiang Peng, Xian-Sheng Hua

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.AI、cs.MM

Comments Accepted at CVPR Workshop NEXD 2025. 21 pages, Project: https://github.com/Lum1104/EIBench

Large Visual-Language Models Are Also Good Classifiers: A Study of In-Context Multimodal Fake News Detection

Ye Jiang, Yimin Wang

专题命中 图文多模态 ：multimodal(title)；cross-modal(abstract)；分类 cs.CL、cs.AI

Comments Withdraw for new experiments

CollEX -- A Multimodal Agentic RAG System Enabling Interactive Exploration of Scientific Collections

Florian Schneider, Narges Baba Ahmadi, Niloufar Baba Ahmadi, Iris Vogel, Martin Semmann, Chris Biemann

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Lumina-OmniLV: A Unified Multimodal Framework for General Low-Level Vision

Yuandong Pu, Le Zhuo, Kaiwen Zhu, Liangbin Xie, Wenlong Zhang, Xiangyu Chen, Peng Gao, Yu Qiao, Chao Dong, Yihao Liu

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

SmolVLM: Redefining small and efficient multimodal models

Andrés Marafioti, Orr Zohar, Miquel Farré, Merve Noyan, Elie Bakouch, Pedro Cuenca, Cyril Zakka, Loubna Ben Allal, Anton Lozhkov, Nouamane Tazi, Vaibhav Srivastav, Joshua Lochner, Hugo Larcher, Mathieu Morlon, Lewis Tunstall, Leandro von Werra, Thomas Wolf

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Vision-Language Models for Acute Tuberculosis Diagnosis: A Multimodal Approach Combining Imaging and Clinical Data

Ananya Ganapthy, Praveen Shastry, Naveen Kumarasami, Anandakumar D, Keerthana R, Mounigasri M, Varshinipriya M, Kishore Prasath Venkatesh, Bargava Subramanian, Kalyan Sivasailam

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 11 pages, 3 figures

HaploVL: A Single-Transformer Baseline for Multi-Modal Understanding

Rui Yang, Lin Song, Yicheng Xiao, Runhui Huang, Yixiao Ge, Ying Shan, Hengshuang Zhao

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

BREEN: Bridge Data-Efficient Encoder-Free Multimodal Learning with Learnable Queries

Tianle Li, Yongming Rao, Winston Hu, Yu Cheng

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Integrating Chain-of-Thought for Multimodal Alignment: A Study on 3D Vision-Language Learning

Yanjun Chen, Yirong Sun, Xinghao Chen, Jian Wang, Xiaoyu Shen, Wenjie Li, Wei Zhang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

PathM3: A Multimodal Multi-Task Multiple Instance Learning Framework for Whole Slide Image Classification and Captioning

Qifeng Zhou, Wenliang Zhong, Yuzhi Guo, Michael Xiao, Hehuan Ma, Junzhou Huang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

PerturboLLaVA: Reducing Multimodal Hallucinations with Perturbative Visual Training

Cong Chen, Mingyu Liu, Chenchen Jing, Yizhou Zhou, Fengyun Rao, Hao Chen, Bo Zhang, Chunhua Shen

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

CoMT: A Novel Benchmark for Chain of Multi-modal Thought on Large Vision-Language Models

Zihui Cheng, Qiguang Chen, Jin Zhang, Hao Fei, Xiaocheng Feng, Wanxiang Che, Min Li, Libo Qin

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted at AAAI 2025; Project Page: https://github.com/czhhzc/CoMT

Quality-Aware Image-Text Alignment for Opinion-Unaware Image Quality Assessment

Lorenzo Agnolucci, Leonardo Galteri, Marco Bertini

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.CL

See What You Are Told: Visual Attention Sink in Large Multimodal Models

Seil Kang, Jinyeong Kim, Junhyeok Kim, Seong Jae Hwang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Chitranuvad: Adapting Multi-Lingual LLMs for Multimodal Translation

Shaharukh Khan, Ayush Tarun, Ali Faraz, Palash Kamble, Vivek Dahiya, Praveen Pokala, Ashish Kulkarni, Chandra Khatri, Abhinav Ravi, Shubham Agarwal

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Journal ref https://aclanthology.org/2024.wmt-1.80/

Beyond Words: Exploring Cultural Value Sensitivity in Multimodal Models

Srishti Yadav, Zhi Zhang, Daniel Hershcovich, Ekaterina Shutova

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Journal ref NAACL 2025

MVAM: Multi-View Attention Method for Fine-grained Image-Text Matching

Wanqing Cui, Rui Cheng, Jiafeng Guo, Xueqi Cheng

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.CL

Comments Published as a conference paper at ECIR 2025

MFC-Bench: Benchmarking Multimodal Fact-Checking with Large Vision-Language Models

Shengkang Wang, Hongzhan Lin, Ziyang Luo, Zhen Ye, Guang Chen, Jing Ma

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments 28 pages, 9 figures

Demographic User Modeling for Social Robotics with Multimodal Pre-trained Models

Hamed Rahimi, Mouad Abrini, Mahdi Khoramshahi, Mohamed Chetouani

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Verifying Cross-modal Entity Consistency in News using Vision-language Models

Sahar Tahmasebi, David Ernst, Eric Müller-Budack, Ralph Ewerth

专题命中 图文多模态 ：cross-modal(title,abstract)；分类 cs.CL、cs.MM

Comments Accepted for publication in: European Conference on Information Retrieval (ECIR) 2025