多模态大模型

AndesVL Technical Report: An Efficient Mobile-side Multimodal Large Language Model

AndesVL 技术报告：一种高效的移动端多模态大语言模型

Zhiwei Jin, Xiaohui Song, Nan Wang, Yafei Liu, Chao Li, Xin Li, Ruichen Wang, Zhihao Li, Qi Qi, Long Cheng, Dongze Hao, Quanlong Zheng, Yanhao Zhang, Haobo Ji, Jian Ma, Zhitong Zheng, Zhenyi Lin, Haolin Deng, Xin Zou, Xiaojie Yin, Ruilin Wang, Liankai Cai, Haijing Liu, Yuqing Qiu, Ke Chen, Zixian Li, Chi Xie, Huafei Li, Chenxing Li, Chuangchuang Wang, Kai Tang, Zhiguang Zhu, Kai Tang, Wenmei Gao, Rui Wang, Jun Wu, Chao Liu, Qin Xie, Chen Chen, Haonan Lu

机构 * AndesVL Team（AndesVL团队） ； OPPO AI Center（OPPO人工智能中心） 纠错

专题命中 图文多模态 ：multimodal(title)；分类 cs.CV、cs.AI

AI总结 AndesVL 是一种高效的移动端多模态大语言模型，通过 1+N LoRA 架构和 QALFT 框架实现高效任务适应和模型压缩，部署在 MediaTek Dimensity 9500 芯片上，达到 6.7 倍解码加速和 30.9% 内存减少。

Comments Tech report of OPPO AndesVL Team

Rethinking Visual Information Processing in Multimodal LLMs

Dongwan Kim, Viresh Ranjan, Takashi Nagata, Arnab Dhua, Amit Kumar K C

机构 * Seoul National University（首尔国立大学） ； Amazon（亚马逊） 纠错

专题命中 图文多模态 ：multimodal(title)；分类 cs.CV、cs.AI

Evaluating Reasoning Faithfulness in Medical Vision-Language Models using Multimodal Perturbations

Johannes Moll, Markus Graf, Tristan Lemke, Nicolas Lenhart, Daniel Truhn, Jean-Benoit Delbrouck, Jiazhen Pan, Daniel Rueckert, Lisa C. Adams, Keno K. Bressem

机构 * Technical University of Munich (TUM)（慕尼黑技术大学） ； TUM University Hospital（慕尼黑技术大学医院） ； German Heart Center TUM University Hospital（慕尼黑技术大学医院德国心脏中心） ； Department of Radiology（放射科） ； Klinikum rechts der Isar TUM University Hospital（慕尼黑技术大学医院右岸诊所） ； Uniklinik RWTH Aachen（亚琛工业大学医院） ； HOPPR IL USA（HOPPR美国） ； University of Oxford（牛津大学） ； Imperial College London（伦敦帝国学院） 纠错

专题命中 图文多模态 ：multimodal(title)；分类 cs.CV、cs.CL

Comments Accepted to ML4H 2025 Proceedings

$\left|\,\circlearrowright\,\boxed{\text{BUS}}\,\right|$: A Large and Diverse Multimodal Benchmark for evaluating the ability of Vision-Language Models to understand Rebus Puzzles

Trishanu Das, Abhilash Nandy, Khush Bajaj, Deepiha S

机构 * Tredence Inc.（特伦德公司） ； Indian Institute of Technology Kharagpur（印度理工学院克拉格浦尔分校） ； Inria Paris-Rocquencourt（巴黎-罗克琴库特研究所） ； Rajiv Gandhi University（拉吉夫·甘地大学） ； Tsinghua University（清华大学） ； Palmer Research Laboratories（帕勒姆研究实验室） 纠错

专题命中 图文多模态 ：multimodal(title)；分类 cs.CV、cs.CL

Comments 7 pages, 5 figures, 4 tables

Language as a Label: Zero-Shot Multimodal Classification of Everyday Postures under Data Scarcity

MingZe Tang, Jubal Chandy Jacob

机构 * Department of Computing Science University of Aberdeen（计算科学系阿伯丁大学） 纠错

专题命中 图文多模态 ：multimodal(title)；分类 cs.CV、cs.AI

Instruction-tuned Self-Questioning Framework for Multimodal Reasoning

You-Won Jang, Yu-Jung Heo, Jaeseok Kim, Minsu Lee, Du-Seong Chang, Byoung-Tak Zhang

专题命中 图文多模态 ：multimodal(title)；分类 cs.CV、cs.AI

Comments This paper was accepted to the "CLVL: 5th Workshop on Closing the Loop Between Vision and Language (ICCV 2023 CLVL workshop)."

Enhancing Multimodal In-Context Learning for Image Classification through Coreset Optimization

Huiyi Chen, Jiawei Peng, Kaihua Tang, Xin Geng, Xu Yang

机构 * Southeast University（东南大学） ； Huawei Singapore Research Center（华为新加坡研究中心） 纠错

专题命中 图文多模态 ：multimodal(title)；分类 cs.CV、cs.AI

Comments 11 pages, 5 figures

Multimodal Political Bias Identification and Neutralization

Cedric Bernard, Xavier Pleimling, Amun Kharel, Chase Vickery

专题命中 图文多模态 ：multimodal(title)；分类 cs.CV、cs.AI

AlignMMBench: Evaluating Chinese Multimodal Alignment in Large Vision-Language Models

Yuhang Wu, Wenmeng Yu, Yean Cheng, Yan Wang, Xiaohan Zhang, Jiazheng Xu, Ming Ding, Yuxiao Dong

机构 * Tsinghua University（清华大学） ； Zhipu AI（智谱AI） ； Peking University（北京大学） 纠错

专题命中 图文多模态 ：multimodal(title)；分类 cs.CV、cs.CL

SingaKids: A Multilingual Multimodal Dialogic Tutor for Language Learning

Zhengyuan Liu, Geyu Lin, Hui Li Tan, Huayun Zhang, Yanfeng Lu, Xiaoxue Gao, Stella Xin Yin, He Sun, Hock Huan Goh, Lung Hsiang Wong, Nancy F. Chen

机构 * Nanyang Technological University, Singapore（新加坡南洋理工大学） ； National Institute of Education (NIE), Singapore（新加坡国家教育研究所） ； Institute for Infocomm Research (I 2 R), A*STAR, Singapore（新加坡资讯与通信研究院（I2R）） 纠错

专题命中 图文多模态 ：multimodal(title)；分类 cs.CL、cs.AI

Comments ACL 2025 Industry Track

QARI-OCR: High-Fidelity Arabic Text Recognition through Multimodal Large Language Model Adaptation

Ahmed Wasfy, Omer Nacar, Abdelakreem Elkhateb, Mahmoud Reda, Omar Elshehy, Adel Ammar, Wadii Boulila

机构 * NAMAA ； KAND CA Corp. ； Prince Sultan University 纠错

专题命中 图文多模态 ：multimodal(title)；分类 cs.CV、cs.AI

Do You Keep an Eye on What I Ask? Mitigating Multimodal Hallucination via Attention-Guided Ensemble Decoding

Yeongjae Cho, Keonwoo Kim, Taebaek Hwang, Sungzoon Cho

机构 * Seoul National University（首尔国立大学） ； Kim & Chang AI&IT System Center（金·昌AI&IT系统中心） ； Waddle Corporation（Waddle公司） 纠错

专题命中 图文多模态 ：multimodal(title)；分类 cs.CV、cs.AI

A Multimodal Pipeline for Clinical Data Extraction: Applying Vision-Language Models to Scans of Transfusion Reaction Reports

Henning Schäfer, Cynthia S. Schmidt, Johannes Wutzkowsky, Kamil Lorek, Lea Reinartz, Johannes Rückert, Christian Temme, Britta Böckmann, Peter A. Horn, Christoph M. Friedrich

机构 * Institute for Transfusion Medicine, University Hospital Essen, Hufelandstraße 55, Essen, Germany（1 输血医学研究所，埃森大学医院，Hufelandstraße 55，德国） ； Department of Computer Science, University of Applied Sciences and Arts Dortmund (FHDO), Emil-Figge Str. 42, Dortmund, Germany（2 计算机科学系，多特蒙德应用科学大学（FHDO），Emil-Figge Str. 42，德国） ； Institute for Medical Informatics, Biometry and Epidemiology (IMIBE), University Hospital Essen, Hufelandstraße 55, Essen, Germany（3 医学信息学、生物统计学和流行病学研究所（IMIBE），埃森大学医院，Hufelandstraße 55，德国） ； Institute for AI in Medicine (IKIM), University Hospital Essen, Girardetstraße 2, Essen, Germany（4 医学人工智能研究所（IKIM），埃森大学医院，Girardetstraße 2，德国） ； Institute of Interventional and Diagnostic Radiology and Neuroradiology, University Hospital Essen, Hufelandstraße 55, Essen, Germany（5 干预性和诊断放射学及神经放射学研究所，埃森大学医院，Hufelandstraße 55，德国） ； Department of Dermatology, University Hospital Essen, Hufelandstraße 55, Essen, Germany（6 皮肤科系，埃森大学医院，Hufelandstraße 55，德国） 纠错

专题命中 图文多模态 ：multimodal(title)；分类 cs.CV、cs.CL

GeoRSMLLM: A Multimodal Large Language Model for Vision-Language Tasks in Geoscience and Remote Sensing

Zilun Zhang, Haozhan Shen, Tiancheng Zhao, Bin Chen, Zian Guan, Yuhao Wang, Xu Jia, Yuxiang Cai, Yongheng Shang, Jianwei Yin

专题命中 图文多模态 ：multimodal(title)；分类 cs.CV、cs.AI

Multimodal Cognitive Reframing Therapy via Multi-hop Psychotherapeutic Reasoning

Subin Kim, Hoonrae Kim, Heejin Do, Gary Geunbae Lee

专题命中 图文多模态 ：multimodal(title)；分类 cs.CL、cs.AI

Comments NAACL 2025 Main

A Multi-Modal Explainability Approach for Human-Aware Robots in Multi-Party Conversation

Iveta Bečková, Štefan Pócoš, Giulia Belgiovine, Marco Matarese, Omar Eldardeer, Alessandra Sciutti, Carlo Mazzola

专题命中 图文多模态 ：multi-modal(title)；分类 cs.CL、cs.AI

Comments 32pp (+6pp sup.mat.) Accepted in Computer Vision and Image Understanding Journal on January 23, 2025. This research received funding Horizon-Europe TERAIS project (G.A. 101079338) and Slovak Research and Development Agency, project no. APVV-21-0105

MM-GEN: Enhancing Task Performance Through Targeted Multimodal Data Curation

Siddharth Joshi, Besmira Nushi, Vidhisha Balachandran, Varun Chandrasekaran, Vibhav Vineet, Neel Joshi, Baharan Mirzasoleiman

专题命中 图文多模态 ：multimodal(title)；分类 cs.CV、cs.CL

Clustering-based Image-Text Graph Matching for Domain Generalization

Nokyung Park, Daewon Chae, Jeongyong Shim, Sangpil Kim, Eun-Sol Kim, Jinkyu Kim

专题命中 图文多模态 ：image-text(title)；分类 cs.CV、cs.AI

RoCOCO: Robustness Benchmark of MS-COCO to Stress-test Image-Text Matching Models

Seulki Park, Daeho Um, Hajung Yoon, Sanghyuk Chun, Sangdoo Yun, Jin Young Choi

专题命中 图文多模态 ：image-text(title)；分类 cs.CV、cs.AI

Comments Accepted to ECCV Synthetic Data for Computer Vision Workshop (Oral)

Vision Search Assistant: Empower Vision-Language Models as Multimodal Search Engines

Zhixin Zhang, Yiyuan Zhang, Xiaohan Ding, Xiangyu Yue

专题命中 图文多模态 ：multimodal(title)；分类 cs.CV、cs.AI

Comments Code is available at https://github.com/cnzzx/VSA

HeGraphAdapter: Tuning Multi-Modal Vision-Language Models with Heterogeneous Graph Adapter

Yumiao Zhao, Bo Jiang, Xiao Wang, Qin Xu, Jin Tang

专题命中 图文多模态 ：multi-modal(title)；分类 cs.CV、cs.MM

FairPIVARA: Reducing and Assessing Biases in CLIP-Based Multimodal Models

Diego A. B. Moreira, Alef Iury Ferreira, Jhessica Silva, Gabriel Oliveira dos Santos, Luiz Pereira, João Medrado Gondim, Gustavo Bonil, Helena Maia, Nádia da Silva, Simone Tiemi Hashiguti, Jefersson A. dos Santos, Helio Pedrini, Sandra Avila

专题命中 图文多模态 ：multimodal(title)；分类 cs.CV、cs.AI

Comments 14 pages, 10 figures. Accepted to 35th British Machine Vision Conference (BMVC 2024), Workshop on Privacy, Fairness, Accountability and Transparency in Computer Vision

BCAmirs at SemEval-2024 Task 4: Beyond Words: A Multimodal and Multilingual Exploration of Persuasion in Memes

Amirhossein Abaskohi, Amirhossein Dabiriaghdam, Lele Wang, Giuseppe Carenini

专题命中 图文多模态 ：multimodal(title)；分类 cs.CV、cs.CL

Comments 12 pages, 5 tables, 2 figures, Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024) @ NAACL 2024

A Multi-Modal Foundation Model to Assist People with Blindness and Low Vision in Environmental Interaction

Yu Hao, Fan Yang, Hao Huang, Shuaihang Yuan, Sundeep Rangan, John-Ross Rizzo, Yao Wang, Yi Fang

专题命中 图文多模态 ：multi-modal(title)；分类 cs.CV、cs.AI

Let's Go Shopping (LGS) -- Web-Scale Image-Text Dataset for Visual Concept Understanding

Yatong Bai, Utsav Garg, Apaar Shanker, Haoming Zhang, Samyak Parajuli, Erhan Bas, Isidora Filipovic, Amelia N. Chu, Eugenia D Fomitcheva, Elliot Branson, Aerin Kim, Somayeh Sojoudi, Kyunghyun Cho

专题命中 图文多模态 ：image-text(title)；分类 cs.CV、cs.AI

Self-Imagine: Effective Unimodal Reasoning with Multimodal Models using Self-Imagination

Syeda Nahida Akter, Aman Madaan, Sangwu Lee, Yiming Yang, Eric Nyberg

专题命中 图文多模态 ：multimodal(title)；分类 cs.CL、cs.AI

Comments 18 pages, 9 figures, 12 tables

Enhancing Multimodal Understanding with CLIP-Based Image-to-Text Transformation

Chang Che, Qunwei Lin, Xinyu Zhao, Jiaxin Huang, Liqiang Yu

专题命中 图文多模态 ：multimodal(title)；分类 cs.CV、cs.AI

Cross-Modal Concept Learning and Inference for Vision-Language Models

Yi Zhang, Ce Zhang, Yushun Tang, Zhihai He

专题命中 图文多模态 ：cross-modal(title)；分类 cs.CV、cs.CL

Self-supervised Multi-modal Training from Uncurated Image and Reports Enables Zero-shot Oversight Artificial Intelligence in Radiology

Sangjoon Park, Eun Sun Lee, Kyung Sook Shin, Jeong Eun Lee, Jong Chul Ye

专题命中 图文多模态 ：multi-modal(title)；分类 cs.CV、cs.CL

Wukong-Reader: Multi-modal Pre-training for Fine-grained Visual Document Understanding

Haoli Bai, Zhiguang Liu, Xiaojun Meng, Wentao Li, Shuang Liu, Nian Xie, Rongfu Zheng, Liangwei Wang, Lu Hou, Jiansheng Wei, Xin Jiang, Qun Liu

专题命中 图文多模态 ：multi-modal(title)；分类 cs.CV、cs.CL