多模态大模型

From Introspection to Best Practices: Principled Analysis of Demonstrations in Multimodal In-Context Learning

Nan Xu, Fei Wang, Sheng Zhang, Hoifung Poon, Muhao Chen

专题命中 图文多模态 ：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments NAACL 2025

Unveiling Uncertainty: A Deep Dive into Calibration and Performance of Multimodal Large Language Models

Zijun Chen, Wenbo Hu, Guande He, Zhijie Deng, Zheng Zhang, Richang Hong

专题命中 图文多模态 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted to COLING 2025

Leveraging Customer Feedback for Multi-modal Insight Extraction

Sandeep Sricharan Mukku, Abinesh Kanagarajan, Pushpendu Ghosh, Chetan Aggarwal

专题命中 图文多模态 ：multi-modal(title,abstract)；image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments NAACL 2024

Mitigate the Gap: Investigating Approaches for Improving Cross-Modal Alignment in CLIP

Sedigheh Eslami, Gerard de Melo

专题命中 图文多模态 ：cross-modal(title,abstract)；multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Vision-Language Meets the Skeleton: Progressively Distillation with Cross-Modal Knowledge for 3D Action Representation Learning

Yang Chen, Tian He, Junfeng Fu, Ling Wang, Jingcai Guo, Ting Hu, Hong Cheng

专题命中 图文多模态 ：cross-modal(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted by IEEE Transactions on Multimedia

Vision Learners Meet Web Image-Text Pairs

Bingchen Zhao, Quan Cui, Hao Wu, Osamu Yoshie, Cheng Yang, Oisin Mac Aodha

专题命中 图文多模态 ：image-text(title,abstract)；multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Project page: https://bzhao.me/MUG/

Unaligning Everything: Or Aligning Any Text to Any Image in Multimodal Models

Shaeke Salman, Md Montasir Bin Shams, Xiuwen Liu

专题命中 图文多模态 ：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 14 pages, 14 figures. arXiv admin note: substantial text overlap with arXiv:2401.15568, arXiv:2402.08473

VEGA: Learning Interleaved Image-Text Comprehension in Vision-Language Large Models

Chenyu Zhou, Mengdan Zhang, Peixian Chen, Chaoyou Fu, Yunhang Shen, Xiawu Zheng, Xing Sun, Rongrong Ji

专题命中 图文多模态 ：image-text(title,abstract)；multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Project Page: https://zhourax.github.io/VEGA/

Groma: Localized Visual Tokenization for Grounding Multimodal Large Language Models

Chuofan Ma, Yi Jiang, Jiannan Wu, Zehuan Yuan, Xiaojuan Qi

专题命中 图文多模态 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL、cs.AI

CapsFusion: Rethinking Image-Text Data at Scale

Qiying Yu, Quan Sun, Xiaosong Zhang, Yufeng Cui, Fan Zhang, Yue Cao, Xinlong Wang, Jingjing Liu

专题命中 图文多模态 ：image-text(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments CVPR 2024. Code & Dataset: https://github.com/baaivision/CapsFusion

Towards Unified Multi-Modal Personalization: Large Vision-Language Models for Generative Recommendation and Beyond

Tianxin Wei, Bowen Jin, Ruirui Li, Hansi Zeng, Zhengyang Wang, Jianhui Sun, Qingyu Yin, Hanqing Lu, Suhang Wang, Jingrui He, Xianfeng Tang

专题命中 图文多模态 ：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CL、cs.AI、cs.MM

Comments ICLR 2024

HyperLLaVA: Dynamic Visual and Language Expert Tuning for Multimodal Large Language Models

Wenqiao Zhang, Tianwei Lin, Jiang Liu, Fangxun Shu, Haoyuan Li, Lei Zhang, He Wanggui, Hao Zhou, Zheqi Lv, Hao Jiang, Juncheng Li, Siliang Tang, Yueting Zhuang

专题命中 图文多模态 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL、cs.AI

Toward Robust Multimodal Learning using Multimodal Foundational Models

Xianbing Zhao, Soujanya Poria, Xuejiao Li, Yixin Chen, Buzhou Tang

专题命中 图文多模态 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Under Review

Structure-CLIP: Towards Scene Graph Knowledge to Enhance Multi-modal Structured Representations

Yufeng Huang, Jiji Tang, Zhuo Chen, Rongsheng Zhang, Xinfeng Zhang, Weijie Chen, Zeng Zhao, Zhou Zhao, Tangjie Lv, Zhipeng Hu, Wen Zhang

专题命中 图文多模态 ：multi-modal(title,abstract)；image-text(abstract)；分类 cs.CL、cs.AI、cs.MM

Comments AAAI 2024, https://github.com/zjukg/Structure-CLIP

Alternating Gradient Descent and Mixture-of-Experts for Integrated Multimodal Perception

Hassan Akbari, Dan Kondratyuk, Yin Cui, Rachel Hornung, Huisheng Wang, Hartwig Adam

专题命中 图文多模态 ：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV、cs.AI、cs.MM

A New Fine-grained Alignment Method for Image-text Matching

Yang Zhang

专题命中 图文多模态 ：image-text(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Models

Ziyi Lin, Chris Liu, Renrui Zhang, Peng Gao, Longtian Qiu, Han Xiao, Han Qiu, Chen Lin, Wenqi Shao, Keqin Chen, Jiaming Han, Siyuan Huang, Yichi Zhang, Xuming He, Hongsheng Li, Yu Qiao

专题命中 图文多模态 ：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Work in progress. Code and demos are released at https://github.com/Alpha-VLLM/LLaMA2-Accessory

MAP: Multimodal Uncertainty-Aware Vision-Language Pre-training Model

Yatai Ji, Junjie Wang, Yuan Gong, Lin Zhang, Yanru Zhu, Hongfa Wang, Jiaxing Zhang, Tetsuya Sakai, Yujiu Yang

专题命中 图文多模态 ：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV、cs.CL、cs.MM

Comments CVPR 2023 Main Track Long Paper

MAMO: Masked Multimodal Modeling for Fine-Grained Vision-Language Representation Learning

Zijia Zhao, Longteng Guo, Xingjian He, Shuai Shao, Zehuan Yuan, Jing Liu

专题命中 图文多模态 ：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV、cs.CL、cs.MM

Comments SIGIR 2023, 10 pages

Is Multimodal Vision Supervision Beneficial to Language?

Avinash Madasu, Vasudev Lal

专题命中 图文多模态 ：multimodal(title)；multi-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

MoMo: A shared encoder Model for text, image and multi-Modal representations

Rakesh Chada, Zhaoheng Zheng, Pradeep Natarajan

专题命中 图文多模态 ：multi-modal(title)；multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation

Wenliang Dai, Lu Hou, Lifeng Shang, Xin Jiang, Qun Liu, Pascale Fung

专题命中 图文多模态 ：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted to ACL 2022

LoopITR: Combining Dual and Cross Encoder Architectures for Image-Text Retrieval

Jie Lei, Xinlei Chen, Ning Zhang, Mengjiao Wang, Mohit Bansal, Tamara L. Berg, Licheng Yu

专题命中 图文多模态 ：image-text(title,abstract)；multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Multimodal Adversarial Defense for Vision-Language Models by Leveraging One-To-Many Relationships

通过利用一对一关系进行多模态对抗防御以提升视觉语言模型

Futa Waseda, Antonio Tejero-de-Pablos, Isao Echizen

机构 * The University of Tokyo（东京大学） ； CyberAgent ； National Institute of Informatics（信息处理研究所） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV、cs.AI

AI总结 本文提出多模态对抗训练方法，通过利用图像与文本之间的一对多关系提升视觉语言模型的对抗鲁棒性。

Comments WACV 2026 Accepted. Code available at https://github.com/CyberAgentAILab/multimodal-adversarial-training

Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Models

Hao Dong, Moru Liu, Kaiyang Zhou, Eleni Chatzi, Juho Kannala, Cyrill Stachniss, Olga Fink

专题命中 图文多模态 ：multimodal(title,abstract)；multimodal foundation model(abstract)；分类 cs.CV、cs.AI

Comments Project page: https://github.com/donghao51/Awesome-Multimodal-Adaptation

On Domain-Adaptive Post-Training for Multimodal Large Language Models

Daixuan Cheng, Shaohan Huang, Ziyu Zhu, Xintong Zhang, Wayne Xin Zhao, Zhongzhi Luan, Bo Dai, Zhenliang Zhang

机构 * BIGAI ； BUAA（北京航空航天大学） ； THU（清华大学） ； BIT（北京理工大学） ； RUC（中国人民大学） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；MLLM(abstract,comments)；分类 cs.CV、cs.CL

Comments EMNLP 2025 Findings, Project Page: https://huggingface.co/AdaptLLM/Adapt-MLLM-to-Domains

DGSeg: Dynamic Gating of Semantic-Spatial Guided Predictions for Reasoning Segmentation

DGSeg：用于推理分割的语义-空间引导预测的动态门控

Ruizhe Zeng, Siyu Cao, Lu Zhang, Zhiyong Liu

机构 * State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所多模态人工智能系统国家重点实验室） ； School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院） ； Nanjing Artificial Intelligence Research of IA（中国科学院自动化所南京人工智能研究院） 纠错

专题命中 图文多模态 ：MLLM(summary_cn,abstract)；multimodal(abstract)；分类 cs.CV

AI总结 研究推理分割问题，提出DGSeg框架，借助MLLM生成语义和空间线索，经动态门控模块自适应融合预测，抑制噪声或冲突区域，在多基准测试中表现出色。

Comments Accepted to ECCV2026

Estimating Uncertainty in Multimodal Foundation Models using Public Internet Data

Shiladitya Dutta, Hongbo Wei, Lars van der Laan, Ahmed M. Alaa

专题命中 图文多模态 ：multimodal(title)；multimodal foundation model(title)；分类 cs.AI

Query-Driven Multimodal Information Extraction from Long Documents

面向长文档的查询驱动多模态信息抽取

Yikai Gao, Ding Xia, Xi Yang

机构 * School of Artificial Intelligence, Jilin University（吉林大学人工智能学院） ； Graduate School of Information Science and Technology, The University of Tokyo（东京大学信息科学与技术研究生院） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；image-text(abstract)；分类 cs.AI、cs.MM

AI总结 针对现有多模态长文档信息抽取范式的不足，本文提出查询驱动的图像-文本联合抽取任务，构建基准ITJoint并设计多智能体框架Q2IT，实验显示Q2IT性能显著优于独立VLMs但仍有提升空间。

The Role of Natural Language Understanding in Multimodal Video-Based Dengue Diagnosis

自然语言理解在基于多模态视频的登革热诊断中的作用

Danial Sharifrazi, Saadat Behzadi, Julakha Jahan Jui, Mojtaba Mohammadi, Nouman Javed, Roohallah Alizadehsani, Prasad N. Paradkar, Asim Bhatti

专题命中 图文多模态 ：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV、cs.AI

AI总结 本研究提出基于YOLO和CLIP的视觉-语言框架，从视频中分类未受感染与DENV2感染的蚊子，帧级准确率达98.54%、灵敏度达99.91%，证实该框架可用于分析感染相关生物行为。