多模态大模型

Reviving the Context: Camera Trap Species Classification as Link Prediction on Multimodal Knowledge Graphs

Vardaan Pahuja, Weidi Luo, Yu Gu, Cheng-Hao Tu, Hong-You Chen, Tanya Berger-Wolf, Charles Stewart, Song Gao, Wei-Lun Chao, Yu Su

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 12 pages, 5 figures

Towards Deconfounded Image-Text Matching with Causal Inference

Wenhui Li, Xinqi Su, Dan Song, Lanjun Wang, Kun Zhang, An-An Liu

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.AI

Comments ACM MM

Journal ref 2023/10/26,Proceedings of the 31st ACM International Conference on Multimedia,6264-6273

Robust Domain Generalization for Multi-modal Object Recognition

Yuxin Qiao, Keqin Li, Junhong Lin, Rong Wei, Chufeng Jiang, Yang Luo, Haoyu Yang

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments 6 pages, 2 figures. This is a preprint version of the article. The final version will be published in the proceedings of the IEEE conference

Contextual Object Detection with Multimodal Large Language Models

Yuhang Zang, Wei Li, Jun Han, Kaiyang Zhou, Chen Change Loy

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments IJCV 2024

Advancing Multimodal Large Language Models with Quantization-Aware Scale Learning for Efficient Adaptation

Jingjing Xie, Yuxin Zhang, Mingbao Lin, Liujuan Cao, Rongrong Ji

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by ACMMM2024

LLaVA-Read: Enhancing Reading Ability of Multimodal Language Models

Ruiyi Zhang, Yufan Zhou, Jian Chen, Jiuxiang Gu, Changyou Chen, Tong Sun

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments NeurIPS 2024 Under Review

Multimodal Misinformation Detection using Large Vision-Language Models

Sahar Tahmasebi, Eric Müller-Budack, Ralph Ewerth

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.MM

Comments Accepted for publication in: Conference on Information and Knowledge Management (CIKM) 2024

Mismatch Quest: Visual and Textual Feedback for Image-Text Misalignment

Brian Gordon, Yonatan Bitton, Yonatan Shafir, Roopal Garg, Xi Chen, Dani Lischinski, Daniel Cohen-Or, Idan Szpektor

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.CL

Journal ref ECCV 2024

Cephalo: Multi-Modal Vision-Language Models for Bio-Inspired Materials Analysis and Design

Markus J. Buehler

专题命中 图文多模态 ：multi-modal(title)；multimodal(abstract)；分类 cs.CV、cs.CL

MuDPT: Multi-modal Deep-symphysis Prompt Tuning for Large Pre-trained Vision-Language Models

Yongzhu Miao, Shasha Li, Jintao Tang, Ting Wang

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Comments The paper has been accepted by ICME 2023

Vision-Language Consistency Guided Multi-modal Prompt Learning for Blind AI Generated Image Quality Assessment

Jun Fu, Wei Zhou, Qiuping Jiang, Hantao Liu, Guangtao Zhai

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by IEEE Signal Processing Letter

Solution for SMART-101 Challenge of CVPR Multi-modal Algorithmic Reasoning Task 2024

Jinwoo Ahn, Junhyeok Park, Min-Jun Kim, Kang-Hyeon Kim, So-Yeong Sohn, Yun-Ji Lee, Du-Seong Chang, Yu-Jung Heo, Eun-Sol Kim

专题命中 图文多模态 ：multi-modal(title)；multimodal(abstract)；分类 cs.CV、cs.AI

Assessing News Thumbnail Representativeness: Counterfactual text can enhance the cross-modal matching ability

Yejun Yoon, Seunghyun Yoon, Kunwoo Park

专题命中 图文多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.CL

Comments ACL 2024 (findings), 16 pages

GLaMM: Pixel Grounding Large Multimodal Model

Hanoona Rasheed, Muhammad Maaz, Sahal Shaji Mullappilly, Abdelrahman Shaker, Salman Khan, Hisham Cholakkal, Rao M. Anwer, Erix Xing, Ming-Hsuan Yang, Fahad S. Khan

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments CVPR 2024

Concept Visualization: Explaining the CLIP Multi-modal Embedding Using WordNet

Loris Giulivi, Giacomo Boracchi

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted for publication at IJCNN 2024

Understanding Multimodal Deep Neural Networks: A Concept Selection View

Chenming Shang, Hengyuan Zhang, Hao Wen, Yujiu Yang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

On Speculative Decoding for Multimodal Large Language Models

Mukul Gagrani, Raghavv Goel, Wonseok Jeon, Junyoung Park, Mingu Lee, Christopher Lott

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted as a spotlight paper to ELVM workshop at CVPR 2024

A Review of Multi-Modal Large Language and Vision Models

Kilian Carolan, Laura Fennelly, Alan F. Smeaton

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Comments 33 pages, 1 figure

Open-Vocabulary Federated Learning with Multimodal Prototyping

Huimin Zeng, Zhenrui Yue, Dong Wang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted at NAACL 2024

Can Language Beat Numerical Regression? Language-Based Multimodal Trajectory Prediction

Inhwan Bae, Junoh Lee, Hae-Gon Jeon

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted at CVPR 2024

m3P: Towards Multimodal Multilingual Translation with Multimodal Prompt

Jian Yang, Hongcheng Guo, Yuwei Yin, Jiaqi Bai, Bing Wang, Jiaheng Liu, Xinnian Liang, Linzheng Cahi, Liqun Yang, Zhoujun Li

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments COLING 2024

Multi-Modal Hallucination Control by Visual Information Grounding

Alessandro Favero, Luca Zancato, Matthew Trager, Siddharth Choudhary, Pramuditha Perera, Alessandro Achille, Ashwin Swaminathan, Stefano Soatto

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Journal ref IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024

Leveraging Chat-Based Large Vision Language Models for Multimodal Out-Of-Context Detection

Fatma Shalabi, Hichem Felouat, Huy H. Nguyen, Isao Echizen

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 13 pages, 6 figures , conference

Summarize the Past to Predict the Future: Natural Language Descriptions of Context Boost Multimodal Object Interaction Anticipation

Razvan-George Pasca, Alexey Gavryushin, Muhammad Hamza, Yen-Ling Kuo, Kaichun Mo, Luc Van Gool, Otmar Hilliges, Xi Wang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Towards Grounded Visual Spatial Reasoning in Multi-Modal Vision Language Models

Navid Rajabi, Jana Kosecka

专题命中 图文多模态 ：multi-modal(title)；image-text(abstract)；分类 cs.CV、cs.CL

Comments Accepted to DMLR @ ICLR 2024

ITEm: Unsupervised Image-Text Embedding Learning for eCommerce

Baohao Liao, Michael Kozielski, Sanjika Hewavitharana, Jiangbo Yuan, Shahram Khadivi, Tomer Lancewicki

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.CL

OmDet: Large-scale vision-language multi-dataset pre-training with multimodal detection network

Tiancheng Zhao, Peng Liu, Kyusong Lee

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Published at IET CV

Scaffolding Coordinates to Promote Vision-Language Coordination in Large Multi-Modal Models

Xuanyu Lei, Zonghan Yang, Xinrui Chen, Peng Li, Yang Liu

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Visual Chain of Thought: Bridging Logical Gaps with Multimodal Infillings

Daniel Rose, Vaishnavi Himakunthala, Andy Ouyang, Ryan He, Alex Mei, Yujie Lu, Michael Saxon, Chinmay Sonar, Diba Mirza, William Yang Wang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Zoom-shot: Fast and Efficient Unsupervised Zero-Shot Transfer of CLIP to Vision Encoders with Multimodal Loss

Jordan Shipard, Arnold Wiliem, Kien Nguyen Thanh, Wei Xiang, Clinton Fookes

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 15 pages