多模态大模型

Why do LLaVA Vision-Language Models Reply to Images in English?

Musashi Hinck, Carolin Holtermann, Matthew Lyle Olson, Florian Schneider, Sungduk Yu, Anahita Bhiwandiwalla, Anne Lauscher, Shaoyen Tseng, Vasudev Lal

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Pre-print

Fake News Detection and Manipulation Reasoning via Large Vision-Language Models

Ruihan Jin, Ruibo Fu, Zhengqi Wen, Shuai Zhang, Yukun Liu, Jianhua Tao

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CL、cs.AI

RAVEN: Multitask Retrieval Augmented Vision-Language Learning

Varun Nagaraj Rao, Siddharth Choudhary, Aditya Deshpande, Ravi Kumar Satzoda, Srikar Appalaraju

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Composition Vision-Language Understanding via Segment and Depth Anything Model

Mingxiao Huo, Pengliang Ji, Haotian Lin, Junchen Liu, Yixiao Wang, Yijun Chen

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Constructing Multilingual Visual-Text Datasets Revealing Visual Multilingual Ability of Vision Language Models

Jesse Atuhurra, Iqra Ali, Tatsuya Hiraoka, Hidetaka Kamigaito, Tomoya Iwakura, Taro Watanabe

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

IntCoOp: Interpretability-Aware Vision-Language Prompt Tuning

Soumya Suvra Ghosal, Samyadeep Basu, Soheil Feizi, Dinesh Manocha

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

mBLIP: Efficient Bootstrapping of Multilingual Vision-LLMs

Gregor Geigle, Abhay Jain, Radu Timofte, Goran Glavaš

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments ALVR Workshop 2024

PUMGPT: A Large Vision-Language Model for Product Understanding

Wei Xue, Zongyi Guo, Baoliang Cui, Zheng Xing, Xiaoyi Zeng, Xiufei Wang, Shuhui Wu, Weiming Lu

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

From Pixels to Prose: A Large Dataset of Dense Image Captions

Vasu Singla, Kaiyu Yue, Sukriti Paul, Reza Shirkavand, Mayuka Jayawardhana, Alireza Ganjdanesh, Heng Huang, Abhinav Bhatele, Gowthami Somepalli, Tom Goldstein

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments pixelprose 16M dataset

Emergent Open-Vocabulary Semantic Segmentation from Off-the-shelf Vision-Language Models

Jiayun Luo, Siddhesh Khandelwal, Leonid Sigal, Boyang Li

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted to CVPR 2024; Earlier version of this paper contained an unintentional error stemming from a bug in the code. This version corrects this error, which had to do with filtering of class names. In consultation with CVPR Program Chairs it was suggested errata be submitted as the updated (fixed) code reinforced original findings (albeit with slightly different final numbers)

MAFA: Managing False Negatives for Vision-Language Pre-training

Jaeseok Byun, Dohoon Kim, Taesup Moon

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments CVPR 2024 camera ready version

Open-Vocabulary Segmentation with Unpaired Mask-Text Supervision

Zhaoqing Wang, Xiaobo Xia, Ziye Chen, Xiao He, Yandong Guo, Mingming Gong, Tongliang Liu

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments 27 pages, 18 figures, 10 tables

HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding

Zhaorun Chen, Zhuokai Zhao, Hongyin Luo, Huaxiu Yao, Bo Li, Jiawei Zhou

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments ICML camera-ready version. Code is released at https://github.com/BillChan226/HALC

CountCLIP -- [Re] Teaching CLIP to Count to Ten

Harshvardhan Mestha, Tejas Agrawal, Karan Bania, Shreyas V, Yash Bhisikar

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Uncovering Bias in Large Vision-Language Models with Counterfactuals

Phillip Howard, Anahita Bhiwandiwalla, Kathleen C. Fraser, Svetlana Kiritchenko

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to the CVPR 2024 Responsible Generative AI (ReGenAI) Workshop

Holistic Visual-Textual Sentiment Analysis with Prior Models

Junyu Chen, Jie An, Hanjia Lyu, Christopher Kanan, Jiebo Luo

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments Published in MIPR 2024

Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Models

Christian Schlarmann, Naman Deep Singh, Francesco Croce, Matthias Hein

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments ICML 2024 Oral

TopViewRS: Vision-Language Models as Top-View Spatial Reasoners

Chengzu Li, Caiqi Zhang, Han Zhou, Nigel Collier, Anna Korhonen, Ivan Vulić

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments 9 pages, 3 figures, 3 tables (21 pages, 4 figures, 15 tables including references and appendices)

Selectively Answering Visual Questions

Julian Martin Eisenschlos, Hernán Maina, Guido Ivetta, Luciana Benotti

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments To be published in the findings of the 2024 Annual Meeting of the Association for Computational Linguistics

OLIVE: Object Level In-Context Visual Embeddings

Timothy Ossowski, Junjie Hu

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments ACL 2024

WAVER: Writing-style Agnostic Text-Video Retrieval via Distilling Vision-Language Models Through Open-Vocabulary Knowledge

Huy Le, Tung Kieu, Anh Nguyen, Ngan Le

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to ICASSP 2024

InsightSee: Advancing Multi-agent Vision-Language Models for Enhanced Visual Understanding

Huaxiang Zhang, Yaojia Mu, Guo-Niu Zhu, Zhongxue Gan

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Evaluating Vision-Language Models on Bistable Images

Artemis Panagopoulou, Coby Melkin, Chris Callison-Burch

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Track Anything Rapter(TAR)

Tharun V. Puthanveettil, Fnu Obaid ur Rahman

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Zero Shot Context-Based Object Segmentation using SLIP (SAM+CLIP)

Saaketh Koundinya Gundavarapu, Arushi Arora, Shreya Agarwal

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments 5 pages, 3 figures

Measuring Vision-Language STEM Skills of Neural Models

Jianhao Shen, Ye Yuan, Srbuhi Mirzoyan, Ming Zhang, Chenguang Wang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments Accepted in ICLR 2024

Revisiting the Robust Generalization of Adversarial Prompt Tuning

Fan Yang, Mingxuan Xia, Sangzhou Xia, Chicheng Ma, Hui Hui

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Promoting AI Equity in Science: Generalized Domain Prompt Learning for Accessible VLM Research

Qinglong Cao, Yuntian Chen, Lu Lu, Hao Sun, Zhenzhong Zeng, Xiaokang Yang, Dongxiao Zhang

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Decoding Emotions in Abstract Art: Cognitive Plausibility of CLIP in Recognizing Color-Emotion Associations

Hanna-Sophia Widhoelzl, Ece Takmaz

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments To appear in the Proceedings of the Annual Meeting of the Cognitive Science Society 2024

Multi-Frame, Lightweight & Efficient Vision-Language Models for Question Answering in Autonomous Driving

Akshay Gopalkrishnan, Ross Greer, Mohan Trivedi

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 9 pages, 3 figures, Accepted at CVPR 2024 Vision and Language for Autonomous Driving and Robotics Workshop