多模态大模型

A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions

Jack Urbanek, Florian Bordes, Pietro Astolfi, Mary Williamson, Vasu Sharma, Adriana Romero-Soriano

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Leveraging VLM-Based Pipelines to Annotate 3D Objects

Rishabh Kabra, Loic Matthey, Alexander Lerchner, Niloy J. Mitra

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Precision Empowers, Excess Distracts: Visual Question Answering With Dynamically Infused Knowledge In Language Models

Manas Jhalani, Annervaz K M, Pushpak Bhattacharyya

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CL

Comments 16 pages, 12 figures

Comparison Visual Instruction Tuning

Wei Lin, Muhammad Jehanzeb Mirza, Sivan Doveh, Rogerio Feris, Raja Giryes, Sepp Hochreiter, Leonid Karlinsky

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Comments Project page: https://wlin-at.github.io/cad_vi ; Huggingface dataset repo: https://huggingface.co/datasets/wlin21at/CaD-Inst

CLIP-Driven Cloth-Agnostic Feature Learning for Cloth-Changing Person Re-Identification

Shuang Li, Jiaxu Leng, Guozhang Li, Ji Gan, Haosheng chen, Xinbo Gao

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

LM4LV: A Frozen Large Language Model for Low-level Vision Tasks

Boyang Zheng, Jinjin Gu, Shijun Li, Chao Dong

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

StrucTexTv3: An Efficient Vision-Language Model for Text-rich Image Perception, Comprehension, and Beyond

Pengyuan Lyu, Yulin Li, Hao Zhou, Weihong Ma, Xingyu Wan, Qunyi Xie, Liang Wu, Chengquan Zhang, Kun Yao, Errui Ding, Jingdong Wang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Dealing with All-stage Missing Modality: Towards A Universal Model with Robust Reconstruction and Personalization

Yunpeng Zhao, Cheng Chen, Qing You Pang, Quanzheng Li, Carol Tang, Beng-Ti Ang, Yueming Jin

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

PaLM2-VAdapter: Progressively Aligned Language Model Makes a Strong Vision-language Adapter

Junfei Xiao, Zheng Xu, Alan Yuille, Shen Yan, Boyu Wang

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments Technical report, 15 pages; v2 fix typos, add additional results in appendix

CLIP-QDA: An Explainable Concept Bottleneck Model

Rémi Kazmierczak, Eloïse Berthier, Goran Frehse, Gianni Franchi

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Journal ref Transactions on Machine Learning Research (05/2024)

Visual Perception by Large Language Model's Weights

Feipeng Ma, Hongwei Xue, Guangting Wang, Yizhou Zhou, Fengyun Rao, Shilin Yan, Yueyi Zhang, Siying Wu, Mike Zheng Shou, Xiaoyan Sun

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Knowledge-grounded Adaptation Strategy for Vision-language Models: Building Unique Case-set for Screening Mammograms for Residents Training

Aisha Urooj Khan, John Garrett, Tyler Bradshaw, Lonie Salkowski, Jiwoong Jason Jeong, Amara Tariq, Imon Banerjee

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Breaking the False Sense of Security in Backdoor Defense through Re-Activation Attack

Mingli Zhu, Siyuan Liang, Baoyuan Wu

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

MindSemantix: Deciphering Brain Visual Experiences with a Brain-Language Model

Ziqi Ren, Jie Li, Xuetong Xue, Xin Li, Fan Yang, Zhicheng Jiao, Xinbo Gao

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments 13 pages, 6 figures

Exploring Interactive Semantic Alignment for Efficient HOI Detection with Vision-language Model

Jihao Dong, Renjie Pan, Hua Yang

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV

CLIP model is an Efficient Online Lifelong Learner

Leyuan Wang, Liuyu Xiang, Yujie Wei, Yunlong Wang, Zhaofeng He

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Scaling Open-Vocabulary Object Detection

Matthias Minderer, Alexey Gritsenko, Neil Houlsby

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Neural Image Compression with Text-guided Encoding for both Pixel-level and Perceptual Fidelity

Hagyeong Lee, Minkyu Kim, Jun-Hyuk Kim, Seungeon Kim, Dokwan Oh, Jaeho Lee

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments The first two authors contributed equally

Fuse & Calibrate: A bi-directional Vision-Language Guided Framework for Referring Image Segmentation

Yichen Yan, Xingjian He, Sihan Chen, Shichen Lu, Jing Liu

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments 12 pages, 4 figures ICIC2024

CLIP with Quality Captions: A Strong Pretraining for Vision Tasks

Pavan Kumar Anasosalu Vasu, Hadi Pouransari, Fartash Faghri, Oncel Tuzel

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

CLIP-KD: An Empirical Study of CLIP Model Distillation

Chuanguang Yang, Zhulin An, Libo Huang, Junyu Bi, Xinqiang Yu, Han Yang, Boyu Diao, Yongjun Xu

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV

Comments CVPR-2024

CLIPose: Category-Level Object Pose Estimation with Pre-trained Vision-Language Knowledge

Xiao Lin, Minghao Zhu, Ronghao Dang, Guangliang Zhou, Shaolong Shu, Feng Lin, Chengju Liu, Qijun Chen

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments 14 pages, 4 figures, 9 tables

Language-Enhanced Latent Representations for Out-of-Distribution Detection in Autonomous Driving

Zhenjiang Mao, Dong-You Jhong, Ao Wang, Ivan Ruchkin

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Comments Presented at the Robot Trust for Symbiotic Societies (RTSS) Workshop, co-located with ICRA 2024

Compositional Learning of Visually-Grounded Concepts Using Reinforcement

Zijun Lin, Haidi Azaman, M Ganesh Kumar, Cheston Tan

专题命中 图文多模态 ：image-text(abstract)；分类 cs.AI

Utilizing Synthetic Data for Medical Vision-Language Pre-training: Bypassing the Need for Real Images

Che Liu, Anand Shah, Wenjia Bai, Rossella Arcucci

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments Accepted by CVPR 2024 Workshop Data Curation and Augmentation in Enhancing Medical Imaging Applications

Semi-supervised Text-based Person Search

Daming Gao, Yang Bai, Min Cao, Hao Dou, Mang Ye, Min Zhang

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments 13 pages

Zero-Shot Distillation for Image Encoders: How to Make Effective Use of Synthetic Data

Niclas Popp, Jan Hendrik Metzen, Matthias Hein

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Adaptive Prompt Learning with Negative Textual Semantics and Uncertainty Modeling for Universal Multi-Source Domain Adaptation

Yuxiang Yang, Lu Wen, Yuanyuan Xu, Jiliu Zhou, Yan Wang

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments Accepted by ICME2024

FLoRA: Enhancing Vision-Language Models with Parameter-Efficient Federated Learning

Duy Phuong Nguyen, J. Pablo Munoz, Ali Jannesari

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.AI

Comments 10 pages, 11 figures

Omniview-Tuning: Boosting Viewpoint Invariance of Vision-Language Pre-training Models

Shouwei Ruan, Yinpeng Dong, Hanqing Liu, Yao Huang, Hang Su, Xingxing Wei

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments 20 pages