多模态大模型 - arXivDaily 专题

2310.07355 2024-10-31 cs.CV cs.LG 57%

IMITATE: Clinical Prior Guided Hierarchical Vision-Language Pre-training

Che Liu, Sibo Cheng, Miaojing Shi, Anand Shah, Wenjia Bai, Rossella Arcucci

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV

Comments Accepted by TMI2024

2410.17827 2024-10-24 cs.AI 57%

RE-tune: Incremental Fine Tuning of Biomedical Vision-Language Models for Multi-label Chest X-ray Classification

Marco Mistretta, Andrew D. Bagdanov

专题命中图文多模态：multimodal(abstract)；分类 cs.AI

Comments Accepted for publication at Medical Imaging meets NeurIPS (NeurIPS23)

2410.17779 2024-10-24 cs.CV 57%

ADEM-VL: Adaptive and Embedded Fusion for Efficient Vision-Language Tuning

Zhiwei Hao, Jianyuan Guo, Li Shen, Yong Luo, Han Hu, Yonggang Wen

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

2405.15574 2024-10-24 cs.CV 57%

Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Models

Byung-Kwan Lee, Chae Won Kim, Beomchan Park, Yong Man Ro

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments Code is available in https://github.com/ByungKwanLee/Meteor

2406.01583 2024-10-22 cs.CV cs.LG 57%

Decomposing and Interpreting Image Representations via Text in ViTs Beyond CLIP

Sriram Balasubramanian, Samyadeep Basu, Soheil Feizi

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments NeurIPS 2024, 31 pages, 15 figures

2410.13421 2024-10-18 cs.CV 57%

Performance of Gaussian Mixture Model Classifiers on Embedded Feature Spaces

Jeremy Chopin, Rozenn Dahyot

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments 8 pages

2410.12790 2024-10-17 cs.CV cs.LG 57%

Dual Prototype Evolving for Test-Time Generalization of Vision-Language Models

Ce Zhang, Simon Stepputtis, Katia Sycara, Yaqi Xie

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2024. Project page: https://zhangce01.github.io/DPE-CLIP

2409.02253 2024-10-17 cs.CV 57%

How to Determine the Preferred Image Distribution of a Black-Box Vision-Language Model?

Saeid Asgari Taghanaki, Joseph Lambourne, Alana Mongkhounsavath

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

Comments Accepted to NeurIPS 2024, Safe Generative AI

2312.14494 2024-10-15 cs.CV 57%

Revisiting Few-Shot Object Detection with Vision-Language Models

Anish Madan, Neehar Peri, Shu Kong, Deva Ramanan

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

Comments The first two authors contributed equally. This work has been accepted to the Neural Information Processing Systems (NeurIPS) 2024 Datasets & Benchmark Track

2410.09382 2024-10-15 cs.CV 57%

CLIP-SCGI: Synthesized Caption-Guided Inversion for Person Re-Identification

Qianru Han, Xinwei He, Zhi Liu, Sannyuya Liu, Ying Zhang, Jinhai Xiang

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV

2308.09779 2024-10-15 cs.CV 57%

EAVL: Explicitly Align Vision and Language for Referring Image Segmentation

Yichen Yan, Xingjian He, Wenxuan Wang, Sihan Chen, Jing Liu

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

Comments 10 pages, 4 figures. arXiv admin note: text overlap with arXiv:2305.14969

2410.08895 2024-10-14 cs.CV 57%

Calibrated Cache Model for Few-Shot Vision-Language Model Adaptation

Kun Ding, Qiang Yu, Haojian Zhang, Gaofeng Meng, Shiming Xiang

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments submitted to IJCV

2410.06626 2024-10-10 cs.CV 57%

Open-RGBT: Open-vocabulary RGB-T Zero-shot Semantic Segmentation in Open-world Environments

Meng Yu, Luojie Yang, Xunjie He, Yi Yang, Yufeng Yue

专题命中图文多模态：image-text(abstract)；分类 cs.CV

2410.06194 2024-10-10 cs.CV 57%

Prompting DirectSAM for Semantic Contour Extraction in Remote Sensing Images

Shiyu Miao, Delong Chen, Fan Liu, Chuanyi Zhang, Yanhui Gu, Shengjie Guo, Jun Zhou

专题命中图文多模态：image-text(abstract)；分类 cs.CV

2405.14206 2024-10-10 cs.CV 57%

LG-VQ: Language-Guided Codebook Learning

Guotao Liang, Baoquan Zhang, Yaowei Wang, Xutao Li, Yunming Ye, Huaibin Wang, Chuyao Luo, Kola Ye, linfeng Luo

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2024

Journal ref NeurIPS 2024

2407.04603 2024-10-08 cs.CV 57%

AWT: Transferring Vision-Language Models via Augmentation, Weighting, and Transportation

Yuhan Zhu, Yuyang Ji, Zhiyu Zhao, Gangshan Wu, Limin Wang

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2024

2410.01407 2024-10-03 cs.CV 57%

AgriCLIP: Adapting CLIP for Agriculture and Livestock via Domain-Specialized Cross-Model Alignment

Umair Nawaz, Muhammad Awais, Hanan Gani, Muzammal Naseer, Fahad Khan, Salman Khan, Rao Muhammad Anwer

专题命中图文多模态：image-text(abstract)；分类 cs.CV

2407.01445 2024-10-03 cs.LG cs.CV 57%

FastCLIP: A Suite of Optimization Techniques to Accelerate CLIP Training with Limited Resources

Xiyuan Wei, Fanjiang Ye, Ori Yonay, Xingyu Chen, Baixi Sun, Dingwen Tao, Tianbao Yang

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments 29 pages

2406.19389 2024-10-02 cs.CV 57%

OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understanding

Tao Zhang, Xiangtai Li, Hao Fei, Haobo Yuan, Shengqiong Wu, Shunping Ji, Chen Change Loy, Shuicheng Yan

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments NeurIPS-2024. Project page: https://lxtgh.github.io/project/omg_llava/

2405.17663 2024-10-02 cs.LG cs.CV 57%

Finding Shared Decodable Concepts and their Negations in the Brain

Cory Efird, Alex Murphy, Joel Zylberberg, Alona Fyshe

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

2409.19899 2024-10-01 cs.CV 57%

OpenKD: Opening Prompt Diversity for Zero- and Few-shot Keypoint Detection

Changsheng Lu, Zheyuan Liu, Piotr Koniusz

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments Accepted by ECCV 2024

2409.19696 2024-10-01 cs.LG cs.CV 57%

Vision-Language Models are Strong Noisy Label Detectors

Tong Wei, Hao-Tian Li, Chun-Shu Li, Jiang-Xin Shi, Yu-Feng Li, Min-Ling Zhang

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Accepted at NeurIPS 2024

2409.19232 2024-10-01 cs.CV 57%

TrojVLM: Backdoor Attack Against Vision Language Models

Weimin Lyu, Lu Pang, Tengfei Ma, Haibin Ling, Chao Chen

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments ECCV 2024

2407.08787 2024-09-27 cs.CV 57%

Data Adaptive Traceback for Vision-Language Foundation Models in Image Classification

Wenshuo Peng, Kaipeng Zhang, Yue Yang, Hao Zhang, Yu Qiao

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments 9 pages,4 figures

2409.15968 2024-09-25 cs.CV 57%

Adversarial Backdoor Defense in CLIP

Junhao Kuang, Siyuan Liang, Jiawei Liang, Kuanrong Liu, Xiaochun Cao

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

2406.06973 2024-09-24 cs.CV 57%

RWKV-CLIP: A Robust Vision-Language Representation Learner

Tiancheng Gu, Kaicheng Yang, Xiang An, Ziyong Feng, Dongnan Liu, Weidong Cai, Jiankang Deng

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments 14 pages, 10 figures, EMNLP2024 Main

2409.14083 2024-09-24 cs.CV 57%

SURf: Teaching Large Vision-Language Models to Selectively Utilize Retrieved Information

Jiashuo Sun, Jihai Zhang, Yucheng Zhou, Zhaochen Su, Xiaoye Qu, Yu Cheng

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments 19 pages, 9 tables, 11 figures

2409.13540 2024-09-23 cs.CV 57%

FullAnno: A Data Engine for Enhancing Image Comprehension of MLLMs

Jing Hao, Yuxiang Zhao, Song Chen, Yanpeng Sun, Qiang Chen, Gang Zhang, Kun Yao, Errui Ding, Jingdong Wang

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments 7 pages, 5 figures, 2 tables

2409.12612 2024-09-20 cs.CV 57%

Enhancing Perception of Key Changes in Remote Sensing Image Change Captioning

Cong Yang, Zuchao Li, Hongzan Jiao, Zhi Gao, Lefei Zhang

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

2409.12580 2024-09-20 cs.CV 57%

LLMs Can Check Their Own Results to Mitigate Hallucinations in Traffic Understanding Tasks

Malsha Ashani Mahawatta Dona, Beatriz Cabrero-Daniel, Yinan Yu, Christian Berger

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

Comments ICTSS 2024, 36th International Conference on Testing Software and Systems

1. 图文多模态 4651 篇

IMITATE: Clinical Prior Guided Hierarchical Vision-Language Pre-training

RE-tune: Incremental Fine Tuning of Biomedical Vision-Language Models for Multi-label Chest X-ray Classification

ADEM-VL: Adaptive and Embedded Fusion for Efficient Vision-Language Tuning

Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Models

Decomposing and Interpreting Image Representations via Text in ViTs Beyond CLIP

Performance of Gaussian Mixture Model Classifiers on Embedded Feature Spaces

Dual Prototype Evolving for Test-Time Generalization of Vision-Language Models

How to Determine the Preferred Image Distribution of a Black-Box Vision-Language Model?

Revisiting Few-Shot Object Detection with Vision-Language Models

CLIP-SCGI: Synthesized Caption-Guided Inversion for Person Re-Identification

EAVL: Explicitly Align Vision and Language for Referring Image Segmentation

Calibrated Cache Model for Few-Shot Vision-Language Model Adaptation

Open-RGBT: Open-vocabulary RGB-T Zero-shot Semantic Segmentation in Open-world Environments

Prompting DirectSAM for Semantic Contour Extraction in Remote Sensing Images

LG-VQ: Language-Guided Codebook Learning

AWT: Transferring Vision-Language Models via Augmentation, Weighting, and Transportation

AgriCLIP: Adapting CLIP for Agriculture and Livestock via Domain-Specialized Cross-Model Alignment

FastCLIP: A Suite of Optimization Techniques to Accelerate CLIP Training with Limited Resources

OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understanding

Finding Shared Decodable Concepts and their Negations in the Brain

OpenKD: Opening Prompt Diversity for Zero- and Few-shot Keypoint Detection

Vision-Language Models are Strong Noisy Label Detectors

TrojVLM: Backdoor Attack Against Vision Language Models

Data Adaptive Traceback for Vision-Language Foundation Models in Image Classification

Adversarial Backdoor Defense in CLIP

RWKV-CLIP: A Robust Vision-Language Representation Learner

SURf: Teaching Large Vision-Language Models to Selectively Utilize Retrieved Information

FullAnno: A Data Engine for Enhancing Image Comprehension of MLLMs

Enhancing Perception of Key Changes in Remote Sensing Image Change Captioning

LLMs Can Check Their Own Results to Mitigate Hallucinations in Traffic Understanding Tasks