多模态大模型 - arXivDaily 专题

2412.00890 2024-12-03 cs.CV 57%

Exploring Large Vision-Language Models for Robust and Efficient Industrial Anomaly Detection

Kun Qian, Tianyu Sun, Wenhong Wang

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV

Comments 14 pages

2411.11496 2024-12-02 cs.CL 57%

Safe + Safe = Unsafe? Exploring How Safe Images Can Be Exploited to Jailbreak Large Vision-Language Models

Chenhang Cui, Gelei Deng, An Zhang, Jingnan Zheng, Yicong Li, Lianli Gao, Tianwei Zhang, Tat-Seng Chua

专题命中图文多模态：multimodal(abstract)；分类 cs.CL

2403.02469 2024-12-02 cs.CV cs.LG 57%

Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review

Iryna Hartsock, Ghulam Rasool

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments 43 pages; paper edited and restructured

2409.14215 2024-11-26 cs.CV 57%

@Bench: Benchmarking Vision-Language Models for Human-centered Assistive Technology

Xin Jiang, Junwei Zheng, Ruiping Liu, Jiahang Li, Jiaming Zhang, Sven Matthiesen, Rainer Stiefelhagen

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

Comments Accepted by WACV 2025, project page: https://junweizheng93.github.io/publications/ATBench/ATBench.html

2411.15720 2024-11-26 cs.CV 57%

Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attacks

Peng Xie, Yequan Bie, Jianda Mao, Yangqiu Song, Yang Wang, Hao Chen, Kani Chen

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

2407.07315 2024-11-22 cs.CV 57%

CosmoCLIP: Generalizing Large Vision-Language Models for Astronomical Imaging

Raza Imam, Mohammed Talha Alam, Umaima Rahman, Mohsen Guizani, Fakhri Karray

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Accepted at SPAICE Conference, ECSAT, UK, 2024

2411.12785 2024-11-21 cs.CV 57%

Joint Vision-Language Social Bias Removal for CLIP

Haoyu Zhang, Yangyang Guo, Mohan Kankanhalli

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

2411.02074 2024-11-19 cs.CV 57%

GraphVL: Graph-Enhanced Semantic Modeling via Vision-Language Models for Generalized Class Discovery

Bhupendra Solanki, Ashwin Nair, Mainak Singha, Souradeep Mukhopadhyay, Ankit Jha, Biplab Banerjee

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Accepted in ACM ICVGIP 2024

2309.06799 2024-11-13 cs.AI physics.geo-ph 57%

When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence System

Hao Zhang, Jin-Jian Xu, Hong-Wei Cui, Lin Li, Yaowen Yang, Chao-Sheng Tang, Niklas Boers

专题命中图文多模态：multi-modal(abstract)；分类 cs.AI

Comments accpeted by IEEE Geoscience and Remote Sensing Magazine

Journal ref IEEE Geoscience and Remote Sensing Magazine, 2024

2410.12183 2024-11-13 cs.CV 57%

TransAgent: Transfer Vision-Language Foundation Models with Heterogeneous Agent Collaboration

Yiwei Guo, Shaobin Zhuang, Kunchang Li, Yu Qiao, Yali Wang

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments NeurIPS 2024

2411.04656 2024-11-08 cs.CV 57%

ICH-SCNet: Intracerebral Hemorrhage Segmentation and Prognosis Classification Network Using CLIP-guided SAM mechanism

Xinlei Yu, Ahmed Elazab, Ruiquan Ge, Hui Jin, Xinchen Jiang, Gangyong Jia, Qing Wu, Qinglei Shi, Changmiao Wang

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV

Comments 6 pages, 2 figures, 3 tables, published to BIBM 2024

2411.03313 2024-11-07 cs.CV 57%

Classification Done Right for Vision-Language Pre-Training

Zilong Huang, Qinghao Ye, Bingyi Kang, Jiashi Feng, Haoqi Fan

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments NeurIPS 2024

2411.00321 2024-11-06 cs.SD eess.AS 57%

MACE: Leveraging Audio for Evaluating Audio Captioning Systems

Satvik Dixit, Soham Deshmukh, Bhiksha Raj

专题命中图文多模态：multimodal(abstract)；分类 eess.AS

2411.01975 2024-11-05 cs.CV eess.IV 57%

SPECTRUM: Semantic Processing and Emotion-informed video-Captioning Through Retrieval and Understanding Modalities

Ehsan Faghihi, Mohammedreza Zarenejad, Ali-Asghar Beheshti Shirazi

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

2402.10376 2024-11-05 cs.LG cs.CV 57%

Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)

Usha Bhalla, Alex Oesterling, Suraj Srinivas, Flavio P. Calmon, Himabindu Lakkaraju

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments 25 pages, 15 figures, NeurIPS 2024. Code is provided at https://github.com/AI4LIFE-GROUP/SpLiCE

2411.01846 2024-11-05 cs.CV 57%

KptLLM: Unveiling the Power of Large Language Model for Keypoint Comprehension

Jie Yang, Wang Zeng, Sheng Jin, Lumin Xu, Wentao Liu, Chen Qian, Ruimao Zhang

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments NeurIPS 2024

2411.01494 2024-11-05 cs.CV 57%

Finding NeMo: Negative-mined Mosaic Augmentation for Referring Image Segmentation

Seongsu Ha, Chaeyun Kim, Donghwa Kim, Junho Lee, Sangho Lee, Joonseok Lee

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments Accepted at ECCV 2024. Project page: https://dddonghwa.github.io/NeMo/

2411.00828 2024-11-05 cs.CV cs.LG 57%

Dreaming Out Loud: A Self-Synthesis Approach For Training Vision-Language Models With Developmentally Plausible Data

Badr AlKhamissi, Yingtian Tang, Abdülkadir Gökce, Johannes Mehrer, Martin Schrimpf

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments Accepted to BabyLM Challenge at CoNLL 2024

2407.08739 2024-11-05 cs.CV 57%

MAVIS: Mathematical Visual Instruction Tuning with an Automatic Data Engine

Renrui Zhang, Xinyu Wei, Dongzhi Jiang, Ziyu Guo, Shicheng Li, Yichi Zhang, Chengzhuo Tong, Jiaming Liu, Aojun Zhou, Bin Wei, Shanghang Zhang, Peng Gao, Chunyuan Li, Hongsheng Li

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

Comments WData and Models will be released at https://github.com/ZrrSkywalker/MAVIS

2410.20883 2024-11-04 cs.CV 57%

Improving Generalization in Visual Reasoning via Self-Ensemble

Tien-Huy Nguyen, Quang-Khai Tran, Anh-Tuan Quang-Hoang

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

2403.02506 2024-10-31 cs.CV cs.LG 57%

Differentially Private Representation Learning via Image Captioning

Tom Sander, Yaodong Yu, Maziar Sanjabi, Alain Durmus, Yi Ma, Kamalika Chaudhuri, Chuan Guo

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments Accepted and presented at ICML 2024

2310.07355 2024-10-31 cs.CV cs.LG 57%

IMITATE: Clinical Prior Guided Hierarchical Vision-Language Pre-training

Che Liu, Sibo Cheng, Miaojing Shi, Anand Shah, Wenjia Bai, Rossella Arcucci

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV

Comments Accepted by TMI2024

2410.17827 2024-10-24 cs.AI 57%

RE-tune: Incremental Fine Tuning of Biomedical Vision-Language Models for Multi-label Chest X-ray Classification

Marco Mistretta, Andrew D. Bagdanov

专题命中图文多模态：multimodal(abstract)；分类 cs.AI

Comments Accepted for publication at Medical Imaging meets NeurIPS (NeurIPS23)

2410.17779 2024-10-24 cs.CV 57%

ADEM-VL: Adaptive and Embedded Fusion for Efficient Vision-Language Tuning

Zhiwei Hao, Jianyuan Guo, Li Shen, Yong Luo, Han Hu, Yonggang Wen

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

2405.15574 2024-10-24 cs.CV 57%

Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Models

Byung-Kwan Lee, Chae Won Kim, Beomchan Park, Yong Man Ro

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments Code is available in https://github.com/ByungKwanLee/Meteor

2406.01583 2024-10-22 cs.CV cs.LG 57%

Decomposing and Interpreting Image Representations via Text in ViTs Beyond CLIP

Sriram Balasubramanian, Samyadeep Basu, Soheil Feizi

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments NeurIPS 2024, 31 pages, 15 figures

2410.13421 2024-10-18 cs.CV 57%

Performance of Gaussian Mixture Model Classifiers on Embedded Feature Spaces

Jeremy Chopin, Rozenn Dahyot

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments 8 pages

2410.12790 2024-10-17 cs.CV cs.LG 57%

Dual Prototype Evolving for Test-Time Generalization of Vision-Language Models

Ce Zhang, Simon Stepputtis, Katia Sycara, Yaqi Xie

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2024. Project page: https://zhangce01.github.io/DPE-CLIP

2409.02253 2024-10-17 cs.CV 57%

How to Determine the Preferred Image Distribution of a Black-Box Vision-Language Model?

Saeid Asgari Taghanaki, Joseph Lambourne, Alana Mongkhounsavath

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

Comments Accepted to NeurIPS 2024, Safe Generative AI

2312.14494 2024-10-15 cs.CV 57%

Revisiting Few-Shot Object Detection with Vision-Language Models

Anish Madan, Neehar Peri, Shu Kong, Deva Ramanan

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

Comments The first two authors contributed equally. This work has been accepted to the Neural Information Processing Systems (NeurIPS) 2024 Datasets & Benchmark Track

1. 图文多模态 4672 篇

Exploring Large Vision-Language Models for Robust and Efficient Industrial Anomaly Detection

Safe + Safe = Unsafe? Exploring How Safe Images Can Be Exploited to Jailbreak Large Vision-Language Models

Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review

@Bench: Benchmarking Vision-Language Models for Human-centered Assistive Technology

Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attacks

CosmoCLIP: Generalizing Large Vision-Language Models for Astronomical Imaging

Joint Vision-Language Social Bias Removal for CLIP

GraphVL: Graph-Enhanced Semantic Modeling via Vision-Language Models for Generalized Class Discovery

When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence System

TransAgent: Transfer Vision-Language Foundation Models with Heterogeneous Agent Collaboration

ICH-SCNet: Intracerebral Hemorrhage Segmentation and Prognosis Classification Network Using CLIP-guided SAM mechanism

Classification Done Right for Vision-Language Pre-Training

MACE: Leveraging Audio for Evaluating Audio Captioning Systems

SPECTRUM: Semantic Processing and Emotion-informed video-Captioning Through Retrieval and Understanding Modalities

Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)

KptLLM: Unveiling the Power of Large Language Model for Keypoint Comprehension

Finding NeMo: Negative-mined Mosaic Augmentation for Referring Image Segmentation

Dreaming Out Loud: A Self-Synthesis Approach For Training Vision-Language Models With Developmentally Plausible Data

MAVIS: Mathematical Visual Instruction Tuning with an Automatic Data Engine

Improving Generalization in Visual Reasoning via Self-Ensemble

Differentially Private Representation Learning via Image Captioning

IMITATE: Clinical Prior Guided Hierarchical Vision-Language Pre-training

RE-tune: Incremental Fine Tuning of Biomedical Vision-Language Models for Multi-label Chest X-ray Classification

ADEM-VL: Adaptive and Embedded Fusion for Efficient Vision-Language Tuning

Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Models

Decomposing and Interpreting Image Representations via Text in ViTs Beyond CLIP

Performance of Gaussian Mixture Model Classifiers on Embedded Feature Spaces

Dual Prototype Evolving for Test-Time Generalization of Vision-Language Models

How to Determine the Preferred Image Distribution of a Black-Box Vision-Language Model?

Revisiting Few-Shot Object Detection with Vision-Language Models