多模态大模型 - arXivDaily 专题

2202.13562 2024-12-10 cs.CV eess.IV 57%

Name Your Style: An Arbitrary Artist-aware Image Style Transfer

Zhi-Song Liu, Li-Wen Wang, Wan-Chi Siu, Vicky Kalogeiton

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments 15 pages, 15 figures

2412.04429 2024-12-06 cs.CV cs.LG 57%

Grounding Descriptions in Images informs Zero-Shot Visual Recognition

Shaunak Halbe, Junjiao Tian, K J Joseph, James Seale Smith, Katherine Stevo, Vineeth N Balasubramanian, Zsolt Kira

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

2412.03927 2024-12-06 cs.CV cs.LG 57%

MegaCOIN: Enhancing Medium-Grained Color Perception for Vision-Language Models

Ming-Chang Chiu, Shicheng Wen, Pin-Yu Chen, Xuezhe Ma

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments 8 pages, 13 tables, 2 figures

2412.03324 2024-12-06 cs.CV 57%

A Stitch in Time Saves Nine: Small VLM is a Precise Guidance for Accelerating Large VLMs

Wangbo Zhao, Yizeng Han, Jiasheng Tang, Zhikai Li, Yibing Song, Kai Wang, Zhangyang Wang, Yang You

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

2406.09400 2024-12-05 cs.CV cs.LG 57%

Yo'LLaVA: Your Personalized Language and Vision Assistant

Thao Nguyen, Haotian Liu, Yuheng Li, Mu Cai, Utkarsh Ojha, Yong Jae Lee

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments NeurIPS 2024; Project page: https://thaoshibe.github.io/YoLLaVA

2412.02978 2024-12-05 cs.CV 57%

Progressive Vision-Language Prompt for Multi-Organ Multi-Class Cell Semantic Segmentation with Single Branch

Qing Zhang, Hang Guo, Siyuan Yang, Qingli Li, Yan Wang

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

2410.01544 2024-12-05 cs.CV 57%

Boosting Weakly-Supervised Referring Image Segmentation via Progressive Comprehension

Zaiquan Yang, Yuhao Liu, Jiaying Lin, Gerhard Hancke, Rynson W. H. Lau

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Accepted to NeurIPS2024

2412.01672 2024-12-03 cs.CV 57%

Gen-SIS: Generative Self-augmentation Improves Self-supervised Learning

Varun Belagali, Srikar Yellapragada, Alexandros Graikos, Saarthak Kapse, Zilinghan Li, Tarak Nath Nandi, Ravi K Madduri, Prateek Prasanna, Joel Saltz, Dimitris Samaras

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Webpage: https://histodiffusion.github.io/docs/publications/gensis

2412.01034 2024-12-03 cs.RO cs.CV cs.LG 57%

Quantization-Aware Imitation-Learning for Resource-Efficient Robotic Control

Seongmin Park, Hyungmin Kim, Wonseok Jeon, Juyoung Yang, Byeongwook Jeon, Yoonseon Oh, Jungwook Choi

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

2412.00890 2024-12-03 cs.CV 57%

Exploring Large Vision-Language Models for Robust and Efficient Industrial Anomaly Detection

Kun Qian, Tianyu Sun, Wenhong Wang

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV

Comments 14 pages

2411.11496 2024-12-02 cs.CL 57%

Safe + Safe = Unsafe? Exploring How Safe Images Can Be Exploited to Jailbreak Large Vision-Language Models

Chenhang Cui, Gelei Deng, An Zhang, Jingnan Zheng, Yicong Li, Lianli Gao, Tianwei Zhang, Tat-Seng Chua

专题命中图文多模态：multimodal(abstract)；分类 cs.CL

2403.02469 2024-12-02 cs.CV cs.LG 57%

Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review

Iryna Hartsock, Ghulam Rasool

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments 43 pages; paper edited and restructured

2409.14215 2024-11-26 cs.CV 57%

@Bench: Benchmarking Vision-Language Models for Human-centered Assistive Technology

Xin Jiang, Junwei Zheng, Ruiping Liu, Jiahang Li, Jiaming Zhang, Sven Matthiesen, Rainer Stiefelhagen

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

Comments Accepted by WACV 2025, project page: https://junweizheng93.github.io/publications/ATBench/ATBench.html

2411.15720 2024-11-26 cs.CV 57%

Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attacks

Peng Xie, Yequan Bie, Jianda Mao, Yangqiu Song, Yang Wang, Hao Chen, Kani Chen

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

2407.07315 2024-11-22 cs.CV 57%

CosmoCLIP: Generalizing Large Vision-Language Models for Astronomical Imaging

Raza Imam, Mohammed Talha Alam, Umaima Rahman, Mohsen Guizani, Fakhri Karray

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Accepted at SPAICE Conference, ECSAT, UK, 2024

2411.12785 2024-11-21 cs.CV 57%

Joint Vision-Language Social Bias Removal for CLIP

Haoyu Zhang, Yangyang Guo, Mohan Kankanhalli

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

2411.02074 2024-11-19 cs.CV 57%

GraphVL: Graph-Enhanced Semantic Modeling via Vision-Language Models for Generalized Class Discovery

Bhupendra Solanki, Ashwin Nair, Mainak Singha, Souradeep Mukhopadhyay, Ankit Jha, Biplab Banerjee

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Accepted in ACM ICVGIP 2024

2309.06799 2024-11-13 cs.AI physics.geo-ph 57%

When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence System

Hao Zhang, Jin-Jian Xu, Hong-Wei Cui, Lin Li, Yaowen Yang, Chao-Sheng Tang, Niklas Boers

专题命中图文多模态：multi-modal(abstract)；分类 cs.AI

Comments accpeted by IEEE Geoscience and Remote Sensing Magazine

Journal ref IEEE Geoscience and Remote Sensing Magazine, 2024

2410.12183 2024-11-13 cs.CV 57%

TransAgent: Transfer Vision-Language Foundation Models with Heterogeneous Agent Collaboration

Yiwei Guo, Shaobin Zhuang, Kunchang Li, Yu Qiao, Yali Wang

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments NeurIPS 2024

2411.04656 2024-11-08 cs.CV 57%

ICH-SCNet: Intracerebral Hemorrhage Segmentation and Prognosis Classification Network Using CLIP-guided SAM mechanism

Xinlei Yu, Ahmed Elazab, Ruiquan Ge, Hui Jin, Xinchen Jiang, Gangyong Jia, Qing Wu, Qinglei Shi, Changmiao Wang

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV

Comments 6 pages, 2 figures, 3 tables, published to BIBM 2024

2411.03313 2024-11-07 cs.CV 57%

Classification Done Right for Vision-Language Pre-Training

Zilong Huang, Qinghao Ye, Bingyi Kang, Jiashi Feng, Haoqi Fan

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments NeurIPS 2024

2411.00321 2024-11-06 cs.SD eess.AS 57%

MACE: Leveraging Audio for Evaluating Audio Captioning Systems

Satvik Dixit, Soham Deshmukh, Bhiksha Raj

专题命中图文多模态：multimodal(abstract)；分类 eess.AS

2411.01975 2024-11-05 cs.CV eess.IV 57%

SPECTRUM: Semantic Processing and Emotion-informed video-Captioning Through Retrieval and Understanding Modalities

Ehsan Faghihi, Mohammedreza Zarenejad, Ali-Asghar Beheshti Shirazi

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

2402.10376 2024-11-05 cs.LG cs.CV 57%

Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)

Usha Bhalla, Alex Oesterling, Suraj Srinivas, Flavio P. Calmon, Himabindu Lakkaraju

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments 25 pages, 15 figures, NeurIPS 2024. Code is provided at https://github.com/AI4LIFE-GROUP/SpLiCE

2411.01846 2024-11-05 cs.CV 57%

KptLLM: Unveiling the Power of Large Language Model for Keypoint Comprehension

Jie Yang, Wang Zeng, Sheng Jin, Lumin Xu, Wentao Liu, Chen Qian, Ruimao Zhang

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments NeurIPS 2024

2411.01494 2024-11-05 cs.CV 57%

Finding NeMo: Negative-mined Mosaic Augmentation for Referring Image Segmentation

Seongsu Ha, Chaeyun Kim, Donghwa Kim, Junho Lee, Sangho Lee, Joonseok Lee

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments Accepted at ECCV 2024. Project page: https://dddonghwa.github.io/NeMo/

2411.00828 2024-11-05 cs.CV cs.LG 57%

Dreaming Out Loud: A Self-Synthesis Approach For Training Vision-Language Models With Developmentally Plausible Data

Badr AlKhamissi, Yingtian Tang, Abdülkadir Gökce, Johannes Mehrer, Martin Schrimpf

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments Accepted to BabyLM Challenge at CoNLL 2024

2407.08739 2024-11-05 cs.CV 57%

MAVIS: Mathematical Visual Instruction Tuning with an Automatic Data Engine

Renrui Zhang, Xinyu Wei, Dongzhi Jiang, Ziyu Guo, Shicheng Li, Yichi Zhang, Chengzhuo Tong, Jiaming Liu, Aojun Zhou, Bin Wei, Shanghang Zhang, Peng Gao, Chunyuan Li, Hongsheng Li

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

Comments WData and Models will be released at https://github.com/ZrrSkywalker/MAVIS

2410.20883 2024-11-04 cs.CV 57%

Improving Generalization in Visual Reasoning via Self-Ensemble

Tien-Huy Nguyen, Quang-Khai Tran, Anh-Tuan Quang-Hoang

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

2403.02506 2024-10-31 cs.CV cs.LG 57%

Differentially Private Representation Learning via Image Captioning

Tom Sander, Yaodong Yu, Maziar Sanjabi, Alain Durmus, Yi Ma, Kamalika Chaudhuri, Chuan Guo

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments Accepted and presented at ICML 2024

1. 图文多模态 4651 篇

Name Your Style: An Arbitrary Artist-aware Image Style Transfer

Grounding Descriptions in Images informs Zero-Shot Visual Recognition

MegaCOIN: Enhancing Medium-Grained Color Perception for Vision-Language Models

A Stitch in Time Saves Nine: Small VLM is a Precise Guidance for Accelerating Large VLMs

Yo'LLaVA: Your Personalized Language and Vision Assistant

Progressive Vision-Language Prompt for Multi-Organ Multi-Class Cell Semantic Segmentation with Single Branch

Boosting Weakly-Supervised Referring Image Segmentation via Progressive Comprehension

Gen-SIS: Generative Self-augmentation Improves Self-supervised Learning

Quantization-Aware Imitation-Learning for Resource-Efficient Robotic Control

Exploring Large Vision-Language Models for Robust and Efficient Industrial Anomaly Detection

Safe + Safe = Unsafe? Exploring How Safe Images Can Be Exploited to Jailbreak Large Vision-Language Models

Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review

@Bench: Benchmarking Vision-Language Models for Human-centered Assistive Technology

Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attacks

CosmoCLIP: Generalizing Large Vision-Language Models for Astronomical Imaging

Joint Vision-Language Social Bias Removal for CLIP

GraphVL: Graph-Enhanced Semantic Modeling via Vision-Language Models for Generalized Class Discovery

When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence System

TransAgent: Transfer Vision-Language Foundation Models with Heterogeneous Agent Collaboration

ICH-SCNet: Intracerebral Hemorrhage Segmentation and Prognosis Classification Network Using CLIP-guided SAM mechanism

Classification Done Right for Vision-Language Pre-Training

MACE: Leveraging Audio for Evaluating Audio Captioning Systems

SPECTRUM: Semantic Processing and Emotion-informed video-Captioning Through Retrieval and Understanding Modalities

Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)

KptLLM: Unveiling the Power of Large Language Model for Keypoint Comprehension

Finding NeMo: Negative-mined Mosaic Augmentation for Referring Image Segmentation

Dreaming Out Loud: A Self-Synthesis Approach For Training Vision-Language Models With Developmentally Plausible Data

MAVIS: Mathematical Visual Instruction Tuning with an Automatic Data Engine

Improving Generalization in Visual Reasoning via Self-Ensemble

Differentially Private Representation Learning via Image Captioning