多模态大模型 - arXivDaily 专题

2412.19650 2024-12-30 cs.CV cs.LG 57%

Toward Modality Gap: Vision Prototype Learning for Weakly-supervised Semantic Segmentation with CLIP

Zhongxing Xu, Feilong Tang, Zhe Chen, Yingxue Su, Zhiyi Zhao, Ge Zhang, Jionglong Su, Zongyuan Ge

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV

2305.14014 2024-12-25 cs.CV 57%

CLIP4STR: A Simple Baseline for Scene Text Recognition with Pre-trained Vision-Language Model

Shuai Zhao, Ruijie Quan, Linchao Zhu, Yi Yang

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV

Comments Accepted by T-IP. A PyTorch re-implementation is at https://github.com/VamosC/CLIP4STR (Credit on GitHub@VamosC)

2412.16418 2024-12-24 cs.CV 57%

Revisiting MLLMs: An In-Depth Analysis of Image Classification Abilities

Huan Liu, Lingyu Xiao, Jiangjiang Liu, Xiaofan Li, Ze Feng, Sen Yang, Jingdong Wang

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

2401.15947 2024-12-24 cs.CV 57%

MoE-LLaVA: Mixture of Experts for Large Vision-Language Models

Bin Lin, Zhenyu Tang, Yang Ye, Jinfa Huang, Junwu Zhang, Yatian Pang, Peng Jin, Munan Ning, Jiebo Luo, Li Yuan

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

Comments update author

2403.20213 2024-12-20 cs.CV 57%

VHM: Versatile and Honest Vision Language Model for Remote Sensing Image Analysis

Chao Pang, Xingxing Weng, Jiang Wu, Jiayu Li, Yi Liu, Jiaxing Sun, Weijia Li, Shuai Wang, Litong Feng, Gui-Song Xia, Conghui He

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Equal contribution: Chao Pang, Xingxing Weng, Jiang Wu; Corresponding author: Gui-Song Xia, Conghui He

2412.14515 2024-12-20 cs.AI cs.PL 57%

Relational Programming with Foundation Models

Ziyang Li, Jiani Huang, Jason Liu, Felix Zhu, Eric Zhao, William Dodds, Neelay Velingker, Rajeev Alur, Mayur Naik

专题命中图文多模态：multi-modal(abstract)；分类 cs.AI

2412.12793 2024-12-18 cs.CV 57%

CRoF: CLIP-based Robust Few-shot Learning on Noisy Labels

Shizhuo Deng, Bowen Han, Jiaqi Chen, Hao Wang, Dongyue Chen, Tong Jia

专题命中图文多模态：image-text(abstract)；分类 cs.CV

2408.09441 2024-12-17 cs.CV 57%

CLIP-CID: Efficient CLIP Distillation via Cluster-Instance Discrimination

Kaicheng Yang, Tiancheng Gu, Xiang An, Haiqiang Jiang, Xiangzi Dai, Ziyong Feng, Weidong Cai, Jiankang Deng

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Accepted by AAAI 2025

2406.18844 2024-12-17 cs.CV 57%

Revisiting Backdoor Attacks against Large Vision-Language Models from Domain Shift

Siyuan Liang, Jiawei Liang, Tianyu Pang, Chao Du, Aishan Liu, Mingli Zhu, Xiaochun Cao, Dacheng Tao

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments 11 pages, 9 figures

2412.10292 2024-12-16 cs.CV 57%

Prompt-Guided Mask Proposal for Two-Stage Open-Vocabulary Segmentation

Yu-Jhe Li, Xinyang Zhang, Kun Wan, Lantao Yu, Ajinkya Kale, Xin Lu

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

Comments 17 pages. Work done during 2023 summer and has been released

2412.09936 2024-12-16 cs.CV 57%

CaLoRAify: Calorie Estimation with Visual-Text Pairing and LoRA-Driven Visual Language Models

Dongyu Yao, Keling Yao, Junhong Zhou, Yinghao Zhang

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Disclaimer: This work is part of a course project and reflects ongoing exploration in the field of vision-language models and calorie estimation. Findings and conclusions are subject to further validation and refinement

2412.09875 2024-12-16 cs.CV 57%

Selective State Space Memory for Large Vision-Language Models

Chee Ng, Yuen Fung

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

2412.07779 2024-12-12 cs.NE cs.AI 57%

Evolution of Thought: Diverse and High-Quality Reasoning via Multi-Objective Optimization

Biqing Qi, Zhouyi Qian, Yiang Luo, Junqi Gao, Dong Li, Kaiyan Zhang, Bowen Zhou

专题命中图文多模态：multi-modal(abstract)；分类 cs.AI

2412.06172 2024-12-12 cs.CV 57%

Robust Noisy Correspondence Learning via Self-Drop and Dual-Weight

Fan Liu, Chenwei Dong, Chuanyi Zhang, Hualiang Zhou, Jun Zhou

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV

2412.07518 2024-12-11 cs.CV 57%

Hallucination Elimination and Semantic Enhancement Framework for Vision-Language Models in Traffic Scenarios

Jiaqi Fan, Jianhua Wu, Hongqing Chu, Quanbo Ge, Bingzhao Gao

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

2412.07247 2024-12-11 cs.CV 57%

Driving with InternVL: Oustanding Champion in the Track on Driving with Language of the Autonomous Grand Challenge at CVPR 2024

Jiahan Li, Zhiqi Li, Tong Lu

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

2412.07226 2024-12-11 cs.CV 57%

Attention Head Purification: A New Perspective to Harness CLIP for Domain Generalization

Yingfan Wang, Guoliang Kang

专题命中图文多模态：image-text(abstract)；分类 cs.CV

2412.05679 2024-12-11 cs.CV 57%

RSUniVLM: A Unified Vision Language Model for Remote Sensing via Granularity-oriented Mixture of Experts

Xu Liu, Zhouhui Lian

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

2409.16278 2024-12-10 cs.CV 57%

Adapting Vision-Language Model with Fine-grained Semantics for Open-Vocabulary Segmentation

Yong Xien Chng, Xuchong Qiu, Yizeng Han, Kai Ding, Wan Ding, Gao Huang

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments 13 pages, 10 figures

2311.12668 2024-12-10 cs.AI cs.CE 57%

From Concept to Manufacturing: Evaluating Vision-Language Models for Engineering Design

Cyril Picard, Kristen M. Edwards, Anna C. Doris, Brandon Man, Giorgio Giannone, Md Ferdous Alam, Faez Ahmed

专题命中图文多模态：multimodal(abstract)；分类 cs.AI

2311.18291 2024-12-10 cs.CV 57%

TLDR: Text Based Last-layer Retraining for Debiasing Image Classifiers

Juhyeon Park, Seokhyeon Jeong, Taesup Moon

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments WACV 2025

2202.13562 2024-12-10 cs.CV eess.IV 57%

Name Your Style: An Arbitrary Artist-aware Image Style Transfer

Zhi-Song Liu, Li-Wen Wang, Wan-Chi Siu, Vicky Kalogeiton

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments 15 pages, 15 figures

2412.04429 2024-12-06 cs.CV cs.LG 57%

Grounding Descriptions in Images informs Zero-Shot Visual Recognition

Shaunak Halbe, Junjiao Tian, K J Joseph, James Seale Smith, Katherine Stevo, Vineeth N Balasubramanian, Zsolt Kira

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

2412.03927 2024-12-06 cs.CV cs.LG 57%

MegaCOIN: Enhancing Medium-Grained Color Perception for Vision-Language Models

Ming-Chang Chiu, Shicheng Wen, Pin-Yu Chen, Xuezhe Ma

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments 8 pages, 13 tables, 2 figures

2412.03324 2024-12-06 cs.CV 57%

A Stitch in Time Saves Nine: Small VLM is a Precise Guidance for Accelerating Large VLMs

Wangbo Zhao, Yizeng Han, Jiasheng Tang, Zhikai Li, Yibing Song, Kai Wang, Zhangyang Wang, Yang You

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

2406.09400 2024-12-05 cs.CV cs.LG 57%

Yo'LLaVA: Your Personalized Language and Vision Assistant

Thao Nguyen, Haotian Liu, Yuheng Li, Mu Cai, Utkarsh Ojha, Yong Jae Lee

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments NeurIPS 2024; Project page: https://thaoshibe.github.io/YoLLaVA

2412.02978 2024-12-05 cs.CV 57%

Progressive Vision-Language Prompt for Multi-Organ Multi-Class Cell Semantic Segmentation with Single Branch

Qing Zhang, Hang Guo, Siyuan Yang, Qingli Li, Yan Wang

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

2410.01544 2024-12-05 cs.CV 57%

Boosting Weakly-Supervised Referring Image Segmentation via Progressive Comprehension

Zaiquan Yang, Yuhao Liu, Jiaying Lin, Gerhard Hancke, Rynson W. H. Lau

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Accepted to NeurIPS2024

2412.01672 2024-12-03 cs.CV 57%

Gen-SIS: Generative Self-augmentation Improves Self-supervised Learning

Varun Belagali, Srikar Yellapragada, Alexandros Graikos, Saarthak Kapse, Zilinghan Li, Tarak Nath Nandi, Ravi K Madduri, Prateek Prasanna, Joel Saltz, Dimitris Samaras

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Webpage: https://histodiffusion.github.io/docs/publications/gensis

2412.01034 2024-12-03 cs.RO cs.CV cs.LG 57%

Quantization-Aware Imitation-Learning for Resource-Efficient Robotic Control

Seongmin Park, Hyungmin Kim, Wonseok Jeon, Juyoung Yang, Byeongwook Jeon, Yoonseon Oh, Jungwook Choi

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

1. 图文多模态 4672 篇

Toward Modality Gap: Vision Prototype Learning for Weakly-supervised Semantic Segmentation with CLIP

CLIP4STR: A Simple Baseline for Scene Text Recognition with Pre-trained Vision-Language Model

Revisiting MLLMs: An In-Depth Analysis of Image Classification Abilities

MoE-LLaVA: Mixture of Experts for Large Vision-Language Models

VHM: Versatile and Honest Vision Language Model for Remote Sensing Image Analysis

Relational Programming with Foundation Models

CRoF: CLIP-based Robust Few-shot Learning on Noisy Labels

CLIP-CID: Efficient CLIP Distillation via Cluster-Instance Discrimination

Revisiting Backdoor Attacks against Large Vision-Language Models from Domain Shift

Prompt-Guided Mask Proposal for Two-Stage Open-Vocabulary Segmentation

CaLoRAify: Calorie Estimation with Visual-Text Pairing and LoRA-Driven Visual Language Models

Selective State Space Memory for Large Vision-Language Models

Evolution of Thought: Diverse and High-Quality Reasoning via Multi-Objective Optimization

Robust Noisy Correspondence Learning via Self-Drop and Dual-Weight

Hallucination Elimination and Semantic Enhancement Framework for Vision-Language Models in Traffic Scenarios

Driving with InternVL: Oustanding Champion in the Track on Driving with Language of the Autonomous Grand Challenge at CVPR 2024

Attention Head Purification: A New Perspective to Harness CLIP for Domain Generalization

RSUniVLM: A Unified Vision Language Model for Remote Sensing via Granularity-oriented Mixture of Experts

Adapting Vision-Language Model with Fine-grained Semantics for Open-Vocabulary Segmentation

From Concept to Manufacturing: Evaluating Vision-Language Models for Engineering Design

TLDR: Text Based Last-layer Retraining for Debiasing Image Classifiers

Name Your Style: An Arbitrary Artist-aware Image Style Transfer

Grounding Descriptions in Images informs Zero-Shot Visual Recognition

MegaCOIN: Enhancing Medium-Grained Color Perception for Vision-Language Models

A Stitch in Time Saves Nine: Small VLM is a Precise Guidance for Accelerating Large VLMs

Yo'LLaVA: Your Personalized Language and Vision Assistant

Progressive Vision-Language Prompt for Multi-Organ Multi-Class Cell Semantic Segmentation with Single Branch

Boosting Weakly-Supervised Referring Image Segmentation via Progressive Comprehension

Gen-SIS: Generative Self-augmentation Improves Self-supervised Learning

Quantization-Aware Imitation-Learning for Resource-Efficient Robotic Control