Vision-Language Models (VLMs), such as CLIP, have significantly advanced zero-shot image recognition. However, their performance remains limited by suboptimal prompt engineering and poor adaptability to target classes. While recent methods attempt to improve prompts through diverse class descriptions, they often rely on heuristic designs, lack versatility, and are vulnerable to outlier prompts. This paper enhances prompt by incorporating class-specific concepts. By treating concepts as latent variables, we rethink zero-shot image classification from a Bayesian perspective, casting prediction as marginalization over the concept space, where each concept is weighted by a prior and a test-image conditioned likelihood. This formulation underscores the importance of both a well-structured concept proposal distribution and the refinement of concept priors. To construct an expressive and efficient proposal distribution, we introduce a multi-stage concept synthesis pipeline driven by LLMs to generate discriminative and compositional concepts, followed by a Determinantal Point Process to enforce diversity. To mitigate the influence of outlier concepts, we propose a training-free, adaptive soft-trim likelihood, which attenuates their impact in a single forward pass. We further provide robustness guarantees and derive multi-class excess risk bounds for our framework. Extensive experiments demonstrate that our method consistently outperforms state-of-the-art approaches, validating its effectiveness in zero-shot image classification. Our code is available at https://github.com/less-and-less-bugs/CGBC.
In the Eye of MLLM: Benchmarking Egocentric Video Intent Understanding with Gaze-Guided Prompting
State Key Laboratory of VR Technology and Systems, School of CSE, Beihang University（虚拟现实技术与系统国家重点实验室，北京航空航天大学计算机科学与工程学院）；College of AI, Tsinghua University（清华大学人工智能学院）
GRASP: Guided Region-Aware Sparse Prompting for Adapting MLLMs to Remote Sensing
School of Computing, Kyung Hee University（京畿大学计算机学院）；School of Information and Software Engineering, University of Electronic Science and Technology of China（电子科技大学信息与软件工程学院）；School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；College of Computer Science and Information Engineering, Harbin Normal University（哈尔滨师范大学计算机科学与信息工程学院）；College of Professional and Continuing Education, The Hong Kong Polytechnic University（香港理工大学专业及继续教育学院）；School of Computer Science and Technology, Tongji University（同济大学计算机科学与技术学院）
Despite the remarkable progress of image captioning, existing captioners typically lack the controllable capability to generate desired image captions, e.g., describing the image in a rough or detailed manner, in a factual or emotional view, etc. In this paper, we show that a unified model is qualified to perform well in diverse domains and freely switch among multiple styles. Such a controllable capability is achieved by embedding the prompt learning into the image captioning framework. To be specific, we design a set of prompts to fine-tune the pre-trained image captioner. These prompts allo…
Prophet: Prompting Large Language Models with Complementary Answer Heuristics for Knowledge-based Visual Question Answering
Key Laboratory of Complex Systems Modeling and Simulation, the School of Computer Science, Hangzhou Dianzi University（复杂系统建模与仿真重点实验室、计算机科学学院、杭州电子大学）；HDU-ITMO Joint Institute, Hangzhou Dianzi University（杭州电子大学-ITMO联合学院）；School of Computer Science and Information Engineering, Hefei University of Technology（计算机科学与信息工程学院、合肥工业大学）；School of Intelligence Science and Engineering, Harbin Institute of Technology, Shenzhen（智能科学与工程学院、哈尔滨工业大学深圳校区）
Interpretable Few-Shot Retinal Disease Diagnosis with Concept-Guided Prompting of Vision-Language Models
Monash University（莫纳什大学）；Centre for Eye Research Australia（澳大利亚眼科研究中心）；Royal Victorian Eye and Ear Hospital（皇家维多利亚眼耳医院）；The University of Melbourne（墨尔本大学）；The Hong Kong Polytechnic University（香港理工大学）；Centre for Eye and Vision Research（眼与视觉研究中心）；Indian Institute of Technology Bombay（印度理工学院孟买分校）；Airdoc-Monash Research Lab（Airdoc-莫纳什研究实验室）
Unleashing Spatial Reasoning in Multimodal Large Language Models via Textual Representation Guided Reasoning
College of AI, Tsinghua University, Beijing, China（清华大学人工智能学院，北京，中国）；Shanghai Artificial Intelligence Laboratory, Shanghai, China（上海人工智能实验室，上海，中国）；The University of Tokyo, Tokyo, Japan（东京大学，东京，日本）
We propose a novel approach for dynamic negative prompting in diffusion models that leverages Vision-Language Models (VLMs) to adaptively generate negative prompts during the denoising process. Unlike traditional Negative Prompting methods that use fixed negative prompts, our method generates intermediate image predictions at specific denoising steps and queries a VLM to produce contextually appropriate negative prompts. We evaluate our approach on various benchmark datasets and demonstrate the trade-offs between negative guidance strength and text-image alignment.
Foundation Model-guided Iteratively Prompting and Pseudo-Labeling for Partially Labeled Medical Image Segmentation
Department of Biomedical Engineering, Columbia University（哥伦比亚大学生物医学工程系）；Department of Radiation Oncology, Columbia University Irving Medical Center（哥伦比亚大学欧文医学中心放射肿瘤科）；Data Science Institute, Columbia University（哥伦比亚大学数据科学研究所）；Herbert Irving Comprehensive Cancer Center, Columbia University（哥伦比亚大学赫伯特·欧文综合癌症中心）；Department of Radiation Oncology, Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院放射肿瘤科）
SAM2-SGP: Enhancing SAM2 for Medical Image Segmentation via Support-Set Guided Prompting
J. Crayton Pruitt Family Department of Biomedical Engineering University of Florida；Department of Electrical \& Computer Engineering University of Florida
Taiyuan University of Technology（太原理工大学）；University of Science and Technology of China（中国科学技术大学）；Second Hospital of Shanxi Medical University（山西医科大学第二医院）
Prompting Video-Language Foundation Models with Domain-specific Fine-grained Heuristics for Video Question Answering
School of Information Science and Technology, Hangzhou Normal University（杭州师范大学信息科学与技术学院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）；School of Computer Science and Technology, Hangzhou Dianzi University（杭州电子科技大学计算机科学与技术学院）
CLIP-DR: Textual Knowledge-Guided Diabetic Retinopathy Grading with Ranking-aware Prompting
University of Exeter（埃克塞特大学）；University of Liverpool（利物浦大学）；Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Sciences（中国科学院宁波材料技术与工程研究所）；Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局高性能计算研究所）
Medical Visual Prompting (MVP): A Unified Framework for Versatile and High-Quality Medical Image Segmentation
School of Automation, Guangdong University of Technology（广东工业大学自动化学院）；School of Computer Science and Technology, Guangdong University of Technology（广东工业大学计算机科学与技术学院）；School of Information Science and Technology, Guangdong University of Foreign Studies（广东外语外贸大学信息科学与技术学院）；Faculty of Science and Technology, University of Macau（澳门大学科技学院）；The First Affiliated Hospital of Sun Yat-Sen University（中山大学附属第一医院）；Sun Yat-sen University Cancer Center（中山大学肿瘤防治中心）；South China Hospital, Shenzhen University（深圳大学华南医院）
Task-Guided Prompting for Unified Remote Sensing Image Restoration
School of Electronic and Information Engineering, Ningbo University of Technology（宁波工程学院电子与信息工程学院）；University of Exeter（埃克塞特大学）；Engineering Research Center of Intelligent Finance, Ministry of Education（教育部智能金融工程研究中心）；School of Computing and Artificial Intelligence, Southwestern University of Finance and Economics（西南财经大学计算机与人工智能学院）
Recent progress has shown that large-scale pre-training using contrastive image-text pairs can be a promising alternative for high-quality visual representation learning from natural language supervision. Benefiting from a broader source of supervision, this new paradigm exhibits impressive transferability to downstream classification tasks and datasets. However, the problem of transferring the knowledge learned from image-text pairs to more complex dense prediction tasks has barely been visited. In this work, we present a new framework for dense prediction by implicitly and explicitly leverag…
PSMP-CLIP: Patch-Prompt SAM and Multi-Semantic Prompting for CLIP-Based Zero-Shot Anomaly Detection
Harbin Engineering University（哈尔滨工程大学）；Key Laboratory of Advanced Marine Communication and Information Technology（先进海洋通信与信息技术重点实验室）；University of Toronto（多伦多大学）；Kanagawa University（神奈川大学）
AuralSAM2: Enabling SAM2 Hear Through Pyramid Audio-Visual Feature Prompting
Department of Engineering Science, University of Oxford（牛津大学工程科学系）；Australian Institute for Machine Learning, Adelaide University（阿德莱德大学人工智能研究所）；Stanford University（斯坦福大学）；University of Central Florida（佛罗里达中央大学）；University of Surrey（萨里大学）
Prompt as Free Lunch: Enhancing Diversity in Source-Free Cross-domain Few-shot Learning through Semantic-Guided Prompting
College of Computer and Data Science, Fuzhou University（福州大学计算机与数据科学学院）；Zhejiang University of Technology（浙江工业大学）；Hangzhou International Innovation Institute, Beihang University（北京航空航天大学杭州国际创新研究院）；INSAIT, Sofia University（保加利亚索菲亚大学INSAIT研究院）
