Patch-Prompt Aligned Bayesian Prompt Tuning for Vision-Language Models
National Key Laboratory of Radar Signal Processing, Xidian University（西安电子科技大学雷达信号处理国家重点实验室）；McCombs School of Business, The University of Texas at Austin（德克萨斯大学奥斯汀分校麦库姆斯商学院）
For downstream applications of vision-language pre-trained models, there has been significant interest in constructing effective prompts. Existing works on prompt engineering, which either require laborious manual designs or optimize the prompt tuning as a point estimation problem, may fail to describe diverse characteristics of categories and limit their applications. We introduce a Bayesian probabilistic resolution to prompt tuning, where the label-specific stochastic prompts are generated hierarchically by first sampling a latent vector from an underlying distribution and then employing a lightweight generative model. Importantly, we semantically regularize the tuning process by minimizing the statistical distance between the visual patches and linguistic prompts, which pushes the stochastic label representations to faithfully capture diverse visual concepts, instead of overfitting the training categories. We evaluate the effectiveness of our approach on four tasks: few-shot image recognition, base-to-new generalization, dataset transfer learning, and domain shifts. Extensive results over 15 datasets show promising transferability and generalization performance of our proposed model, both quantitatively and qualitatively.
PAIQ: Patch-Aligned Semantic Injection via Residual Rotation
Tsinghua University（清华大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；National University of Singapore（新加坡国立大学）；Nanyang Technological University（南洋理工大学）
Critical Patch-Aware Sparse Prompting with Decoupled Training for Continual Learning on the Edge
School of Computer Science and Engineering, Chung-Ang University（中央大学计算机科学与工程学院）；Department of Artificial Intelligence, Chung-Ang University（中央大学人工智能系）
DevPrompt: Deviation-Based Prompt Learning for One-Normal ShotImage Anomaly Detection
Department of Electrical and Computer Engineering, Concordia University（电气与计算机工程系，康科迪亚大学）；Computer Research Institute of Montreal (CRIM)（蒙特利尔计算机研究 institute）
Analyzing Fine-Grained Alignment and Enhancing Vision Understanding in Multimodal Language Models
Department of Computer Science and Engineering, The Ohio State University（计算机科学与工程系，俄亥俄州立大学）；Translational Data Analytics Institute, The Ohio State University（转化数据分析研究所，俄亥俄州立大学）；Department of Biomedical Informatics, The Ohio State University（生物医学信息学系，俄亥俄州立大学）
PSMP-CLIP: Patch-Prompt SAM and Multi-Semantic Prompting for CLIP-Based Zero-Shot Anomaly Detection
Harbin Engineering University（哈尔滨工程大学）；Key Laboratory of Advanced Marine Communication and Information Technology（先进海洋通信与信息技术重点实验室）；University of Toronto（多伦多大学）；Kanagawa University（神奈川大学）
SED-MVS: Segmentation-Driven and Edge-Aligned Deformation Multi-View Stereo with Depth Restoration and Occlusion Constraint
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；School of Electrical Engineering and Computer Science, The University of Queensland（昆士兰大学电气工程与计算机科学学院）；Hunan Police Academy（湖南警察学院）；Cooperative MediaNet Innovation Center, Shanghai Jiao Tong University（上海交通大学协同媒体网络创新中心）；School of Information Science & Engineering, Lanzhou University（兰州大学信息科学与工程学院）；Agricultural Information Institute, Chinese Academy of Agricultural Sciences（中国农业科学院农业信息研究所）
MSPE: Multi-Scale Patch Embedding Prompts Vision Transformers to Any Resolution
School of Artificial Intelligence, UCAS（中国科学院大学人工智能学院）；State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA（中国科学院自动化研究所多模态人工智能系统国家重点实验室）；Centre for Artificial Intelligence and Robotics, HKISI-CAS（中国科学院香港中文大学信息工程研究所人工智能与机器人中心）
DaLPSR: Leverage Degradation-Aligned Language Prompt for Real-World Image Super-Resolution
School of Computer and Information Engineering, Jiangxi Normal University（江西师范大学计算机与信息工程学院）；School of Digital Industry, Jiangxi Normal University（江西师范大学数字产业学院）；Institute of Advanced Technology, University of Science and Technology of China（中国科学技术大学先进技术研究院）；Noah’s Ark Lab, Huawei Technologies Ltd.（华为技术有限公司诺亚方舟实验室）
PromptHub: Enhancing Multi-Prompt Visual In-Context Learning with Locality-Aware Fusion, Concentration and Alignment
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院，清华大学）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学深圳分校）；Meituan, Beijing（美团，北京）
LoR-VP: Low-Rank Visual Prompting for Efficient Vision Model Adaptation
Rutgers University（罗格斯大学）；Zhejiang University（浙江大学）；Red Hat AI Innovation（红帽人工智能创新中心）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）；NVIDIA Research（英伟达研究院）
Less is more: Summarizing Patch Tokens for efficient Multi-Label Class-Incremental Learning
University of Trento（特伦托大学）；LTCI, Télécom Paris（法国巴黎高等电信学院LTCI实验室）；Institut Polytechnique de Paris（巴黎理工学院）；University of Aberdeen（阿伯丁大学）；Fondazione Bruno Kessler（布鲁诺·凯塞勒基金会）
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；vivo Mobile Communication Co., Ltd.（维沃移动通信有限公司）；Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）；Pengcheng Laboratory（鹏城实验室）
MergeSlide: Continual Model Merging and Task-to-Class Prompt-Aligned Inference for Lifelong Learning on Whole Slide Images
Nara Institute of Science and Technology（奈良科学技术研究所）；Graduate School of Data Science, Chonnam National University（全南国立大学数据科学研究生院）；University of Information Technology, Viet Nam National University Ho Chi Minh City（越南胡志明市国家大学信息科技大学）；ETIS (UMR 8051), CY Cergy Paris University, ENSEA, CNRS（ETIS（UMR 8051），CY巴黎中央大学，ENSEA，CNRS）
VMBench: A Benchmark for Perception-Aligned Video Motion Generation
AMAP, Alibaba Group（阿里巴巴集团AMAP）；CRISE, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所CRISE）；School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）
SemanticControl: A Training-Free Approach for Handling Loosely Aligned Visual Conditions in ControlNet
Department of Computer Science and Engineering（计算机科学与工程系）；Korea University（韩国大学）；Electrical and Computer Engineering（电气与计算机工程系）；University of Michigan（密歇根大学）
