Vision-language models (VLMs) like CLIP have been adapted for Multi-Label Recognition (MLR) with partial annotations by leveraging prompt-learning, where positive and negative prompts are learned for each class to associate their embeddings with class presence or absence in the shared vision-text feature space. While this approach improves MLR performance by relying on VLM priors, we hypothesize that learning negative prompts may be suboptimal, as the datasets used to train VLMs lack image-caption pairs explicitly focusing on class absence. To analyze the impact of positive and negative prompt learning on MLR, we introduce PositiveCoOp and NegativeCoOp, where only one prompt is learned with VLM guidance while the other is replaced by an embedding vector learned directly in the shared feature space without relying on the text encoder. Through empirical analysis, we observe that negative prompts degrade MLR performance, and learning only positive prompts, combined with learned negative embeddings (PositiveCoOp), outperforms dual prompt learning approaches. Moreover, we quantify the performance benefits that prompt-learning offers over a simple vision-features-only baseline, observing that the baseline displays strong performance comparable to dual prompt learning approach (DualCoOp), when the proportion of missing labels is low, while requiring half the training compute and 16 times fewer parameters
Rethinking the Role of Prompting Strategies in LLM Test-Time Scaling: A Perspective of Probability Theory
MAIS, Institute of Automation, Chinese Academy of Sciences（自动化研究所，中国科学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；University of California, Santa Barbara（加州大学圣巴巴拉分校）；Beijing Wenge Technology Co., Ltd（北京文格科技有限公司）
Enhancing Text Annotation through Rationale-Driven Collaborative Few-Shot Prompting
Institute of Artificial Intelligence and Future Networks, Beijing Normal University（北京师范大学人工智能与未来网络研究院）；Hong Kong Baptist University（香港浸会大学）；BNU-HKBU United International College（北京师范大学-香港浸会大学联合国际学院）；College of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）
Beyond the Leaderboard: Rethinking Medical Benchmarks for Large Language Models
Stanford University（斯坦福大学）；Shenzhen University（深圳大学）；The Chinese University of Hong Kong（香港中文大学）；City University of Hong Kong（香港城市大学）；Renmin University of China（中国人民大学）
University of Rochester（罗切斯特大学）；Tencent AI Lab（腾讯人工智能实验室）；University of Pennsylvania（宾夕法尼亚大学）
Despite the success of large language models (LLMs) in various natural language processing (NLP) tasks, the stored knowledge in these models may inevitably be incomplete, out-of-date, or incorrect. This motivates the need to utilize external knowledge to assist LLMs. Unfortunately, current methods for incorporating external knowledge often require additional training or fine-tuning, which can be costly and may not be feasible for LLMs. To address this issue, we propose a novel post-processing approach, rethinking with retrieval (RR), which retrieves relevant external knowledge based on the dec…
Rethinking LLM-as-a-Judge: Representation-as-a-Judge with Small Language Models via Semantic Capacity Asymmetry
Ping An Technology (Shenzhen) Co., Ltd.（平安科技（深圳）有限公司）；University of Pittsburgh（匹兹堡大学）；University of Maryland, College Park（马里兰大学学院公园分校）；University of Connecticut（康涅狄格大学）
Lawma: The Power of Specialization for Legal Annotation
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Tübingen AI Center（图宾根人工智能中心）；Max Planck Institute for Software Systems（马克斯·普朗克软件系统研究所）；ETH Zurich（苏黎世联邦理工学院）；Max Planck Institute for Research on Collective Goods（马克斯·普朗克集体研究研究所）；Washington University in St. Louis - School of Law（圣路易斯华盛顿大学法学院）；University of Virginia School of Law（弗吉尼亚大学法学院）
GeoBench: Rethinking Multimodal Geometric Problem-Solving via Hierarchical Evaluation
Shanghai Jiao Tong University（上海交通大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Fudan University（复旦大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
Rethinking the Chain-of-Thought: The Roles of In-Context Learning and Pre-trained Priors
School of Intelligence Science and Technology, National Key Laboratory for Novel Software Technology, Nanjing University（智能科学与技术学院，新型软件技术国家重点实验室，南京大学）；School of Computing and Information Systems, Singapore Management University（计算与信息系统学院，新加坡管理大学）；Central South University（中南大学）；School of Global Education and Development, International Chinese Language Education, University of Chinese Academy of Social Sciences（全球教育与发展学院，国际中文教育，中国社会科学院）
Visual Prompting in Multimodal Large Language Models: A Survey
UC San Diego（加州大学圣地亚哥分校）；Dartmouth College（达特茅斯学院）；Rutgers University（罗格斯大学）；UC Los Angeles（加州大学洛杉矶分校）；Adobe Research（奥多比研究院）；The University of New South Wales（新南威尔士大学）；CSIRO’s Data61（联邦科学与工业研究组织Data61）
Seeing the Trees for the Forest: Rethinking Weakly-Supervised Medical Visual Grounding
Australian Institute for Machine Learning, University of Adelaide（澳大利亚机器学习研究所，阿德莱德大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Macquarie University（麦考瑞大学）；Hanoi University of Science and Technology（河内科学技术大学）；University of Wollongong（沃林根大学）；Flinders University（弗林德斯大学）
University of Campinas（坎皮纳斯大学）；Université Gustave Eiffel（古斯塔夫·埃菲尔大学）
Despite the progress of interactive image segmentation methods, high-quality pixel-level annotation is still time-consuming and laborious - a bottleneck for several deep learning applications. We take a step back to propose interactive and simultaneous segment annotation from multiple images guided by feature space projection. This strategy is in stark contrast to existing interactive segmentation methodologies, which perform annotation in the image domain. We show that feature space annotation achieves competitive results with state-of-the-art methods in foreground segmentation datasets: iCoS…
Quality Assured: Rethinking Annotation Strategies in Imaging AI
German Cancer Research Center（德国癌症研究中心）；Helmholtz Imaging（亥姆霍兹成像中心）；Heidelberg University（海德堡大学）；University of Minnesota（明尼苏达大学）；National Center for Tumor Diseases（国家肿瘤疾病中心）
Annotated images are required for both supervised model training and evaluation in image classification. Manually annotating images is arduous and expensive, especially for multi-labeled images. A recent trend for conducting such laboursome annotation tasks is through crowdsourcing, where images are annotated by volunteers or paid workers online (e.g., workers of Amazon Mechanical Turk) from scratch. However, the quality of crowdsourcing image annotations cannot be guaranteed, and incompleteness and incorrectness are two major concerns for crowdsourcing annotations. To address such concerns, w…
Rethinking the Learning Paradigm for Facial Expression Recognition
University of Trento \& Fondazione Bruno Kessler Trento Trento Italy；University of Trento Trento Trento Italy；University of Trento \& Fondazione Bruno Kessler；University of Trento
hutom；Yonsei University College of Medicine（延世大学医学院）
As the area of application of deep neural networks expands to areas requiring expertise, e.g., in medicine and law, more exquisite annotation processes for expert knowledge training are required. In particular, it is difficult to guarantee generalization performance in the clinical field in the case of expert knowledge training where opinions may differ even among experts on annotations. To raise the issue of the annotation generation process for expertise training of CNNs, we verified the annotations for surgical phase recognition of laparoscopic cholecystectomy and subtotal gastrectomy for g…
Pose estimation of the human body and hands is a fundamental problem in computer vision, and learning-based solutions require a large amount of annotated data. In this work, we improve the efficiency of the data annotation process for 3D pose estimation problems with Active Learning (AL) in a multi-view setting. AL selects examples with the highest value to annotate under limited annotation budgets (time and cost), but choosing the selection strategy is often nontrivial. We present a framework to efficiently extend existing single-view AL strategies. We then propose two novel AL strategies tha…
As deep networks require large amounts of accurately labeled training data, a strategy to collect sufficiently large and accurate annotations is as important as innovations in recognition methods. This is especially true for building Computer Aided Detection (CAD) systems for chest X-rays where domain expertise of radiologists is required to annotate the presence and location of abnormalities on X-ray images. However, there lacks concrete evidence that provides guidance on how much resource to allocate for data annotation such that the resulting CAD system reaches desired performance. Without …
MILA (Quebec Artificial Intelligence Institute)（魁北克人工智能研究所MILA）；McGill University（麦吉尔大学）；NeuroRx Research（NeuroRx研究公司）；University of Edinburgh（爱丁堡大学）；The Alan Turing Institute（艾伦·图灵研究所）
Generalization is an important attribute of machine learning models, particularly for those that are to be deployed in a medical context, where unreliable predictions can have real world consequences. While the failure of models to generalize across datasets is typically attributed to a mismatch in the data distributions, performance gaps are often a consequence of biases in the 'ground-truth' label annotations. This is particularly important in the context of medical image segmentation of pathological structures (e.g. lesions), where the annotation process is much more subjective, and affecte…
