Pair-wise loss functions have been extensively studied and shown to continuously improve the performance of deep metric learning (DML). However, they are primarily designed with intuition based on simple toy examples, and experimentally identifying the truly effective design is difficult in complicated, real-world cases. In this paper, we provide a new methodology for systematically studying weighting strategies of various pair-wise loss functions, and rethink pair weighting with an embedding memory. We delve into the weighting mechanisms by decomposing the pair-wise functions, and study positive and negative weights separately using direct weight assignment. This allows us to study various weighting functions deeply and systematically via weight curves, and identify a number of meaningful, comprehensive and insightful facts, which come up with our key observation on memory-based DML: it is critical to mine hard negatives and discard easy negatives which are less informative and redundant, but weighting on positive pairs is not helpful. This results in an efficient but surprisingly simple rule to design the weighting scheme, making it significantly different from existing mini-batch based methods which design various sophisticated loss functions to weight pairs carefully. Finally, we conduct extensive experiments on three large-scale visual retrieval benchmarks, and demonstrate the superiority of memory-based DML over recent mini-batch based approaches, by using a simple contrastive loss with momentum-updated memory.
Rethinking Multiple Instance Learning for Whole Slide Image Classification: A Good Instance Classifier is All You Need
Digital Medical Research Center, School of Basic Medical Science, Fudan University（复旦大学基础医学院数字医学研究中心）；Fudan University Shanghai Cancer Center（复旦大学附属肿瘤医院）
Contrastive Quantization with Code Memory for Unsupervised Image Retrieval
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学清华-深圳国际研究生院）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Shenzhen University（深圳大学）；Peng Cheng Laboratory（鹏城实验室）
Surrey Institute for People-Centred AI（苏格兰人本人工智能研究所）；Centre For Vision, Speech and Signal Processing (CVSSP)（视觉、语音和信号处理中心）；Jiangnan University（江南大学）；Carnegie Mellon University（卡内基梅隆大学）
Rethinking Image Forgery Detection via Soft Contrastive Learning and Unsupervised Clustering
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；Guangdong Key Laboratory of Intelligent Information Processing, College of Electronics and Information Engineering, Shenzhen University（广东智能信息处理重点实验室、电子与信息工程学院）
Knowledge Boosting: Rethinking Medical Contrastive Vision-Language Pre-Training
Southeast University（东南大学）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Case Western Reserve University（凯斯西储大学）；Centre de Recherche en Information Biomédicale Sino-Français（中法生物医学信息研究中心）
More performant and scalable: Rethinking contrastive vision-language pre-training of radiology in the LLM era
School of Biomedical Engineering, Division of Life Sciences；Medicine, University of Science；Technology of China (USTC), Hefei Anhui, 230026, China Center for Medical Imaging, Robotics, Analytic Computing \& Learning (MIRACLE), Suzhou Institute for Advance Research, USTC, Suzhou Jiangsu, 215123, China The First Affiliated Hospital of USTC, Division of Life Sciences；Medicine, USTC, Hefei Anhui, 230001, China Jiangsu Provincial Key Laboratory of Multimodal Digital Twin Technology, Suzhou Jiangsu, 215123, China State Key Laboratory of Precision
本文提出利用 LLMs 自动从放射学报告中提取诊断标签以低成本构建大规模“银标准”数据集，从而促进大规模监督预训练，仅使用 3D ResNet-18 结合原始 CLIP 训练便在零样本诊断和跨模态检索任务上取得了最先进的性能。
Rethinking the Sample Relations for Few-Shot Classification
School of Big Data and Software Engineering, Chongqing University（重庆大学大数据与软件学院）；Chongqing University（重庆大学）；Fowler College of Business, San Diego State University（圣地亚哥州立大学福勒商学院）；San Diego State University（圣地亚哥州立大学）
One-Shot Medical Video Object Segmentation via Temporal Contrastive Memory Networks
Wuhan University of Technology（武汉理工大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；MedAI Technology (Wuxi) Co. Ltd.（MedAI科技（无锡）有限公司）；Technical University of Munich（慕尼黑工业大学）
Memory-aided Contrastive Consensus Learning for Co-salient Object Detection
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；ETH Zurich（苏黎世联邦理工学院）；Inception Institute of Artificial Intelligence（启元实验室）；Harbin Institute of Technology（哈尔滨工业大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Video-based Generalized Category Discovery via Memory-Guided Consistency-Aware Contrastive Learning
Laboratory for Big Data and Decision（大数据与决策实验室）；National University of Defense Technology（国防科技大学）；Department of Information Engineering and Computer Science（信息工程与计算机科学系）
