Recent advancements in large-scale visual-language pre-trained models have led to significant progress in zero-/few-shot anomaly detection within natural image domains. However, the substantial domain divergence between natural and medical images limits the effectiveness of these methodologies in medical anomaly detection. This paper introduces a novel lightweight multi-level adaptation and comparison framework to repurpose the CLIP model for medical anomaly detection. Our approach integrates multiple residual adapters into the pre-trained visual encoder, enabling a stepwise enhancement of visual features across different levels. This multi-level adaptation is guided by multi-level, pixel-wise visual-language feature alignment loss functions, which recalibrate the model's focus from object semantics in natural imagery to anomaly identification in medical images. The adapted features exhibit improved generalization across various medical data types, even in zero-shot scenarios where the model encounters unseen medical modalities and anatomical regions during training. Our experiments on medical anomaly detection benchmarks demonstrate that our method significantly surpasses current state-of-the-art models, with an average AUC improvement of 6.24% and 7.33% for anomaly classification, 2.03% and 2.37% for anomaly segmentation, under the zero-shot and few-shot settings, respectively. Source code is available at: https://github.com/MediaBrain-SJTU/MVFA-AD
MediCLIP: Adapting CLIP for Few-shot Medical Image Anomaly Detection
College of Information and Engineering, Capital Normal University（首都师范大学信息工程学院）；Peking University Third Hospital（北京大学第三医院）；School of Artificial Intelligence, Beijing University of Posts and Telecommunications（北京邮电大学人工智能学院）
A Generalizable Feature Extractor for Alzheimer's-Related Brain MRI Tasks
McConnell Brain Imaging Centre, Montreal Neurological Institute, McGill University（麦吉尔大学蒙特利尔神经学研究所麦康奈尔脑成像中心）；Department of Biomedical Engineering, McGill University（麦吉尔大学生物医学工程系）
SAMamba3D: adapting Segment Anything for generalizable 3D segmentation of multiphase pore-scale images
Department of Earth Science and Engineering, Imperial College London（帝国理工学院伦敦地球科学与工程系）；College of Artificial Intelligence, China University of Petroleum (Beijing)（中国石油大学（北京）人工智能学院）；College of Petroleum Engineering, China University of Petroleum (Beijing)（中国石油大学（北京）石油工程学院）
Tsinghua University（清华大学）；Huazhong University of Science and Technology（华中科技大学）
Industrial vision anomaly detection plays a critical role in the advanced intelligent manufacturing process, while some limitations still need to be addressed under such a context. First, existing reconstruction-based methods struggle with the identity mapping of trivial shortcuts where the reconstruction error gap is legible between the normal and abnormal samples, leading to inferior detection capabilities. Then, the previous studies mainly concentrated on the convolutional neural network (CNN) models that capture the local semantics of objects and neglect the global context, also resulting …
DictAS: A Framework for Class-Generalizable Few-Shot Anomaly Segmentation via Dictionary Lookup
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Casivision；Longmen Laboratory（龙门实验室）；HDU；UTS；UCLA（加州大学洛杉矶分校）；Tsinghua University（清华大学）
Action Hints: Semantic Typicality and Context Uniqueness for Generalizable Skeleton-based Video Anomaly Detection
National Key Laboratory of Human-Machine Hybrid Augmented Intelligence（人机混合增强智能国家重点实验室）；National Engineering Research Center for Visual Information（视觉信息工程国家级工程研究中心）；Institute of Artificial Intelligence and Robotics（人工智能与机器人研究所）；Robotics, Xi'an Jiaotong University（机器人学，西安交通大学）；School of Electronics and Control Engineering（电子控制工程学院）
Adapt2Reward: Adapting Video-Language Models to Generalizable Robotic Rewards via Failure Prompts
School of Software Technology, Zhejiang University（浙江大学软件学院）；School of Computer Sciene and Technology, Hangzhou Dianzi University（杭州电子科技大学计算机学院）；China Mobile (Zhejiang) Research & Innovation Institute（中国移动（浙江）研究院）；State Key Lab of CAD&CG, Zhejiang University（浙江大学CAD&CG国家重点实验室）；The Hong Kong Polytechnic University（香港理工大学）；Zhiyuan Research Institute（智源研究院）；School of Computer Sciene and Technology, Xidian University（西安电子科技大学计算机学院）
Cranberry-Lemon University（蔓越莓柠檬大学）；University of the Witwatersrand（金山大学）
Anomaly detection methods require high-quality features. In recent years, the anomaly detection community has attempted to obtain better features using advances in deep self-supervised feature learning. Surprisingly, a very promising direction, using pretrained deep features, has been mostly overlooked. In this paper, we first empirically establish the perhaps expected, but unreported result, that combining pretrained features with simple anomaly detection and segmentation methods convincingly outperforms, much more complex, state-of-the-art methods. In order to obtain further performance gain…
Shanghai Jiao Tong University（上海交通大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；National University of Singapore（新加坡国立大学）；King’s College London（伦敦国王学院）
This paper considers few-shot anomaly detection (FSAD), a practical yet under-studied setting for anomaly detection (AD), where only a limited number of normal images are provided for each category at training. So far, existing FSAD studies follow the one-model-per-category learning paradigm used for standard AD, and the inter-category commonality has not been explored. Inspired by how humans detect anomalies, i.e., comparing an image in question to normal images, we here leverage registration, an image alignment task that is inherently generalizable across categories, as the proxy task, to tr…
ResAD: A Simple Framework for Class Generalizable Anomaly Detection
School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University（上海交通大学电子信息与电气工程学院）；China Pacific Insurance (Group) Co., Ltd.（中国太平洋保险（集团）股份有限公司）；MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University（上海交通大学人工智能研究院、教育部人工智能重点实验室）
Polytechnique Montréal（蒙特利尔理工学院）；Institute of Biomedical Engineering（生物医学工程研究所）
We present a local anomaly detection method in videos. As opposed to most existing methods that are computationally expensive and are not very generalizable across different video scenes, we propose an adversarial framework that learns the temporal local appearance variations by predicting the appearance of a normally behaving object in the next frame of a scene by only relying on its current and past appearances. In the presence of an abnormally behaving object, the reconstruction error between the real and the predicted next appearance of that object indicates the likelihood of an anomaly. O…
Robust Distribution Alignment for Industrial Anomaly Detection under Distribution Shift
Institute for Infocomm Research, A*STAR（新加坡科研局信息通信研究院）；Nanyang Technological University（南洋理工大学）；South China University of Technology（华南理工大学）；National University of Singapore（新加坡国立大学）
Shenzhen Institute of Advanced Study, UESTC（电子科技大学深圳高等研究院）
Anomaly detection aims to separate anomalies from normal samples, and the pretrained network is promising for anomaly detection. However, adapting the pretrained features would be confronted with the risk of pattern collapse when finetuning on one-class training data. In this paper, we propose an anomaly detection framework called constrained adaptive projection with pretrained features (CAP). Combined with pretrained features, a simple linear projection head applied on a specific input and its k most similar pretrained normal representations is designed for feature adaptation, and a reformed …
Zero-shot Generalizable Incremental Learning for Vision-Language Object Detection
Institute of Automation, Chinese Academy of Sciences (CAS)（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Science, UCAS（中国科学院大学人工智能学院）；Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属第六人民医院）
Unsupervised representation learning has proved to be a critical component of anomaly detection/localization in images. The challenges to learn such a representation are two-fold. Firstly, the sample size is not often large enough to learn a rich generalizable representation through conventional techniques. Secondly, while only normal samples are available at training, the learned features should be discriminative of normal and anomalous samples. Here, we propose to use the "distillation" of features at various layers of an expert network, pre-trained on ImageNet, into a simpler cloner network…
MadCLIP: Few-shot Medical Anomaly Detection with CLIP
Department of Computer Science, University of Verona（威尼斯大学计算机科学系）；AI for Good (AIGO) Research Unit, Istituto Italiano di Tecnologia（意大利技术研究院人工智能为善研究单元）
U-TTT: Towards Generalizable PET Image Denoising via Test-Time Training
School of Biological Science and Medical Engineering, Beihang University（北京航空航天大学生物科学与医学工程学院）；Department of Biomedical Engineering, Tsinghua University（清华大学生物医学工程系）；School of Aerospace Engineering, Tsinghua University（清华大学航天航空学院）；ByteDance Inc.（字节跳动有限公司）
Demographic-Aware Self-Supervised Anomaly Detection Pretraining for Equitable Rare Cardiac Diagnosis
Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）；National University of Defense Technology（国防科技大学）；Ruijin Hospital, Shanghai Jiao Tong University School of Medicine（瑞金医院，上海交通大学医学院）；Renji Hospital, Shanghai Jiao Tong University School of Medicine（仁济医院，上海交通大学医学院）
Scale-Aware Pre-Training for Human-Centric Visual Perception: Enabling Lightweight and Generalizable Models
Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China（深圳先进研究院，电子科学与技术大学）；Center for Future Media, University of Electronic Science and Technology of China（未来媒体中心，电子科学与技术大学）；TongJi University（同济大学）
VMAD: Visual-enhanced Multimodal Large Language Model for Zero-Shot Anomaly Detection
University of Science and Technology of China（中国科学技术大学）；Northeastern University（东北大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）
ADPretrain: Advancing Industrial Anomaly Detection via Anomaly Representation Pretraining
School of Information Science and Electronic Engineering, Shanghai Jiao Tong University（上海交通大学信息科学与电子工程学院）；MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University（上海交通大学人工智能MOE重点实验室）；College of Artificial Intelligence, Nanjing Agricultural University（南京农业大学人工智能学院）；Key Laboratory of Livestock Farming Equipment, Ministry of Agriculture and Rural Affairs, Nanjing Agricultural University（南京农业大学农业机械化关键实验室）
TokenCLIP: Token-wise Prompt Learning for Zero-shot Anomaly Detection
Zhejiang University（浙江大学）；Tencent YouTu Lab（腾讯YouTu实验室）；Singapore Management University（新加坡管理学院）；Qilu University of Technology (Shandong Academy of Sciences)（青岛科技大学（山东科学院））
