This paper presents a novel evaluation framework for Out-of-Distribution (OOD) detection that aims to assess the performance of machine learning models in more realistic settings. We observed that the real-world requirements for testing OOD detection methods are not satisfied by the current testing protocols. They usually encourage methods to have a strong bias towards a low level of diversity in normal data. To address this limitation, we propose new OOD test datasets (CIFAR-10-R, CIFAR-100-R, and ImageNet-30-R) that can allow researchers to benchmark OOD detection performance under realistic distribution shifts. Additionally, we introduce a Generalizability Score (GS) to measure the generalization ability of a model during OOD detection. Our experiments demonstrate that improving the performance on existing benchmark datasets does not necessarily improve the usability of OOD detection models in real-world scenarios. While leveraging deep pre-trained features has been identified as a promising avenue for OOD detection research, our experiments show that state-of-the-art pre-trained models tested on our proposed datasets suffer a significant drop in performance. To address this issue, we propose a post-processing stage for adapting pre-trained features under these distribution shifts before calculating the OOD scores, which significantly enhances the performance of state-of-the-art pre-trained models on our benchmarks.
TTA-OOD: Test-time Augmentation for Improving Out-of-Distribution Detection in Gastrointestinal Vision
Nepal Applied Mathematics and Informatics Institute for research(NAAMII)(尼泊尔应用数学与信息学研究所(NAAMII));Fogsphere(Redev AI Ltd.)(Fogsphere(Redev AI有限公司));School of Natural and Computing Sciences, University of Aberdeen(阿伯丁大学自然与计算科学学院);West Virginia University(西弗吉尼亚大学)
ANTS: Adaptive Negative Textual Space Shaping for OOD Detection via Test-Time MLLM Understanding and Reasoning
The Hong Kong Polytechnic University(香港理工大学);Eastern Institute of Technology, Ningbo(宁波东部技术研究院);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳));Zhongguancun Academy(中关村学院)
Prototype-based Optimal Transport for Out-of-Distribution Detection
School of Biomedical Engineering, University of Science and Technology of China(中国科学技术大学生物医学工程学院);Data Darkness Lab, MIRACLE Center, Suzhou Institute for Advanced Research, USTC(中国科学技术大学苏州高等研究院MIRACLE中心Data Darkness实验室)
Average of Pruning: Improving Performance and Stability of Out-of-Distribution Detection
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
Your Data Is Not Perfect: Towards Cross-Domain Out-of-Distribution Detection in Class-Imbalanced Data
College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算学院和数据科学学院);KINDI Computing Research Center, College of Engineering, Qatar University, Doha(卡塔尔大学工程学院KINDI计算研究中心,多哈)
Knowledge Regularized Negative Feature Tuning of Vision-Language Models for Out-of-Distribution Detection
Hong Kong Polytechnic University(香港理工大学);Eastern Institute of Technology(东部技术研究所);Stanford University(斯坦福大学);Ningbo Institute of Digital Twin, Eastern Institute of Technology(宁波数字孪生研究所,东部技术研究所);Institute for Clarity in Documentation(文档清晰研究所);Inria Paris-Rocquencourt(巴黎-罗克琴堡研究所);Rajiv Gandhi University(拉贾·甘地大学);Tsinghua University(清华大学);Palmer Research Laboratories(帕勒研究实验室)
Improving Wildlife Out-of-Distribution Detection: Africas Big Five
Institute for Artificial Intelligent Systems(人工智能系统研究所);University of Johannesburg(约翰内斯堡大学);Department of Electrical Engineering(电气工程系);Linköping University(利马大学)
GOOD: Training-Free Guided Diffusion Sampling for Out-of-Distribution Detection
Nanjing University(南京大学);Fudan University(复旦大学);Carnegie Mellon University(卡内基梅隆大学);Chinese Academy of Sciences(中国科学院);Nanyang Technological University(南洋理工大学)
Long-Tailed Out-of-Distribution Detection with Refined Separate Class Learning
Nanjing Agricultural University(南京农业大学);Nanjing University(南京大学);Shandong University(山东大学);Nanjing University of Posts and Telecommunications(南京邮电大学);Southeast University(东南大学)
On the Learnability of Out-of-distribution Detection
Australian Artificial Intelligence Institute(澳大利亚人工智能研究所);University of Technology Sydney(悉尼科技大学);University of Wisconsin Madison(威斯康星大学麦迪逊分校);University of Melbourne(墨尔本大学);Hong Kong Baptist University(香港浸会大学)
Advancing Out-of-Distribution Detection through Data Purification and Dynamic Activation Function Design
Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院空天信息创新研究院);University of Chinese Academy of Sciences(中国科学院大学);Zhejiang University(浙江大学)
Out-of-Distribution Detection Should Use Conformal Prediction (and Vice-versa?)
IRT Saint Exupery(圣埃克苏佩里IRT研究院);Artificial and Natural Intelligence Toulouse Institute(图卢兹人工与自然智能研究所);SNCF(法国国家铁路公司);Mathematical Institute of Toulouse(图卢兹数学研究所)
COOkeD: Ensemble-based OOD detection in the era of zero-shot CLIP
Aalborg University(奥尔堡大学);ENSTA, Institut Polytechnique de Paris(法国国立高等先进技术学校,巴黎综合理工学院);University of Barcelona(巴塞罗那大学);Computer Vision Center(计算机视觉中心)