Advancing Out-of-Distribution Detection through Data Purification and Dynamic Activation Function Design
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Zhejiang University（浙江大学）
Yingrui Ji, Yao Zhu, Zhigang Li, Jiansheng Chen, Yunlong Kong, Jingbo Chen
In the dynamic realms of machine learning and deep learning, the robustness and reliability of models are paramount, especially in critical real-world applications. A fundamental challenge in this sphere is managing Out-of-Distribution (OOD) samples, significantly increasing the risks of model misclassification and uncertainty. Our work addresses this challenge by enhancing the detection and management of OOD samples in neural networks. We introduce OOD-R (Out-of-Distribution-Rectified), a meticulously curated collection of open-source datasets with enhanced noise reduction properties. In-Distribution (ID) noise in existing OOD datasets can lead to inaccurate evaluation of detection algorithms. Recognizing this, OOD-R incorporates noise filtering technologies to refine the datasets, ensuring a more accurate and reliable evaluation of OOD detection algorithms. This approach not only improves the overall quality of data but also aids in better distinguishing between OOD and ID samples, resulting in up to a 2.5\% improvement in model accuracy and a minimum 3.2\% reduction in false positives. Furthermore, we present ActFun, an innovative method that fine-tunes the model's response to diverse inputs, thereby improving the stability of feature extraction and minimizing specificity issues. ActFun addresses the common problem of model overconfidence in OOD detection by strategically reducing the influence of hidden units, which enhances the model's capability to estimate OOD uncertainty more accurately. Implementing ActFun in the OOD-R dataset has led to significant performance enhancements, including an 18.42\% increase in AUROC of the GradNorm method and a 16.93\% decrease in FPR95 of the Energy method. Overall, our research not only advances the methodologies in OOD detection but also emphasizes the importance of dataset integrity for accurate algorithm evaluation.
Adversarial Training on Purification (AToP): Advancing Both Robustness and Generalization
Tokyo University of Agriculture and Technology（东京农工大学）；RIKEN Center for Advanced Intelligence Project (RIKEN AIP)（理化学研究所先进智能项目中心）；Hangzhou Dianzi University（杭州电子科技大学）
Microsoft Research（微软研究院）；Carnegie Mellon University（卡内基梅隆大学）
Despite the empirical success of using Adversarial Training to defend deep learning models against adversarial perturbations, so far, it still remains rather unclear what the principles are behind the existence of adversarial perturbations, and what adversarial training does to the neural network to remove them. In this paper, we present a principle that we call Feature Purification, where we show one of the causes of the existence of adversarial examples is the accumulation of certain small dense mixtures in the hidden weights during the training process of a neural network; and more importan…
FLARE: Toward Universal Dataset Purification against Backdoor Attacks
School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院）；School of Information Technology, Deakin University（德克萨斯大学信息学院）；School of Information and Communication Technology, Griffith University（格里菲斯大学信息与通信技术学院）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）
MicroCrackAttentionNeXt: Advancing Microcrack Detection in Wave Field Analysis Using Deep Neural Networks through Feature Visualization
Christian Albrechts University（基尔基督教-阿尔布雷希茨大学）；Aligarh Muslim University（阿里格尔穆斯林大学）；Texas A&M University（德克萨斯农工大学）；Institute of Computer Science（计算机科学研究所）
TabKAN: Advancing Tabular Data Analysis using Kolmogorov-Arnold Network
Department of Computer Science, University of Kentucky（卡内基梅隆大学计算机科学系）；Independent Researcher（独立研究者）；Institute for Biomedical Informatics, University of Kentucky（生物医学信息学研究所）
Indian Institute of Information Technology, Sri City（印度信息技术学院，斯里城）；Aalborg University（奥尔堡大学）；Pioneer Centre for AI（人工智能先锋中心）
Deep representation learning has gained significant momentum in advancing text-dependent speaker verification (TD-SV) systems. When designing deep neural networks (DNN) for extracting bottleneck features, key considerations include training targets, activation functions, and loss functions. In this paper, we systematically study the impact of these choices on the performance of TD-SV. For training targets, we consider speaker identity, time-contrastive learning (TCL) and auto-regressive prediction coding with the first being supervised and the last two being self-supervised. Furthermore, we st…
Flow Matching with In-Context Priors for Out-of-Distribution Brain Dynamics
Hertie Institute for AI in Brain Health, University of Tübingen（赫蒂人工智能脑健康研究所，图宾根大学）；Tübingen AI Center, University of Tübingen（图宾根人工智能中心，图宾根大学）；Charité – Universitätsmedizin Berlin, Department of Psychiatry and Psychotherapy（柏林夏里特医学院，精神病学与心理治疗系）；German Center for Mental Health (DZPG), partner site Tübingen（德国心理健康中心（DZPG），图宾根合作站点）
