数据污染下无监督鲁棒异常检测的迭代方法
An Iterative Method for Unsupervised Robust Anomaly Detection Under Data Contamination
AI总结：
本文提出一种模型无关且对超参数不敏感的迭代学习框架，通过样本级正常性权重减少数据污染对无监督异常检测的影响，并在多个数据集上验证其鲁棒性提升。
摘要：
大多数深度异常检测模型由于异常本身具有多样且不一致的性质而难以定义，因此通常基于从数据集中学习正常模式。因此，在训练数据集中不存在异常数据的假设下学习正常模式已成为常见做法，我们称之为正常性假设。然而，在实践中，由于真实数据分布包含异常尾部（即污染数据集）的性质，正常性假设常常被违反。因此，该假设与实际训练数据之间的差距会对异常检测模型的学习产生不利影响。在这项工作中，我们提出了一种学习框架来缩小这一差距并实现更好的正常性表示。我们的关键思想是识别样本级别的正常性并将其用作重要性权重，该权重在训练过程中迭代更新。我们的框架被设计为模型无关且对超参数不敏感，因此它适用于广泛的现有方法而无需仔细的参数调整。我们将我们的框架应用于三种不同的代表性深度异常检测方法，这些方法分为单类分类、概率模型和基于重建的方法。此外，我们阐述了迭代方法终止条件的重要性，并提出了一种受异常检测目标启发的终止准则。我们验证了我们的框架在五个异常检测基准数据集和两个图像数据集上，在不同污染比率水平下提高了异常检测模型的鲁棒性。在各种污染数据集上，我们的框架提高了三种代表性异常检测方法的性能，以ROC曲线下面积衡量。
英文摘要：
Most deep anomaly detection models are based on learning normality from datasets due to the difficulty of defining abnormality by its diverse and inconsistent nature. Therefore, it has been a common practice to learn normality under the assumption that anomalous data are absent in a training dataset, which we call normality assumption. However, in practice, the normality assumption is often violated due to the nature of real data distributions that includes anomalous tails, i.e., a contaminated dataset. Thereby, the gap between the assumption and actual training data affects detrimentally in learning of an anomaly detection model. In this work, we propose a learning framework to reduce this gap and achieve better normality representation. Our key idea is to identify sample-wise normality and utilize it as an importance weight, which is updated iteratively during the training. Our framework is designed to be model-agnostic and hyperparameter insensitive so that it applies to a wide range of existing methods without careful parameter tuning. We apply our framework to three different representative approaches of deep anomaly detection that are classified into one-class classification-, probabilistic model-, and reconstruction-based approaches. In addition, we address the importance of a termination condition for iterative methods and propose a termination criterion inspired by the anomaly detection objective. We validate that our framework improves the robustness of the anomaly detection models under different levels of contamination ratios on five anomaly detection benchmark datasets and two image datasets. On various contaminated datasets, our framework improves the performance of three representative anomaly detection methods, measured by area under the ROC curve.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
