异常检测中的聪明汉斯效应
The Clever Hans Effect in Anomaly Detection
摘要:
当学习的模型基于“错误”的特征产生正确的预测时,就会发生“聪明的汉斯”效应。这种影响破坏了ML模型的泛化能力,并且未被标准验证技术检测到,这种影响经常被监督学习观察到,其中训练算法利用了数据中的虚假相关性。聪明的汉斯是否也出现在无监督学习中,以及以何种形式出现,这个问题到目前为止几乎没有受到关注。因此,本文将贡献一个可解释的AI(XAI)过程,可以突出不同类型的流行异常检测模型所使用的相关特征。我们的分析表明,聪明的汉斯效应是广泛的异常检测和发生在许多(意想不到的)形式。有趣的是,在这种情况下,观察到的聪明汉斯效应并不是由于数据,而是由于异常检测模型本身的结构使得它们无法检测到真正相关的特征,即使有大量的数据点可用。总的来说,我们的工作有助于警告在实际应用中不受限制地使用现有的异常检测模型,但它也指出了一种可能的出路聪明汉斯困境,具体来说,通过允许多个异常模型相互取消各自的结构弱点,共同产生一个更好,更值得信赖的异常检测器。
英文摘要:
The 'Clever Hans' effect occurs when the learned model produces correct predictions based on the 'wrong' features. This effect which undermines the generalization capability of an ML model and goes undetected by standard validation techniques has been frequently observed for supervised learning where the training algorithm leverages spurious correlations in the data. The question whether Clever Hans also occurs in unsupervised learning, and in which form, has received so far almost no attention. Therefore, this paper will contribute an explainable AI (XAI) procedure that can highlight the relevant features used by popular anomaly detection models of different type. Our analysis reveals that the Clever Hans effect is widespread in anomaly detection and occurs in many (unexpected) forms. Interestingly, the observed Clever Hans effects are in this case not so much due to the data, but due to the anomaly detection models themselves whose structure makes them unable to detect the truly relevant features, even though vast amounts of data points are available. Overall, our work contributes a warning against an unrestrained use of existing anomaly detection models in practical applications, but it also points at a possible way out of the Clever Hans dilemma, specifically, by allowing multiple anomaly models to mutually cancel their individual structural weaknesses to jointly produce a better and more trustworthy anomaly detector.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。