arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.00560cs.LGcs.AI
收藏

朋友还是敌人?利用可控过拟合进行异常检测

Friend or Foe? Harnessing Controllable Overfitting for Anomaly Detection

Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所基础模型研究中心);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Objecteye Inc.(Objecteye公司)

Long Qian, Bingke Zhu, Yingying Chen, Ming Tang, Jinqiao Wang

AI总结:

该研究提出可控过拟合异常检测框架(COAD),通过异常保留商(ARQ)量化过拟合程度并确定最优区间,结合相对异常分布指数(RADI)提升检测性能,在单类和多类异常检测任务中达到最先进水平。

摘要:

过拟合传统上被视为对异常检测有害,因为过度的泛化常常限制了模型对细微异常的敏感性。我们的工作通过引入基于可控过拟合的异常检测(COAD)这一新颖框架来挑战这一传统观点,该框架策略性地利用过拟合来增强异常判别能力。我们提出了异常保留商(ARQ),这是一种系统量化过拟合程度的新指标,能够识别一个最优的黄金过拟合区间,在该区间内模型对异常的敏感性达到最大化而不牺牲泛化能力。为了全面捕捉过拟合如何影响检测性能,我们进一步提出了相对异常分布指数(RADI),该指标通过显式建模正常与异常分数分布之间的分离,优于传统的AUROC。理论上,RADI利用ARQ来跟踪和评估过拟合如何影响异常检测,提供了一种理解过拟合动态与模型效能之间关系的综合方法。我们还严格验证了高斯噪声作为伪异常生成器的统计有效性,增强了该方法广泛适用性。实证评估表明,我们的可控过拟合方法在单类和多类异常检测任务中均达到了最先进(SOTA)的性能,从而将过拟合重新定义为一种强大策略而非局限。

英文摘要:

Overfitting has traditionally been viewed as detrimental to anomaly detection, where excessive generalization often limits models' sensitivity to subtle anomalies. Our work challenges this conventional view by introducing Controllable Overfitting-based Anomaly Detection (COAD), a novel framework that strategically leverages overfitting to enhance anomaly discrimination capabilities. We propose the Aberrance Retention Quotient (ARQ), a novel metric that systematically quantifies the extent of overfitting, enabling the identification of an optimal golden overfitting interval wherein model sensitivity to anomalies is maximized without sacrificing generalization. To comprehensively capture how overfitting affects detection performance, we further propose the Relative Anomaly Distribution Index (RADI), a metric superior to traditional AUROC by explicitly modeling the separation between normal and anomalous score distributions. Theoretically, RADI leverages ARQ to track and evaluate how overfitting impacts anomaly detection, offering an integrated approach to understanding the relationship between overfitting dynamics and model efficacy. We also rigorously validate the statistical efficacy of Gaussian noise as pseudo-anomaly generators, reinforcing the method's broad applicability. Empirical evaluations demonstrate that our controllable overfitting method achieves State-Of-The-Art(SOTA) performance in both one-class and multi-class anomaly detection tasks, thus redefining overfitting as a powerful strategy rather than a limitation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑