arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.01984cs.CV
收藏

AUPIMO:以高速度和低容差重新定义视觉异常检测基准

AUPIMO: Redefining Visual Anomaly Detection Benchmarks with High Speed and Low Tolerance

Mines Paris, PSL University(巴黎国立高等矿业学院,巴黎文理研究大学);Intel(英特尔)

Joao P. C. Bertoldo, Dick Ameln, Ashwin Vaidya, Samet Akçay

AI总结:

本文提出新指标 PIMO,以每图像召回率和仅基于正常图像的低假阳性容差,弥补 AUROC 与 AUPRO 的不足,重新定义视觉异常检测基准并挑战 MVTec AD 和 VisA 已被解决的观点。

摘要:

视觉异常检测研究的最新进展表明,在 MVTec 和 VisA 等公开基准数据集上,AUROC 和 AUPRO 分数已趋近于完美召回,给人一种这些基准已接近被解决的印象。然而,高 AUROC 和 AUPRO 分数并不总能反映定性性能,这限制了这些指标在实际应用中的有效性。我们认为,由于缺乏合适的评估指标而造成的人为上限制约了该领域的发展,因此重新审视用于评价算法的评估指标至关重要。为此,我们提出了 Per-IMage Overlap(PIMO),这是一种新指标,旨在解决 AUROC 和 AUPRO 的不足。PIMO 保留了现有指标基于召回率的特性,但引入了两个区别:曲线(以及相应的曲线下面积)按每张图像分配,其 X 轴仅依赖于正常图像。按图像计算召回率简化了实例分数索引,并且对噪声标注更具鲁棒性。正如我们所展示的,它还加快了计算速度,并使得可以使用统计检验来比较模型。通过对正常图像上的假阳性施加低容差,PIMO 提供了一种增强的模型验证流程,并突出了跨数据集的性能差异。我们的实验表明,PIMO 提供了实用优势和细致的性能洞察,重新定义了异常检测基准——尤其是挑战了 MVTec AD 和 VisA 数据集已被当代模型解决这一认知。可在 GitHub 获取:https://github.com/jpcbertoldo/aupimo。

英文摘要:

Recent advances in visual anomaly detection research have seen AUROC and AUPRO scores on public benchmark datasets such as MVTec and VisA converge towards perfect recall, giving the impression that these benchmarks are near-solved. However, high AUROC and AUPRO scores do not always reflect qualitative performance, which limits the validity of these metrics in real-world applications. We argue that the artificial ceiling imposed by the lack of an adequate evaluation metric restrains progression of the field, and it is crucial that we revisit the evaluation metrics used to rate our algorithms. In response, we introduce Per-IMage Overlap (PIMO), a novel metric that addresses the shortcomings of AUROC and AUPRO. PIMO retains the recall-based nature of the existing metrics but introduces two distinctions: the assignment of curves (and respective area under the curve) is per-image, and its X-axis relies solely on normal images. Measuring recall per image simplifies instance score indexing and is more robust to noisy annotations. As we show, it also accelerates computation and enables the usage of statistical tests to compare models. By imposing low tolerance for false positives on normal images, PIMO provides an enhanced model validation procedure and highlights performance variations across datasets. Our experiments demonstrate that PIMO offers practical advantages and nuanced performance insights that redefine anomaly detection benchmarks -- notably challenging the perception that MVTec AD and VisA datasets have been solved by contemporary models. Available on GitHub: https://github.com/jpcbertoldo/aupimo.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑