arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.23136cs.LG
收藏

基于聚类分析和视觉专家准则的异常值检测方法

A method for outlier detection based on cluster analysis and visual expert criteria

Madrid Open University, UDIMA(马德里开放大学);Universidad Politécnica de Madrid(马德里理工大学)

Juan A. Lara, David Lizcano, Víctor Rampérez, Javier Soriano

AI总结:

本文提出一种基于聚类分析和视觉专家准则的异常值检测方法,利用四个准则模拟专家视觉识别,在稳定测量和脑电图数据上验证,假阳性率低于2%,可靠性超99%。

摘要:

异常值检测是广泛领域中发生的一个重要问题。异常值是欺诈行为、机械故障、人为错误或仅仅是自然偏差的结果。许多数据挖掘应用执行异常值检测,通常作为初步步骤,以过滤掉异常值并构建更具代表性的模型。在本文中,我们提出了一种基于聚类过程的异常值检测方法。本文提出的方案背后的目的是克服许多现有异常值检测技术的特殊性,这些技术未能考虑领域对象的固有分散性。该异常值检测方法基于四个准则,这些准则旨在表示人类(各领域专家)在分析聚类后如何在对象集合中视觉识别异常值。这相对于其他基于聚类的异常值检测技术具有优势,后者基于对聚类的纯数值分析。我们的方案已在来自两个不同领域的数据(特别是时间序列)上进行了评估,结果令人满意:稳定测量学(研究人类平衡相关功能的医学分支)和脑电图(用于诊断神经系统疾病的神经学检查)。为验证所提方法,我们研究了方法的异常值检测和运行时间效率。回归分析的结果证实,我们的方案在不同领域中检测异常数据是有用的,假阳性率低于2%,可靠性大于99%。

英文摘要:

Outlier detection is an important problem occurring in a wide range of areas. Outliers are the outcome of fraudulent behaviour, mechanical faults, human error, or simply natural deviations. Many data mining applications perform outlier detection, often as a preliminary step in order to filter out outliers and build more representative models. In this paper, we propose an outlier detection method based on a clustering process. The aim behind the proposal outlined in this paper is to overcome the specificity of many existing outlier detection techniques that fail to take into account the inherent dispersion of domain objects. The outlier detection method is based on four criteria designed to represent how human beings (experts in each domain) visually identify outliers within a set of objects after analysing the clusters. This has an advantage over other clustering-based outlier detection techniques that are founded on a purely numerical analysis of clusters. Our proposal has been evaluated, with satisfactory results, on data (particularly time series) from two different domains: stabilometry, a branch of medicine studying balance-related functions in human beings and electroencephalography (EEG), a neurological exploration used to diagnose nervous system disorders. To validate the proposed method, we studied method outlier detection and efficiency in terms of runtime. The results of regression analyses confirm that our proposal is useful for detecting outlier data in different domains, with a false positive rate of less than 2% and a reliability greater than 99%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑