物联网传感器大数据异常检测与冗余消除
Anomaly Detection and Redundancy Elimination of Big Sensor Data in Internet of Things
摘要：
在大数据和物联网时代，海量的传感器数据随着物联网而聚集。传感器网络捕获的大量数据被认为包含了非常有用和有价值的信息。然而，由于各种原因，接收到的传感器数据经常出现异常。因此，需要有效的异常检测方法来保证这些传感器节点收集的数据的质量。由于传感器数据通常在时间和空间上是相关的，因此并非所有收集的数据都对进一步的数据处理和分析有价值。为了消除采集到的海量传感器数据中的冗余信息，需要对数据进行预处理.在本文中，所提出的工作定义了一个传感器数据预处理框架。它主要由两部分组成，传感器数据异常检测和传感器数据冗余消除。第一部分提出了基于主统计量分析和贝叶斯网络的传感器数据异常检测方法。然后，提出了基于静态贝叶斯网络（SBN）和动态贝叶斯网络（DBN）的传感器数据冗余消除方法。提出了用于消除静态数据集中冗余数据的静态传感器数据冗余检测算法（SSDRDA）和用于实时消除冗余数据的实时传感器数据冗余检测算法（RSDRDA）。使用真实世界收集的传感器数据集验证了所提出方法的效率和有效性。
英文摘要：
In the era of big data and Internet of things, massive sensor data are gathered with Internet of things. Quantity of data captured by sensor networks are considered to contain highly useful and valuable information. However, for a variety of reasons, received sensor data often appear abnormal. Therefore, effective anomaly detection methods are required to guarantee the quality of data collected by those sensor nodes. Since sensor data are usually correlated in time and space, not all the gathered data are valuable for further data processing and analysis. Preprocessing is necessary for eliminating the redundancy in gathered massive sensor data. In this paper, the proposed work defines a sensor data preprocessing framework. It is mainly composed of two parts, i.e., sensor data anomaly detection and sensor data redundancy elimination. In the first part, methods based on principal statistic analysis and Bayesian network is proposed for sensor data anomaly detection. Then, approaches based on static Bayesian network (SBN) and dynamic Bayesian networks (DBNs) are proposed for sensor data redundancy elimination. Static sensor data redundancy detection algorithm (SSDRDA) for eliminating redundant data in static datasets and real-time sensor data redundancy detection algorithm (RSDRDA) for eliminating redundant sensor data in real-time are proposed. The efficiency and effectiveness of the proposed methods are validated using real-world gathered sensor datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
