用并行向量Quantization -- PVQ实现流数据采样
Sampling Streaming Data with Parallel Vector Quantization -- PVQ
摘要：
企业数据在云中的积累吸引了更多的企业应用程序到云计算中，从而产生了数据引力。因此，网络流量变得更加以云为中心。这种以云为中心的流量的增加对由于类不平衡而设计用于流数据的学习系统提出了新的挑战。类的数量在从数据流构建的分类器的准确性中起着至关重要的作用。在本文中，我们提出了一种基于矢量量化的采样方法，它大大减少了数据流中的类不平衡。我们通过使用常用的ML模型构建方法对网络流量和异常数据集进行实验来证明其有效性; TensorFlow后端的多层感知器，支持向量机，K-最近邻和随机森林。我们使用并行处理、批处理和随机选择样本构建模型。我们表明，分类模型的准确性提高时，数据流与我们的方法进行预处理。我们使用这些分类器的开箱即用超参数和自动sklearn进行超参数优化。
英文摘要：
Accumulation of corporate data in the cloud has attracted more enterprise applications to the cloud creating data gravity. As a consequence, network traffic has become more cloud centric. This increase in cloud centric traffic poses new challenges in designing learning systems for streaming data due to class imbalance. The number of classes plays a vital role in the accuracy of the classifiers built from the data streams. In this paper, we present a vector quantization-based sampling method, which substantially reduces the class imbalance in data streams. We demonstrate its effectiveness by conducting experiments on network traffic and anomaly dataset with commonly used ML model building methods; Multilayered Perceptron on TensorFlow backend, Support Vector Machines, K-Nearest Neighbour, and Random Forests. We built models using parallel processing, batch processing, and randomly selecting samples. We show that the accuracy of classification models improves when the data streams are pre-processed with our method. We used out of the box hyper-parameters of these classifiers and auto sklearn for hyperparameter optimization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
