Data Nuggets：一种在保留数据结构的同时缩减大数据的方法
Data Nuggets: A Method for Reducing Big Data While Preserving Data Structure
AI总结：
针对大数据聚类受限于 P×N(N-1) 计算量且随机抽样易破坏边缘结构的问题，提出由中心、权重和尺度参数构成的 data nuggets 缩减方法，证明协方差估计量一致性，并在百万级流式细胞术数据上完成 PCA 与加权 K-means 分析。
摘要：
具有 N×P 维（其中 N 极大）的大数据给数据分析带来了新的挑战，尤其是在创建有意义的数据簇方面。诸如 K-means 或层次聚类之类的聚类技术，是对大型数据集执行探索性分析的常用方法。遗憾的是，由于阶为 P×N(N-1) 的计算所产生的内存或时间限制，这些方法并不总能应用于大数据。为规避这一问题，通常的做法是将聚类技术应用于从数据集中抽取的随机样本；然而，其缺点在于数据集的结构（尤其是边缘处的结构）不一定能够得到保持。我们通过“data nuggets（数据金块）”的概念提出了一种新的解决方案，它将大型数据集缩减为一小批数据 nugget，每个 nugget 都包含中心、权重和尺度参数。随后，这些 data nuggets 被输入到以计算上更高效的方式计算主成分分析和聚类等方法的算法中。我们证明了基于 data nuggets 的协方差估计量的一致性，并将 data nuggets 方法应用于一个包含超过一百万个观测值的流式细胞术数据集，利用 PCA 和面向加权观测值的 K-means 聚类执行探索性分析。本文的补充材料可在线获取。
英文摘要：
Big data, with NxP dimension where N is extremely large, has created new challenges for data analysis, particularly in the realm of creating meaningful clusters of data. Clustering techniques, such as K-means or hierarchical clustering are popular methods for performing exploratory analysis on large datasets. Unfortunately, these methods are not always possible to apply to big data due to memory or time constraints generated by calculations of order PxN(N-1). To circumvent this problem, typically, the clustering technique is applied to a random sample drawn from the dataset: however, a weakness is that the structure of the dataset, particularly at the edges, is not necessarily maintained. We propose a new solution through the concept of "data nuggets", which reduce a large dataset into a small collection of nuggets of data, each containing a center, weight, and scale parameter. The data nuggets are then input into algorithms that compute methods such as principal components analysis and clustering in a more computationally efficient manner. We show the consistency of the data nuggets-based covariance estimator and apply the methodology of data nuggets to perform exploratory analysis of a flow cytometry dataset containing over one million observations using PCA and K-means clustering for weighted observations. Supplementary materials for this article are available online.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
