大数据汇集中需要多少样本：一种差异消息重要性度量
How Many Samples Required in Big Data Collection: A Differential Message Importance Measure
摘要：
信息收集是大数据中的一个基本问题，其中样本集的大小起着非常重要的作用。这项工作考虑到信息收集过程中考虑到消息的重要性。与微分熵类似，我们定义了微分消息重要性度量（DMIM）作为连续随机变量的消息重要性度量。证明了DMIM的变化可以描述一组样本值的分布与理论分布之间的差距。实际上，DMIM的偏差等价于Kolmogorov-Smirnov统计量，但它提供了一种表征分布拟合优度的新方法。数值结果表明DMIM的一些基本性质和建议的近似值的准确性。随着DMIM偏差的减小，经验分布逐渐逼近真实分布，这对实际系统中采样点的选取具有指导意义。
英文摘要：
Information collection is a fundamental problem in big data, where the size of sampling sets plays a very important role. This work considers the information collection process by taking message importance into account. Similar to differential entropy, we define differential message importance measure (DMIM) as a measure of message importance for continuous random variable. It is proved that the change of DMIM can describe the gap between the distribution of a set of sample values and a theoretical distribution. In fact, the deviation of DMIM is equivalent to Kolmogorov-Smirnov statistic, but it offers a new way to characterize the distribution goodness-of-fit. Numerical results show some basic properties of DMIM and the accuracy of the proposed approximate values. Furthermore, it is also obtained that the empirical distribution approaches the real distribution with decreasing of the DMIM deviation, which contributes to the selection of suitable sampling points in actual system.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
