识别异常的舍入或截断数据
Identifying Excessively Rounded or Truncated Data
摘要:
所有数据都是数字化的,因此本质上是整数而不是真正的实数。通常,这不会引起困难,因为截断或舍入通常发生在噪声电平以下。然而,在某些情况下,当仪器或数据传输和存储系统的设计对数据或随后的数据分析的考虑不够理想时,数字化的效果可能与数据中包含的重要特征相当。在这些情况下,信息在截断过程中无可避免地丢失。虽然存在处理截断数据的技术,我们提出了一个简单的方法,使我们能够检测到这个问题之前的数据分析阶段。它基于最佳直方图分箱算法,该算法可以识别数字化的统计结构何时与数据集本身的统计结构相同。
英文摘要:
All data are digitized, and hence are essentially integers rather than true real numbers. Ordinarily this causes no difficulties since the truncation or rounding usually occurs below the noise level. However, in some instances, when the instruments or data delivery and storage systems are designed with less than optimal regard for the data or the subsequent data analysis, the effects of digitization may be comparable to important features contained within the data. In these cases, information has been irrevocably lost in the truncation process. While there exist techniques for dealing with truncated data, we propose a straightforward method that will allow us to detect this problem before the data analysis stage. It is based on an optimal histogram binning algorithm that can identify when the statistical structure of the digitization is on the order of the statistical structure of the data set itself.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。