捕获大容量流处理中的数据不确定性
Capturing Data Uncertainty in High-Volume Stream Processing
摘要：
我们提出了一个数据流系统的设计和开发，捕捉数据的不确定性，从数据汇集查询处理，最终结果生成。我们的系统专注于自然建模为连续随机变量的数据。对于这样的数据，我们的系统采用基于概率和统计理论的方法来捕获数据的不确定性，并将这种方法集成到大容量流处理中。我们的系统的第一个组成部分捕获来自传感设备的原始数据流的不确定性。由于这样的原始数据流可能是高噪声的，并且可能没有携带足够的信息用于查询处理，我们的系统采用数据生成过程的概率模型和流速度推断来将原始数据转换为具有不确定性度量的所需格式。第二个组件捕获数据通过查询操作符传播时的不确定性。为了有效地量化查询操作符的结果不确定性，我们探索了各种基于概率和统计理论的技术来以流速度计算结果分布。我们目前正在与一组科学家合作，使用从危险天气监测和对象跟踪和监测领域（最终在实际系统中）收集的痕迹来评估我们的系统。
英文摘要：
We present the design and development of a data stream system that captures data uncertainty from data collection to query processing to final result generation. Our system focuses on data that is naturally modeled as continuous random variables. For such data, our system employs an approach grounded in probability and statistical theory to capture data uncertainty and integrates this approach into high-volume stream processing. The first component of our system captures uncertainty of raw data streams from sensing devices. Since such raw streams can be highly noisy and may not carry sufficient information for query processing, our system employs probabilistic models of the data generation process and stream-speed inference to transform raw data into a desired format with an uncertainty metric. The second component captures uncertainty as data propagates through query operators. To efficiently quantify result uncertainty of a query operator, we explore a variety of techniques based on probability and statistical theory to compute the result distribution at stream speed. We are currently working with a group of scientists to evaluate our system using traces collected from the domains of (and eventually in the real systems for) hazardous weather monitoring and object tracking and monitoring.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
