AHA:可扩展的替代历史分析用于操作时间序列应用
AHA: Scalable Alternative History Analysis for Operational Timeseries Applications
AI总结:
AHA通过高效处理高维时间序列数据,提供高准确性与低成本的替代历史分析解决方案。
摘要:
许多操作系统会收集关于用户/系统的高维时间序列数据,这些数据涉及关键性能指标。例如,互联网服务提供商、内容分发网络和视频交付服务会收集与用户会话相关的用户体验指标,这些指标通常伴随着元数据(如位置、设备、ISP)。在这样的历史数据上,操作人员和数据分析师经常需要进行回顾性分析;例如,分析异常检测算法、尝试不同的警报配置、评估新算法等。我们将此类工作负载称为操作数据集的替代历史分析。我们展示在这样的设置中,传统数据处理解决方案(如数据仓库、采样、草图、大数据系统)要么具有高昂的操作成本,要么无法保证准确的回放。我们设计并实现了一个系统,称为AHA(替代历史分析),以克服这两个挑战,为高维数据提供成本效益和准确性。AHA的设计基于关于此类工作负载的分析和实证洞察:1)底层统计的可分解性;2)在属性-值组合方面,活跃子群体数量的稀疏性;3)现代分析数据库中聚合操作的效率结构。通过多个真实世界数据集以及一家大型视频分析公司生产流水线的案例研究,我们证明AHA在广泛的下游任务中提供了100%的准确性,并且与传统方法相比,总拥有成本(即计算+存储)低至85倍。
英文摘要:
Many operational systems collect high-dimensional timeseries data about users/systems on key performance metrics. For instance, ISPs, content distribution networks, and video delivery services collect quality of experience metrics for user sessions associated with metadata (e.g., location, device, ISP). Over such historical data, operators and data analysts often need to run retrospective analysis; e.g., analyze anomaly detection algorithms, experiment with different configurations for alerts, evaluate new algorithms, and so on. We refer to this class of workloads as alternative history analysis for operational datasets. We show that in such settings, traditional data processing solutions (e.g., data warehouses, sampling, sketching, big-data systems) either pose high operational costs or do not guarantee accurate replay. We design and implement a system, called AHA (Alternative History Analytics), that overcomes both challenges to provide cost efficiency and fidelity for high-dimensional data. The design of AHA is based on analytical and empirical insights about such workloads: 1) the decomposability of underlying statistics; 2) sparsity in terms of active number of subpopulations over attribute-value combinations; and 3) efficiency structure of aggregation operations in modern analytics databases. Using multiple real-world datasets and as well as case-studies on production pipelines at a large video analytics company, we show that AHA provides 100% accuracy for a broad range of downstream tasks and up to 85x lower total cost of ownership (i.e., compute + storage) compared to conventional methods.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。