arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.05692cs.DC
收藏

隐私保护的数据分析运行时指标共享用于性能建模

Privacy-Preserving Sharing of Data Analytics Runtime Metrics for Performance Modeling

Jonathan Will, Dominik Scheinert, Jan Bode, Cedric Kring, Seraphin Zunzer, Lauritz Thamsen

AI总结:

提出基于差分隐私和数据合成的隐私保护方法共享运行时指标,用于大规模数据分析性能建模,在数据稀缺时保持预测精度。

摘要:

大规模数据分析工作负载的性能建模可以提高集群资源分配和作业调度的效率。然而,这些工作负载的性能受到诸多因素的影响,如作业输入和分配的集群资源。因此,性能模型需要大量的训练数据。这些数据可以通过在合作组织之间交换运行时指标来获得。然而,并非所有组织都愿意公开披露此类元数据。我们提出了一种基于差分隐私和数据合成的隐私保护方法来共享运行时指标。我们对来自736个Spark作业执行的性能数据的评估表明,完全匿名的训练数据在很大程度上保持了性能预测的准确性,尤其是在可用原始数据极少的情况下。当可用原始数据样本不超过30个时,使用合成训练数据平均仅导致性能模型准确性降低百分之一。

英文摘要:

Performance modeling for large-scale data analytics workloads can improve the efficiency of cluster resource allocations and job scheduling. However, the performance of these workloads is influenced by numerous factors, such as job inputs and the assigned cluster resources. As a result, performance models require significant amounts of training data. This data can be obtained by exchanging runtime metrics between collaborating organizations. Yet, not all organizations may be inclined to publicly disclose such metadata. We present a privacy-preserving approach for sharing runtime metrics based on differential privacy and data synthesis. Our evaluation on performance data from 736 Spark job executions indicates that fully anonymized training data largely maintains performance prediction accuracy, particularly when there is minimal original data available. With 30 or fewer available original data samples, the use of synthetic training data resulted only in a one percent reduction in performance model accuracy on average.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑