arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2212.13359cs.SEcs.AIcs.LG
收藏

基于贝叶斯神经网络的高度可配置软件系统不确定性性能预测

Uncertainty-Aware Performance Prediction for Highly Configurable Software Systems via Bayesian Neural Networks

RMIT University(皇家墨尔本理工大学);Huaqiao University(华侨大学);The University of Newcastle(纽卡斯尔大学)

Huong Ha, Zongwen Fan, Hongyu Zhang

摘要:

可配置软件系统在许多重要的应用领域中被采用。了解系统在所有配置下的性能对于防止错误配置导致的潜在性能问题至关重要。然而,由于配置的数量可能非常大,因此不可能在所有配置下测量系统性能。因此,一种常见的方法是从有限的测量数据中构建预测模型,以预测所有配置的性能作为标量值。然而,已经指出,存在来自数据汇集或建模过程的不同来源的不确定性,这可能使标量预测不一定准确。为了解决这个问题,我们提出了一种基于贝叶斯深度学习的方法,即BDLPerf,它可以将不确定性纳入预测模型。BDLPerf可以提供配置性能的标量预测以及这些标量预测的相应置信区间。我们还开发了一种新的不确定性校准技术,以确保贝叶斯预测模型生成的置信区间的可靠性。最后,我们提出了一种有效的超参数调整技术,以便在合理的时间内训练预测模型,同时实现高精度。我们在10个真实系统上的实验结果表明,BDLPerf在标量性能预测和置信区间估计方面都比现有方法具有更高的准确性。

英文摘要:

Configurable software systems are employed in many important application domains. Understanding the performance of the systems under all configurations is critical to prevent potential performance issues caused by misconfiguration. However, as the number of configurations can be prohibitively large, it is not possible to measure the system performance under all configurations. Thus, a common approach is to build a prediction model from a limited measurement data to predict the performance of all configurations as scalar values. However, it has been pointed out that there are different sources of uncertainty coming from the data collection or the modeling process, which can make the scalar predictions not certainly accurate. To address this problem, we propose a Bayesian deep learning based method, namely BDLPerf, that can incorporate uncertainty into the prediction model. BDLPerf can provide both scalar predictions for configurations' performance and the corresponding confidence intervals of these scalar predictions. We also develop a novel uncertainty calibration technique to ensure the reliability of the confidence intervals generated by a Bayesian prediction model. Finally, we suggest an efficient hyperparameter tuning technique so as to train the prediction model within a reasonable amount of time whilst achieving high accuracy. Our experimental results on 10 real-world systems show that BDLPerf achieves higher accuracy than existing approaches, in both scalar performance prediction and confidence interval estimation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑