云基准测试中的采样：批判性综述与方法学指南
Sampling in Cloud Benchmarking: A Critical Review and Methodological Guidelines
AI总结：
该研究针对云基准测试采样缺乏系统方法导致结果不可比的问题，通过批判性综述发现非概率采样普及等三大问题，提出方法学指南以提升研究的普适性、可复现性与可靠性。
摘要：
云基准测试会受到资源争用、网络延迟、硬件异构性等因素，以及基准测试设计中所做决策导致的性能波动影响。尤其是基准测试设计者的采样策略，会对基准测试结果产生显著影响。尽管这一事实已广为人知，但目前尚未有系统性方法能让采样结果具备可比性，也无法指导基准测试设计者选择基准测试中使用的采样策略。为找出系统性问题，我们对近期云计算研究中的采样工作开展了批判性综述。我们的分析发现了值得关注的趋势：（i）非概率采样的高普及率，（ii）对单一基准测试的过度依赖，以及（iii）样本访问受限。为解决这些问题并提升采样的透明度，我们为研究者和审稿人提出了方法学指南。我们希望本工作能助力提升研究结果的普适性、可复现性与可靠性。
英文摘要：
Cloud benchmarks suffer from performance fluctuations caused by resource contention, network latency, hardware heterogeneity, and other factors along with decisions taken in the benchmark design. In particular, the sampling strategy of benchmark designers can significantly influence benchmark results. Despite this well-known fact, no systematic approach has been devised so far to make sampling results comparable and guide benchmark designers in choosing their sampling strategy for use within benchmarks. To identify systematic problems, we critically review sampling in recent cloud computing research. Our analysis identifies concerning trends: (i) a high prevalence of non-probability sampling, (ii) over-reliance on a single benchmark, and (iii) restricted access to samples. To address these issues and increase transparency in sampling, we propose methodological guidelines for researchers and reviewers. We hope that our work contributes to improving the generalizability, reproducibility, and reliability of research results.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
