arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2212.08540cs.SE
收藏

软件非功能属性自动改进基准的综合调查

A Comprehensive Survey of Benchmarks for Automated Improvement of Software's Non-Functional Properties

Aymeric Blot, Justyna Petke

AI总结:

针对软件非功能属性自动改进缺乏标准基准的问题,本调查系统梳理了386篇相关论文,发现执行时间最受关注(62%),多目标改进罕见(5%),并提供了基准选择建议,以促进该领域研究。

摘要:

性能是现代软件的一个关键质量属性。尽管近年来关于自动改进软件执行时间、能耗、内存消耗等方面的研究激增,但此类工作明显缺乏标准基准。同时,这些基准对当前软件的代表性也不明确。此外,非功能属性经常被单独作为改进目标,而忽略了可能对其他属性产生的负面影响。为了促进更多关于软件非功能属性自动改进的研究,我们进行了一项调查,收集了以往工作中使用的基准。我们考虑了5个主要的软件工程工作在线存储库:ACM数字图书馆、IEEE Xplore、Scopus、Google Scholar和ArXiV。我们收集了5000篇出版物(其中3749篇为唯一),并对其进行了系统审查,以识别实证改进软件非功能属性的工作。我们确定了386篇相关论文。我们发现,执行时间是最常被作为改进目标的属性(在62%的相关论文中),而多目标改进很少被考虑(5%)。静态方法占主导地位(在53%的论文中),而探索性方法(进化方法占18%,非进化方法占14%)在过去10年中越来越受欢迎。在386篇论文中,只有40%描述了使用基准套件(而非单一软件)的工作,其中SPEC最受欢迎(在33篇论文中被覆盖)。我们还为未来工作中的基准选择提供了建议,例如指出缺乏覆盖Python或JavaScript的工作。我们在专门网页https://bloa.github.io/nfunc_survey/上提供了386篇论文中发现的所有程序。我们希望这项工作能促进更多关于软件非功能属性自动改进的研究。

英文摘要:

Performance is a key quality of modern software. Although recent years have seen a spike in research on automated improvement of software's execution time, energy, memory consumption, etc., there is a noticeable lack of standard benchmarks for such work. It is also unclear how such benchmarks are representative of current software. Furthermore, frequently non-functional properties of software are targeted for improvement one-at-a-time, neglecting potential negative impact on other properties. In order to facilitate more research on automated improvement of non-functional properties of software, we conducted a survey gathering benchmarks used in previous work. We considered 5 major online repositories of software engineering work: ACM Digital Library, IEEE Xplore, Scopus, Google Scholar, and ArXiV. We gathered 5000 publications (3749 unique), which were systematically reviewed to identify work that empirically improves non-functional properties of software. We identified 386 relevant papers. We find that execution time is the most frequently targeted property for improvement (in 62% of relevant papers), while multi-objective improvement is rarely considered (5%). Static approaches are prevalent (in 53% of papers), with exploratory approaches (evolutionary in 18% and non-evolutionary in 14% of papers) increasingly popular in the last 10 years. Only 40% of 386 papers describe work that uses benchmark suites, rather than single software, of those SPEC is most popular (covered in 33 papers). We also provide recommendations for choice of benchmarks in future work, noting, e.g., lack of work that covers Python or JavaScript. We provide all programs found in the 386 papers on our dedicated webpage at https://bloa.github.io/nfunc_survey/ We hope that this effort will facilitate more research on the topic of automated improvement of software's non-functional properties.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑