大型工业数据库管理系统中的Flaky测试：固定问题报告的实证研究
Flaky Tests in a Large Industrial Database Management System: An Empirical Study of Fixed Issue Reports for SAP HANA
AI总结：
本研究通过分析SAP HANA的问题报告，发现并发问题是最常见的测试不稳定性原因，并提出利用LLMs进行标记的方法以提高测试稳定性分析的效率。
摘要：
Flaky测试在多次执行相同版本的源代码时会产生不同的结果。因此，它们为代码质量提供了模糊的信号，并干扰了代码更改的自动化评估。尽管多种因素可能导致测试不稳定性，但解决测试不稳定性的方法通常针对特定原因进行调整。然而，测试不稳定的常见根本原因可能因编程语言、应用领域或软件项目的规模而异。由于手动标记不稳定的测试是耗时且繁琐的，本工作提出了一种利用模型内部和模型间一致性的LLMs-as-annotators方法，用于标记与固定不稳定性问题相关的相关根本原因类别。这使我们能够了解问题报告中普遍的不稳定性类别。我们在SAP HANA的背景下评估了我们的标记方法。我们的结果表明，SAP HANA的测试最常遇到与并发性相关的问题（23%，559个分析的问题报告中有130个）。此外，我们的结果表明，不同的测试类型面临不同的不稳定性挑战。因此，我们鼓励未来关于不稳定性缓解的研究考虑在不同测试类型上评估所提出方法的泛化能力。
英文摘要：
Flaky tests yield different results when executed multiple times for the same version of the source code. Thus, they provide an ambiguous signal about the quality of the code and interfere with the automated assessment of code changes. While a variety of factors can cause test flakiness, approaches to fix flaky tests are typically tailored to address specific causes. However, the prevalent root causes of flaky tests can vary depending on the programming language, application domain, or size of the software project. Since manually labeling flaky tests is time-consuming and tedious, this work proposes an LLMs-as-annotators approach that leverages intra- and inter-model consistency to label issue reports related to fixed flakiness issues with the relevant root cause category. This allows us to gain an overview of prevalent flakiness categories in the issue reports. We evaluated our labeling approach in the context of SAP HANA, a large industrial database management system. Our results suggest that SAP HANA's tests most commonly suffer from issues related to concurrency (23%, 130 of 559 analyzed issue reports). Moreover, our results suggest that different test types face different flakiness challenges. Therefore, we encourage future research on flakiness mitigation to consider evaluating the generalizability of proposed approaches across different test types.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
