arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.03071cs.SE
收藏

状态字段覆盖率:一种衡量预言机质量的指标

State Field Coverage: A Metric for Oracle Quality

IMDEA Software Institute(IMDEA软件研究所);University of Rio Cuarto(里奥夸尔托大学);Guangdong Technion-Israel Institute of Technology(广东以色列理工学院)

Facundo Molina, Nazareno Aguirre, Alessandra Gorla

AI总结:

针对现有预言机质量评估指标通用性不足或难以指导改进的问题,提出静态计算的状态字段覆盖率指标,实验证明其与预言机故障检测能力高度相关,可有效指导预言机优化。

摘要:

测试在发现软件缺陷方面的有效性,不仅取决于测试输入的特性及其对软件的测试 thoroughness( thoroughness 此处为测试覆盖程度),还取决于用于判断软件是否按预期运行的预言机(oracle)的质量。因此,评估预言机质量对提升测试流程的整体有效性至关重要。现有指标已被用于该目的,但它们要么无法为指导预言机改进提供全面依据,要么仅针对特定类型的预言机定制,从而限制了其通用性。\n 本文提出了状态字段覆盖率(state field coverage),这是一种用于评估预言机质量的新型指标。该指标衡量的是,在测试执行过程中,预言机可能访问的、由类字段静态定义的对象状态的比例。我们提出该指标的核心直觉是:状态字段覆盖率更高的预言机更有可能检测到被测软件中的故障,因为它们会检查更大比例的对象状态,以判断测试是否通过。\n 我们实现了一种静态计算状态字段覆盖率指标的机制。由于是静态计算,该指标效率很高,并且能够通过识别尚未被检查的状态字段,为改进测试预言机提供直接指导。我们通过涉及273个表示不变量(representation invariants)和249,027条测试断言的实验对状态字段覆盖率进行了评估。结果表明,状态字段覆盖率是一种非常适合评估预言机质量的指标,因为它与以变异得分(mutation score)衡量的预言机故障检测能力高度相关。

英文摘要:

The effectiveness of testing in uncovering software defects depends not only on the characteristics of the test inputs and how thoroughly they exercise the software, but also on the quality of the oracles used to determine whether the software behaves as expected. Therefore, assessing the quality of oracles is crucial to improve the overall effectiveness of the testing process. Existing metrics have been used for this purpose, but they either fail to provide a comprehensive basis for guiding oracle improvement, or they are tailored to specific types of oracles, thus limiting their generality. In this paper, we introduce state field coverage, a novel metric for assessing oracle quality. This metric measures the proportion of an object's state, as statically defined by its class fields, that an oracle may access during test execution. The main intuition of our metric is that oracles with a higher state field coverage are more likely to detect faults in the software under analysis, as they inspect a larger portion of the object states to determine whether tests pass or not. We implement a mechanism to statically compute the state field coverage metric. Being statically computed, the metric is efficient and provides direct guidance for improving test oracles by identifying state fields that remain unexamined. We evaluate state field coverage through experiments involving 273 representation invariants and 249,027 test assertions. The results show that state field coverage is a well-suited metric for assessing oracle quality, as it strongly correlates with the oracles' fault-detection ability, measured by mutation score.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑