arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2607.00511cs.SE
收藏

大型语言模型用于多语言等价变异检测:一项扩展的实证研究

Large Language Models for Multi-Lingual Equivalent Mutant Detection: An Extended Empirical Study

Tianjin University(天津大学);College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部);Kyushu University(九州大学)

Honglin Shu, Zhao Tian, Dong Wang, Junji Yu, Jiazhe Zhang, Xuejie Cao, Junjie Chen, Yasutaka Kamei

AI总结:

本文首次全面实证研究大型语言模型(LLMs)在等价变异检测(EMD)中的应用,使用Java和C变异对评估其性能,发现基于LLM的方法在F1分数上优于传统方法,且微调后的代码嵌入策略检测精度最高,同时展现出跨语言泛化能力。

摘要:

变异测试是一种确保软件质量的强大技术。然而,等价变异的存在引入了不必要的成本和偏差,限制了其实际有效性。尽管已经提出了许多等价变异检测(EMD)方法,但它们常常面临不同的挑战:纯代码分析方法可能受限于对特定编译器基础设施的依赖,而现有的机器学习方法仍然受限于稀缺的训练数据和对未见变异的有限泛化能力。大型语言模型(LLMs)最近通过更好地捕捉程序语义,在多种代码相关任务中展现出卓越的性能。然而,它们在EMD中的潜力仍然很大程度上未被探索,特别是在多语言背景下。本文首次对LLMs用于EMD进行了全面的实证研究,使用3,302个Java和1,088个C变异对,与最先进的方法进行基准测试,探索策略变化,评估效率,并评估跨语言泛化能力。实验结果表明,基于LLM的方法在F1分数上高于评估的传统方法,其中微调后的代码嵌入在测试策略中取得了最高的检测精度。此外,基于LLM的方法在有效性和效率之间取得了实际平衡,推理时间与现有机器学习模型相当。重要的是,微调后的LLMs展示了跨编程语言的可测量泛化能力。这些发现确立了LLMs作为应对等价变异检测长期挑战的可行且高效的方法,为推进变异测试的实际应用提供了新方向。

英文摘要:

Mutation testing is a powerful technique for ensuring software quality. However, the presence of equivalent mutants introduces unnecessary costs and biases, limiting its practical effectiveness. Although numerous equivalent mutant detection (EMD) methods have been proposed, they often face distinct challenges: pure-code analysis methods can be limited by their reliance on specific compiler infrastructures, while existing machine-learning approaches remain constrained by scarce training data and limited generalization to unseen mutants. Large language models (LLMs) have recently demonstrated remarkable performance across diverse code-related tasks by better capturing program semantics. Yet their potential for EMD remains largely unexplored, particularly in the multi-lingual context. This paper presents the first comprehensive empirical study on LLMs for EMD, using 3,302 Java and 1,088 C mutant pairs to benchmark against state-of-the-art methods, explore strategy variations, assess efficiency, and evaluate cross-lingual generalization. Experimental results show that LLM-based approaches achieve higher F1-scores than the evaluated traditional methods, with fine-tuned code embedding yielding the highest detection accuracy among the tested strategies. Moreover, LLM-based approaches strike a practical balance between effectiveness and efficiency with inference times comparable to existing machine-learning models. Importantly, fine-tuned LLMs demonstrate measurable generalization across programming languages. These findings establish LLMs as a viable and efficient approach for tackling the longstanding challenge of equivalent mutant detection, offering new directions for advancing mutation testing in practice.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑