用于等价变异体检测的大语言模型：我们进展到了什么程度？
Large Language Models for Equivalent Mutant Detection: How Far Are We?
AI总结：
本文在3,302对方法级Java变异体上实证评估LLM的等价变异体检测能力，发现微调代码嵌入策略F1平均提升35.69%，并兼具效果与成本优势。
摘要：
变异测试对于确保软件质量至关重要。然而，众所周知，等价变异体的存在会引入冗余成本和偏差问题，从而阻碍变异测试在实际应用中的有效性。尽管已经提出了许多等价变异体检测（EMD）技术，但由于训练数据稀缺以及向未见过的变异体泛化所面临的挑战，这些技术表现出一定局限性。近年来，大语言模型（LLM）已被广泛应用于各种代码相关任务，并通过更准确地捕捉程序语义展现出卓越性能。然而，LLM在等价变异体检测中的性能在很大程度上仍不明确。本文在3,302对方法级Java变异体上开展实证研究，以全面考察LLM用于等价变异体检测的有效性和效率。具体而言，我们评估LLM相对于现有EMD技术的性能，考察LLM的各种策略，评价EMD技术之间的正交性，并测量训练和推理的时间开销。研究结果表明，基于LLM的技术显著优于现有技术（即在F1-score方面平均提升35.69%），其中微调代码嵌入策略最为有效。此外，基于LLM的技术在成本（相对较低的训练和推理时间）与有效性之间实现了出色的平衡。基于这些发现，我们进一步讨论了模型规模和嵌入质量的影响，并为未来研究提供了若干有前景的方向。本工作首次考察LLM在等价变异体检测中的应用，证实了其有效性和效率。
英文摘要：
Mutation testing is vital for ensuring software quality. However, the presence of equivalent mutants is known to introduce redundant cost and bias issues, hindering the effectiveness of mutation testing in practical use. Although numerous equivalent mutant detection (EMD) techniques have been proposed, they exhibit limitations due to the scarcity of training data and challenges in generalizing to unseen mutants. Recently, large language models (LLMs) have been extensively adopted in various code-related tasks and have shown superior performance by more accurately capturing program semantics. Yet the performance of LLMs in equivalent mutant detection remains largely unclear. In this paper, we conduct an empirical study on 3,302 method-level Java mutant pairs to comprehensively investigate the effectiveness and efficiency of LLMs for equivalent mutant detection. Specifically, we assess the performance of LLMs compared to existing EMD techniques, examine the various strategies of LLMs, evaluate the orthogonality between EMD techniques, and measure the time overhead of training and inference. Our findings demonstrate that LLM-based techniques significantly outperform existing techniques (i.e., the average improvement of 35.69% in terms of F1-score), with the fine-tuned code embedding strategy being the most effective. Moreover, LLM-based techniques offer an excellent balance between cost (relatively low training and inference time) and effectiveness. Based on our findings, we further discuss the impact of model size and embedding quality, and provide several promising directions for future research. This work is the first to examine LLMs in equivalent mutant detection, affirming their effectiveness and efficiency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
