arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.01253cs.SE
收藏

TraceLLM:利用大型语言模型与提示工程增强需求可追溯性

TraceLLM: Leveraging Large Language Models with Prompt Engineering for Enhanced Requirements Traceability

KFUPM(国王法赫德石油矿物大学);Imam Abdulrahman Bin Faisal University(伊玛目阿卜杜勒拉赫曼·本·费萨尔大学);SDAIA–KFUPM Joint Research Center of Artificial Intelligence, KFUPM(沙特数据与人工智能中心-国王法赫德石油矿物大学人工智能联合研究中心);University of Quebec in Montreal(蒙特利尔魁北克大学)

Nouf Alturayeif, Irfan Ahmad, Jameleddine Hassine

AI总结:

本文提出TraceLLM框架,通过提示工程和示例选择,利用大型语言模型在零样本和少样本设置下提升需求可追溯性,在多个基准数据集上取得最优F2分数。

摘要:

需求可追溯性,即建立和维护需求与各种软件开发工件之间关系的过程,对于确保系统完整性并在软件开发生命周期(SDLC)中满足需求至关重要。传统方法,包括手动和信息检索模型,劳动密集、易出错且受限于低精度。最近,大型语言模型(LLMs)通过高级语言理解展示了支持软件工程任务的潜力。然而,在系统化设计和评估用于提取准确追溯链接的提示方面存在显著差距。本文介绍了TraceLLM,一个通过提示工程和示例选择来增强需求可追溯性的系统框架。我们的方法包括严格的数据集划分、迭代提示优化、用上下文角色和领域知识进行丰富,以及在零样本和少样本设置下的评估。我们使用八个最先进的LLM在四个代表不同领域(航空航天、医疗保健)和工件类型(需求、设计元素、测试用例、法规)的基准数据集上评估提示的泛化性和鲁棒性。TraceLLM取得了最先进的F2分数,优于传统的IR基线、微调模型和先前基于LLM的方法。我们还探讨了示例选择策略的影响,发现标签感知、基于多样性的采样特别有效。总体而言,我们的发现强调,可追溯性性能不仅取决于模型能力,还关键取决于提示工程的质量。此外,所取得的性能表明,TraceLLM可以支持半自动化的可追溯性工作流程,其中候选链接由人类分析师审查和验证。

英文摘要:

Requirements traceability, the process of establishing and maintaining relationships between requirements and various software development artifacts, is paramount for ensuring system integrity and fulfilling requirements throughout the Software Development Life Cycle (SDLC). Traditional methods, including manual and information retrieval models, are labor-intensive, error-prone, and limited by low precision. Recently, Large Language Models (LLMs) have demonstrated potential for supporting software engineering tasks through advanced language comprehension. However, a substantial gap exists in the systematic design and evaluation of prompts tailored to extract accurate trace links. This paper introduces TraceLLM, a systematic framework for enhancing requirements traceability through prompt engineering and demonstration selection. Our approach incorporates rigorous dataset splitting, iterative prompt refinement, enrichment with contextual roles and domain knowledge, and evaluation across zero- and few-shot settings. We assess prompt generalization and robustness using eight state-of-the-art LLMs on four benchmark datasets representing diverse domains (aerospace, healthcare) and artifact types (requirements, design elements, test cases, regulations). TraceLLM achieves state-of-the-art F2 scores, outperforming traditional IR baselines, fine-tuned models, and prior LLM-based methods. We also explore the impact of demonstration selection strategies, identifying label-aware, diversity-based sampling as particularly effective. Overall, our findings highlight that traceability performance depends not only on model capacity but also critically on the quality of prompt engineering. In addition, the achieved performance suggests that TraceLLM can support semi-automated traceability workflows in which candidate links are reviewed and validated by human analysts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑