推进跟踪恢复评估-在软件工程环境中应用的信息检索
Advancing Trace Recovery Evaluation - Applied Information Retrieval in a Software Engineering Context
摘要:
软件系统的成功开发涉及软件工件之间的有效导航。结构化信息的一种实践方法是在工件之间建立跟踪链接,这也是几个开发标准所强制执行的实践。不幸的是,在不断发展的系统中手动维护跟踪链接是一项繁琐的任务。为了解决这个问题,一些研究人员提出了治疗的捕获和恢复跟踪链接的信息检索(IR)的问题。该工作包含对基于IR的痕量回收的先前评价的系统性文献综述(SLR)。我们发现,大多数以前的评估都是以技术为导向的,在“IR评估的洞穴”中进行,使用小数据集作为实验输入。此外,源于学生项目的软件工件经常被用于评估。我们在可追溯性研究人员中进行了一项调查,发现大多数人认为学生的工件只能部分代表工业同行。我们的研究结果需要更多的案例研究,以评估基于IR的痕量恢复在工业环境的全部复杂性。此外,本论文有助于在两个实验与工业软件工件的IR为基础的痕迹恢复的经验证据的身体。以技术为导向的实验突出了数据集之间的明显依赖性和基于IR的痕迹恢复的准确性,与SLR的发现一致。以人为本的实验研究了不同质量水平的工具输出如何影响工程师的跟踪精度。最后,我们提出了如何在IR研究的一般领域的工具和方法进行评估,并提出了一个分类的评估上下文为基础的跟踪恢复。
英文摘要:
Successful development of software systems involves efficient navigation among software artifacts. One state-of-practice approach to structure information is to establish trace links between artifacts, a practice that is also enforced by several development standards. Unfortunately, manually maintaining trace links in an evolving system is a tedious task. To tackle this issue, several researchers have proposed treating the capture and recovery of trace links as an Information Retrieval (IR) problem. The work contains a Systematic Literature Review (SLR) of previous evaluations of IR-based trace recovery. We show that a majority of previous evaluations have been technology-oriented, conducted in "the cave of IR evaluation", using small datasets as experimental input. Also, software artifacts originating from student projects have frequently been used in evaluations. We conducted a survey among traceability researchers, and found that a majority consider student artifacts to be only partly representative to industrial counterparts. Our findings call for additional case studies to evaluate IR-based trace recovery within the full complexity of an industrial setting. Also, this thesis contributes to the body of empirical evidence of IR-based trace recovery in two experiments with industrial software artifacts. The technology-oriented experiment highlights the clear dependence between datasets and the accuracy of IR-based trace recovery, in line with findings from the SLR. The human-oriented experiment investigates how different quality levels of tool output affect the tracing accuracy of engineers. Finally, we present how tools and methods are evaluated in the general field of IR research, and propose a taxonomy of evaluation contexts tailored for IR-based trace recovery.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。