敏捷工作量估算：我们已经解决了这个问题吗？第二次复制研究的见解（GPT2SP复制报告）
Agile Effort Estimation: Have We Solved the Problem Yet? Insights From A Second Replication Study (GPT2SP Replication Report)
摘要：
Fu和Tantithamthavorn最近提出了GPT 2SP，这是一种基于transformer的深度学习模型，用于用户故事的SP估计。他们在Choetkiertikul等人共享的数据集上经验性地评估了GPT 2SP的性能，包括16个项目，共23，313个问题。他们将GPT 2SP与两个基线（即朴素平均值和中位数估计值）以及Choetkiertikul等人先前提出的方法（从现在起我们将其称为DL 2SP）进行基准测试，用于项目内和跨项目估计场景，并评估GPT 2SP的每个组件对SP估计准确性的贡献程度。他们的结果表明，GPT 2SP优于DL 2SP，在项目内场景中比MAE提高了6%-47%，在跨项目场景中提高了3%-46%。然而，当我们试图使用Fu和Tantithamthavorn提供的GPT 2SP源代码来重现他们的实验时，我们发现了平均绝对误差（MAE）计算中的一个错误，这可能夸大了他们工作中报告的GPT 2SP的准确性。因此，我们发布了一个pull请求来修复这样的bug，该请求已被接受并合并到他们的存储库https://github.com/awsm-research/gpt2sp/pull/2中。在这份报告中，我们描述了我们通过使用GPT 2SP的固定版本来复制原始论文中针对RQ 1和RQ 2进行的实验所获得的结果。在最初的研究之后，我们分析了每个项目中所有问题的估计方法的平均绝对误差（MAE）的结果，但为了完整性，我们还报告了平均绝对误差（MdAE）和标准准确度（SA）。
英文摘要：
Fu and Tantithamthavorn have recently proposed GPT2SP, a Transformer-based deep learning model for SP estimation of user stories. They empirically evaluated the performance of GPT2SP on a dataset shared by Choetkiertikul et al including 16 projects with a total of 23,313 issues. They benchmarked GPT2SP against two baselines (namely the naive Mean and Median estimators) and the method previously proposed by Choetkiertikul et al. (which we will refer to as DL2SP from now on) for both within- and cross-project estimation scenarios, and evaluated the extent to which each components of GPT2SP contribute towards the accuracy of the SP estimates. Their results show that GPT2SP outperforms DL2SP with a 6%-47% improvement over MAE for the within-project scenario and a 3%-46% improvement for the cross-project scenarios. However, when we attempted to use the GPT2SP source code made available by Fu and Tantithamthavorn to reproduce their experiments, we found a bug in the computation of the Mean Absolute Error (MAE), which may have inflated the GPT2SP's accuracy reported in their work. Therefore, we had issued a pull request to fix such a bug, which has been accepted and merged into their repository at https://github.com/awsm-research/gpt2sp/pull/2. In this report, we describe the results we achieved by using the fixed version of GPT2SP to replicate the experiments conducted in the original paper for RQ1 and RQ2. Following the original study, we analyse the results considering the Medan Absolute Error (MAE) of the estimation methods over all issues in each project, but we also report the Median Absolute Error (MdAE) and the Standard accuracy (SA) for completeness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
