arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2109.14097cs.SEcs.IRcs.LG
收藏

多少数据分析才足够?机器学习分类的ROI及其在需求相关分类中的应用

How Much Data Analytics is Enough? The ROI of Machine Learning Classification and its Application to Requirements Dependency Classification

University of Calgary(卡尔加里大学);Dept. Computer Science(计算机科学系);Dept. Electrical and Software Engineering(电气与软件工程系);Haskayne School of Business(哈斯凯恩商学院)

Gouri Deshpande, Guenther Ruhe, Chad Saunders

摘要:

机器学习(ML)可以大大提高组织的效率和有效性,并广泛用于软件工程的不同目的。然而,ML技术的选择和实现几乎完全依赖于准确性标准。因此,对于希望实现ML投资收益的组织来说,这种狭隘的方法忽略了ML活动在整个ML生命周期中的预期成本的关键考虑因素,同时未能考虑到可能从拟议活动中获得的收益。我们提出了一种方法,通过提高投资回报率(ROI)考虑的准确性标准来解决这一差距。具体来说,我们分析了两种最先进的ML技术的性能:随机森林和双向编码器表示Transformers(BERT),基于两个公开数据集的准确性和ROI。具体来说,我们比较了需求依赖关系提取的决策(i)完全基于准确性,以及(ii)扩展到包括投资回报率分析。因此,我们提出了根据所使用的训练数据的程度来选择ML分类技术的建议。我们的研究结果表明,与基于准确性作为唯一标准的决策相比,将ROI作为附加标准可以极大地影响ML选择

英文摘要:

Machine Learning (ML) can substantially improve the efficiency and effectiveness of organizations and is widely used for different purposes within Software Engineering. However, the selection and implementation of ML techniques rely almost exclusively on accuracy criteria. Thus, for organizations wishing to realize the benefits of ML investments, this narrow approach ignores crucial considerations around the anticipated costs of the ML activities across the ML lifecycle, while failing to account for the benefits that are likely to accrue from the proposed activity. We present findings for an approach that addresses this gap by enhancing the accuracy criterion with return on investment (ROI) considerations. Specifically, we analyze the performance of the two state-of-the-art ML techniques: Random Forest and Bidirectional Encoder Representations from Transformers (BERT), based on accuracy and ROI for two publicly available data sets. Specifically, we compare decision-making on requirements dependency extraction (i) exclusively based on accuracy and (ii) extended to include ROI analysis. As a result, we propose recommendations for selecting ML classification techniques based on the degree of training data used. Our findings indicate that considering ROI as additional criteria can drastically influence ML selection when compared to decisions based on accuracy as the sole criterion

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑