效率误称
The Efficiency Misnomer
摘要:
模型效率是开发和部署机器学习模型的关键方面。推理时间和延迟直接影响用户体验,有些应用有硬性要求。除了推理成本,模型训练也有直接的财务和环境影响。虽然有许多成熟的度量(成本指标)来衡量模型的效率,研究人员和从业人员往往认为这些指标是相互关联的,并报告只有少数。在本文中,我们深入讨论了常见的成本指标,它们的优点和缺点,以及它们如何相互矛盾。我们演示了如何不完整的成本指标报告可能会导致部分结论和模糊或不完整的图片的实际考虑不同的模型。我们进一步提出建议,以改善效率指标的报告。
英文摘要:
Model efficiency is a critical aspect of developing and deploying machine learning models. Inference time and latency directly affect the user experience, and some applications have hard requirements. In addition to inference costs, model training also have direct financial and environmental impacts. Although there are numerous well-established metrics (cost indicators) for measuring model efficiency, researchers and practitioners often assume that these metrics are correlated with each other and report only few of them. In this paper, we thoroughly discuss common cost indicators, their advantages and disadvantages, and how they can contradict each other. We demonstrate how incomplete reporting of cost indicators can lead to partial conclusions and a blurred or incomplete picture of the practical considerations of different models. We further present suggestions to improve reporting of efficiency metrics.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。