框架税：NLP研究与部署中推理效率的差异
The Framework Tax: Disparities Between Inference Efficiency in NLP Research and Deployment
AI总结：
本研究揭示“框架税”现象，即深度学习框架瓶颈导致NLP模型计算效率提升未转化为实际推理延迟改善，并通过案例研究分析其成因。
摘要：
对自然语言处理系统计算效率的日益关注推动了高效模型架构的设计以及底层硬件加速器的改进。然而，由此带来的计算吞吐量增加和浮点运算减少并未直接转化为墙钟推理延迟的改善。我们证明，这些差异在很大程度上可归因于深度学习框架引入的瓶颈。我们将这一现象称为“框架税”，并观察到随着硬件速度随时间提升，这种差异正在扩大。在本研究中，我们通过一系列案例研究来考察这一现象，分析模型设计决策、框架范式以及硬件平台对总模型延迟的影响。代码可在https://github.com/JaredFern/Framework-Tax获取。
英文摘要：
Increased focus on the computational efficiency of NLP systems has motivated the design of efficient model architectures and improvements to underlying hardware accelerators. However, the resulting increases in computational throughput and reductions in floating point operations have not directly translated to improvements in wall-clock inference latency. We demonstrate that these discrepancies can be largely attributed to bottlenecks introduced by deep learning frameworks. We denote this phenomenon as the \textit{framework tax}, and observe that the disparity is growing as hardware speed increases over time. In this work, we examine this phenomenon through a series of case studies analyzing the effects of model design decisions, framework paradigms, and hardware platforms on total model latency. Code is available at https://github.com/JaredFern/Framework-Tax.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
