arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.21389cs.CLcs.LG
收藏

任务特定效率分析:当小型语言模型超越大型语言模型

Task-Specific Efficiency Analysis: When Small Language Models Outperform Large Language Models

San Francisco State University - Department of Computer Science(旧金山州立大学-计算机科学系);Carnegie Mellon University - Department of Computer Science(卡内基梅隆大学-计算机科学系);Columbia University - Department of Computer Science(哥伦比亚大学-计算机科学系);Cornell University - Department of Computer Science(康奈尔大学-计算机科学系);Pennsylvania State University - Department of Biochemistry and Molecular Biology(宾夕法尼亚州立大学-生物化学与分子生物学系)

Jinghan Cao, Yu Ma, Xinjin Li, Qingyang Ren, Xiangyun Chen

AI总结:

本文通过对比16个模型在五个NLP任务上的效率,提出PER指标,发现小型模型在效率上表现更优,为高效推理场景提供依据。

摘要:

大型语言模型虽性能优异,但计算成本高,不适合资源受限部署。本文首次系统分析16个模型在五个不同NLP任务上的效率,引入性能-效率比(PER)指标,通过几何平均法整合精度、吞吐量、内存和延迟。系统评估显示,小型模型(0.5-3B参数)在所有任务中均取得更高的PER分数。这些发现为优先考虑推理效率而非微小精度提升的生产环境部署提供了量化基础。

英文摘要:

Large Language Models achieve remarkable performance but incur substantial computational costs unsuitable for resource-constrained deployments. This paper presents the first comprehensive task-specific efficiency analysis comparing 16 language models across five diverse NLP tasks. We introduce the Performance-Efficiency Ratio (PER), a novel metric integrating accuracy, throughput, memory, and latency through geometric mean normalization. Our systematic evaluation reveals that small models (0.5--3B parameters) achieve superior PER scores across all given tasks. These findings establish quantitative foundations for deploying small models in production environments prioritizing inference efficiency over marginal accuracy gains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑