arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.19342cs.CL
收藏

大语言模型在工业部署中经济上可行吗?

Are Large Language Models Economically Viable for Industry Deployment?

DSEU-Okhla;University of Delhi(德里大学);Macquarie University(麦考瑞大学);Center for SDGC(SDGC研究中心);Stanford University(斯坦福大学)

Abdullah Mohammad, Sushant Kumar Ray, Pushkar Arora, Rafiq Ali, Ebad Shabbir, Gautam Siddharth Kashyap, Jiechao Gao, Usman Naseem

AI总结:

本文提出EDGE-EVAL框架,评估大语言模型在工业场景中的全生命周期性能,引入五个部署指标,揭示模型在经济性和能效上的效率前沿及异常现象。

摘要:

生成AI通过大语言模型(LLMs)在医疗决策支持、金融分析、企业检索和对话自动化等领域得到越来越多的工业应用,其中可靠性、效率和成本控制至关重要。在这些场景中,模型必须满足对能源、延迟和硬件利用的严格约束,而不仅仅是准确性。然而,现有的评估流程仍以准确性为中心,导致部署评估差距——即模型评估中缺乏运营和经济标准。为解决这一差距,我们提出了EDGE-EVAL——一个面向行业的基准评估框架,该框架在遗留的NVIDIA Tesla T4 GPU上对LLM的整个生命周期进行评估。在三个工业任务上评估LLaMA和Qwen的变体,我们引入了五个部署指标——经济平衡点(Nbreak)、每瓦智能(IPW)、系统密度(ρsys)、冷启动税(Ctax)和量化保真度(Qret),捕捉盈利能力、能效、硬件扩展、无服务器可行性以及压缩安全性。我们的结果揭示了一个明显的效率前沿——参数少于2B的模型在经济和生态维度上优于更大的基线。LLaMA-3.2-1B(INT4)在14个请求(中位数)内达到ROI平衡点,其每瓦智能比7B模型高3倍,并在4位量化下达到每GB每秒6,900个标记的速度。我们进一步揭示了一个效率异常——虽然QLoRA减少了内存占用,但对小型模型的适应能量增加了高达7倍,挑战了关于边缘部署中量化感知训练的普遍假设。

英文摘要:

Generative AI-powered by Large Language Models (LLMs)-is increasingly deployed in industry across healthcare decision support, financial analytics, enterprise retrieval, and conversational automation, where reliability, efficiency, and cost control are critical. In such settings, models must satisfy strict constraints on energy, latency, and hardware utilization-not accuracy alone. Yet prevailing evaluation pipelines remain accuracy-centric, creating a Deployment-Evaluation Gap-the absence of operational and economic criteria in model assessment. To address this gap, we present EDGE-EVAL-a industry-oriented benchmarking framework that evaluates LLMs across their full lifecycle on legacy NVIDIA Tesla T4 GPUs. Benchmarking LLaMA and Qwen variants across three industrial tasks, we introduce five deployment metrics-Economic Break-Even (Nbreak), Intelligence-Per-Watt (IPW ), System Density (\r{ho}sys), Cold-Start Tax (Ctax), and Quantization Fidelity (Qret)-capturing profitability, energy efficiency, hardware scaling, serverless feasibility, and compression safety. Our results reveal a clear efficiency frontier-models in the <2B parameter class dominate larger baselines across economic and ecological dimensions. LLaMA-3.2-1B (INT4) achieves ROI break-even in 14 requests (median), delivers 3x higher energy-normalized intelligence than 7B models, and exceeds 6,900 tokens/s/GB under 4-bit quantization. We further uncover an efficiency anomaly-while QLoRA reduces memory footprint, it increases adaptation energy by up to 7x for small models-challenging prevailing assumptions about quantization-aware training in edge deployment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑