绿色大语言模型技术实践:现有技术在工业中提升大语言模型应用能效的有效性如何?
Green LLM Techniques in Action: How Effective Are Existing Techniques for Improving the Energy Efficiency of LLM-Based Applications in Industry?
AI总结:
本文评估了四种技术在工业应用中提升大语言模型能效的有效性,发现提示优化和量化可显著降低能耗,但会损害准确性,而小模型与大模型协作在不显著影响其他指标的情况下实现了显著节能。
摘要:
大语言模型(LLM)的快速普及引发了对其大量能耗的担忧,尤其是在工业规模部署时更为明显。尽管已有多种技术被提出以解决这一问题,但关于将其应用于LLM工业应用的有效性仍缺乏实证证据。为填补这一空白,我们分析了荷兰IT服务公司Schuberg Philis的聊天机器人应用。随后,我们选择了四种技术:小模型与大模型协作、提示优化、量化和批处理,并在八个变体中应用这些技术,然后进行了实验以研究它们对能耗、准确性和响应时间的影响,与未优化的基线进行比较。我们的结果表明,某些技术,如提示优化和2位量化,能够显著减少能耗,有时甚至减少90%。然而,这些技术对准确性产生了负面影响,程度在实践中不可接受。唯一在不显著损害其他质量的情况下实现显著且强劲能耗减少的技术是通过Nvidia的Prompt Task和Complexity Classifier(NPCC)的小模型与大模型协作,通过提示复杂度阈值。这表明在实践中减少LLM应用的能耗并不困难。然而,提高其能效,即在不损害其他质量的情况下减少能耗,仍具有挑战性。我们的研究为朝着这一目标迈进提供了实用见解。
英文摘要:
The rapid adoption of large language models (LLMs) has raised concerns about their substantial energy consumption, especially when deployed at industry scale. While several techniques have been proposed to address this, limited empirical evidence exists regarding the effectiveness of applying them to LLM-based industry applications. To fill this gap, we analyzed a chatbot application in an industrial context at Schuberg Philis, a Dutch IT services company. We then selected four techniques, namely Small and Large Model Collaboration, Prompt Optimization, Quantization, and Batching, applied them to the application in eight variations, and then conducted experiments to study their impact on energy consumption, accuracy, and response time compared to the unoptimized baseline. Our results show that several techniques, such as Prompt Optimization and 2-bit Quantization, managed to reduce energy use significantly, sometimes by up to 90%. However, these techniques especially impacted accuracy negatively, to a degree that is not acceptable in practice. The only technique that achieved significant and strong energy reductions without harming the other qualities substantially was Small and Large Model Collaboration via Nvidia's Prompt Task and Complexity Classifier (NPCC) with prompt complexity thresholds. This highlights that reducing the energy consumption of LLM-based applications is not difficult in practice. However, improving their energy efficiency, i.e., reducing energy use without harming other qualities, remains challenging. Our study provides practical insights to move towards this goal.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。