CEGI：衡量 SLM 与 VLM 在效率和碳排放之间的权衡
CEGI: Measuring the trade-off between efficiency and carbon emissions for SLMs and VLMs
AI总结：
提出 CEGI 指标，在4项任务上量化 Qwen 与 LLaMA 系列 SLM/VLM 的性能增益、参数量与碳排放权衡，表明微调和低比特量化可兼顾性能与环保。
摘要：
本文分析了小型语言模型（SLM）和视觉语言模型（VLM）的性能，并在4项基本任务上评估模型性能与碳排放之间的权衡：图像描述、视觉问答（VQA）、对话摘要以及文本到SQL转换。研究选取属于 Qwen 和 LLaMA 架构系列的多种 SLM 和 VLM，并根据以参数量表示的模型规模、量化级别和微调参数对不同变体进行评估。研究计算了各模型变体的性能和碳排放。为量化模型性能与碳排放之间的权衡，我们提出了一种名为 CEGI（Carbon Efficient Gain Index，碳效率增益指数）的新指标。该指标表示每百万可训练参数对应的每单位百分比增益所产生的碳排放。这一指标提供了一种归一化度量，用于比较模型在性能提升相对于其环境成本方面的效率。实验结果表明，微调 SLM 和 VLM 可以达到与大型语言模型（LLM）相当的性能水平，同时产生显著更少的碳排放。我们的发现表明，较大模型在准确率上的边际增益并不能证明碳排放的大幅增加是合理的。利用较低比特的量化级别，所提出的指标进一步提高了能源效率，且不会损害性能。本研究强调了在高性能与环境可持续性之间取得平衡，并为选择适合环境友好型 AI 开发的模型提供了有价值的指标。
英文摘要：
This paper analyzes the performance of Small Language Models (SLMs) and Vision Language Models (VLMs) and evaluates the trade-off between model performance and carbon emissions across 4 essential tasks: Image Captioning, Visual Question Answering (VQA), Dialogue Summarization and Text-to-SQL conversion. Various SLMs and VLMs belonging to the Qwen and LLaMA architecture family are chosen and variants based on model size in terms of the number of parameters, quantization level and fine-tuning parameters are evaluated. The model variant's performance and carbon emissions are calculated. To quantify the trade-off between model performance and carbon emissions, we introduce a novel metric called CEGI (Carbon Efficient Gain Index). This metric represents the carbon emission per unit percentage gain per million trainable parameters . This metric provides a normalized measure to compare model's efficiency in terms of performance improvement relative to their environmental cost. The experiment's outcome demonstrates that fine-tuning SLMs and VLMs can achieve performance levels comparable to Large Language Models (LLMs) while producing significantly less carbon emissions. Our findings suggest that the marginal gains in accuracy from larger models do not justify the substantial increase in carbon emissions. Leveraging lower-bit quantization levels, the proposed metric further enhances energy efficiency without compromising performance. This study highlights balancing high performance and environmental sustainability. It offers a valuable metric for selecting models suitable for environmentally-friendly AI development.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
