CHESS：通过通道级阈值与选择性稀疏化优化大语言模型推理
CHESS: Optimizing LLM Inference via Channel-Wise Thresholding and Selective Sparsification
AI总结：
本文提出CHESS方法，通过通道级阈值和选择性稀疏化优化大语言模型激活稀疏化，在八个下游任务上以更少激活参数实现更低性能下降，并将推理速度提升最高1.27倍。
摘要：
在边缘设备上部署大语言模型（LLMs）面临显著挑战，原因在于其巨大的计算开销和内存需求。激活稀疏化可以通过减少推理过程中被激活的神经元数量来缓解这些资源挑战。现有方法通常基于激活张量的统计信息采用基于阈值的稀疏化。然而，它们没有建模激活稀疏化对性能的影响，导致性能下降并非最优。为了解决这些局限性，本文重新形式化了激活稀疏化问题，以明确刻画激活稀疏性与模型性能之间的关系。随后，本文提出CHESS，一种通过通道级阈值和选择性稀疏化实现的通用激活稀疏化方法。首先，通道级阈值为前馈网络（FFN）层中的每个激活通道分配一个独特的阈值。然后，选择性稀疏化将基于阈值的激活稀疏化应用于注意力模块中的特定层。最后，我们详细介绍了用于加速大语言模型推理的稀疏核实现。实验结果表明，所提出的CHESS在八个下游任务上实现了更低的性能下降，同时比现有方法激活更少的参数，从而将大语言模型推理速度提升最高达1.27倍。
英文摘要：
Deploying large language models (LLMs) on edge devices presents significant challenges due to the substantial computational overhead and memory requirements. Activation sparsification can mitigate these resource challenges by reducing the number of activated neurons during inference. Existing methods typically employ thresholding-based sparsification based on the statistics of activation tensors. However, they do not model the impact of activation sparsification on performance, resulting in suboptimal performance degradation. To address the limitations, this paper reformulates the activation sparsification problem to explicitly capture the relationship between activation sparsity and model performance. Then, this paper proposes CHESS, a general activation sparsification approach via CHannel-wise thrEsholding and Selective Sparsification. First, channel-wise thresholding assigns a unique threshold to each activation channel in the feed-forward network (FFN) layers. Then, selective sparsification involves applying thresholding-based activation sparsification to specific layers within the attention modules. Finally, we detail the implementation of sparse kernels to accelerate LLM inference. Experimental results demonstrate that the proposed CHESS achieves lower performance degradation over eight downstream tasks while activating fewer parameters than existing methods, thus speeding up the LLM inference by up to 1.27x.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
