arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新，休息一下吧，周末愉快~~
arXiv 2404.08763cs.LGcs.CL

CATS：面向大语言模型稀疏性的上下文感知阈值化方法

CATS: Contextually-Aware Thresholding for Sparsity in Large Language Models

University College London（伦敦大学学院）；Oxford University（牛津大学）；Stanford University（斯坦福大学）

Donghyun Lee, Je-Yong Lee, Genghan Zhang, Mo Tiwari, Azalia Mirhoseini

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对大语言模型推理成本高、现有激活稀疏化方法易致性能下降的问题，提出基于新型非线性激活函数的CATS框架，可适配多种基础模型，在高稀疏度下保持性能并实现推理加速。

摘要：

大语言模型（LLMs）已极大推动了AI应用的发展，但由于推理成本极高，其部署仍面临挑战。近期研究通过提升激活稀疏性来降低LLMs的计算成本，但会导致下游任务性能显著下降。本文提出一种用于稀疏化基础LLMs激活、降低推理成本的新框架，名为Contextually Aware Thresholding for Sparsity（CATS，上下文感知稀疏阈值化）。CATS架构相对简单、易于实现且效果显著，其核心是一种新型非线性激活函数。研究表明，CATS可应用于多种基础模型，包括Mistral-7B和Llama2-7B，且在下游任务性能上优于现有稀疏化技术。更具体地说，基于CATS的模型即使在50%的激活稀疏度下，无需任何微调，其下游任务性能通常也能达到基础模型的98%-99%。此外，在进行微调时，基于CATS的模型比同类技术收敛更快，任务表现更优。最后，我们开发了一款定制GPU内核以高效实现CATS，将CATS的激活稀疏性转化为实际 wall-clock 时间加速。这款定制CATS内核实现使Llama-7B和Mistral-7B的token生成 wall-clock 推理延迟降低了约15%。

英文摘要：

Large Language Models (LLMs) have dramatically advanced AI applications, yet their deployment remains challenging due to their immense inference costs. Recent studies ameliorate the computational costs of LLMs by increasing their activation sparsity but suffer from significant performance degradation on downstream tasks. In this work, we introduce a new framework for sparsifying the activations of base LLMs and reducing inference costs, dubbed Contextually Aware Thresholding for Sparsity (CATS). CATS is relatively simple, easy to implement, and highly effective. At the heart of our framework is a new non-linear activation function. We demonstrate that CATS can be applied to various base models, including Mistral-7B and Llama2-7B, and outperforms existing sparsification techniques in downstream task performance. More precisely, CATS-based models often achieve downstream task performance within 1-2% of their base models without any fine-tuning and even at activation sparsity levels of 50%. Furthermore, CATS-based models converge faster and display better task performance than competing techniques when fine-tuning is applied. Finally, we develop a custom GPU kernel for efficient implementation of CATS that translates the activation of sparsity of CATS to real wall-clock time speedups. Our custom kernel implementation of CATS results in a ~15% improvement in wall-clock inference latency of token generation on both Llama-7B and Mistral-7B.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑