FLRC:面向高效大语言模型推理的细粒度低秩压缩器
FLRC: Fine-grained Low-Rank Compressor for Efficient LLM Inference
AI总结:
提出FLRC细粒度低秩压缩器,逐层优化秩分配并采用渐进式低秩解码,在摘要任务上ROUGE-L最高提升17%,实现高效且稳健的LLM推理。
摘要:
尽管大语言模型(LLM)已取得显著性能,但其庞大的参数量阻碍了在资源受限硬件上的部署。低秩压缩可同时减少内存占用和计算需求,但跨所有层采用统一压缩比常导致显著性能下降,且先前方法在解码阶段表现不佳。为解决这些问题,我们提出细粒度低秩压缩器(FLRC),它能高效地为每一层确定最优秩分配,并结合渐进式低秩解码以维持文本生成质量。在多个基准上的综合实验证明了FLRC的优越性,在摘要任务上相较于最先进的低秩压缩方法,ROUGE-L指标最高提升17%,从而建立了一个更稳健、更高效的大语言模型推理改进框架。
英文摘要:
Although large language models (LLM) have achieved remarkable performance, their enormous parameter counts hinder deployment on resource-constrained hardware. Low-rank compression can reduce both memory usage and computational demand, but applying a uniform compression ratio across all layers often leads to significant performance degradation, and previous methods perform poorly during decoding. To address these issues, we propose the Fine-grained Low-Rank Compressor (FLRC), which efficiently determines an optimal rank allocation for each layer, and incorporates progressive low-rank decoding to maintain text generation quality. Comprehensive experiments on diverse benchmarks demonstrate the superiority of FLRC, achieving up to a 17% improvement in ROUGE-L on summarization tasks compared to state-of-the-art low-rank compression methods, establishing a more robust and efficient framework to improve LLM inference.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。