arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2411.05045cs.CL
收藏

性能引导的大语言模型知识蒸馏用于大规模高效文本分类

Performance-Guided LLM Knowledge Distillation for Efficient Text Classification at Scale

Amazon(亚马逊公司)

Flavio Di Palo, Prateek Singhi, Bilal Fadlallah

AI总结:

提出性能引导的知识蒸馏方法PGKD,通过师生蒸馏与主动学习循环,将大语言模型知识高效迁移至小模型,实现文本分类中高达130倍加速和25倍成本降低,并优于传统基线。

摘要:

大语言模型(LLMs)在推理时因高计算需求面临重大挑战。为解决这一问题,我们提出了性能引导的知识蒸馏(PGKD),一种用于生产环境文本分类应用的高性价比、高吞吐量解决方案。PGKD利用师生知识蒸馏将大语言模型的知识蒸馏到更小、任务特定的模型中。PGKD在学生模型与大语言模型之间建立主动学习循环;大语言模型持续生成新的训练数据,利用难负样本挖掘、学生模型验证性能和早停协议来指导数据生成。通过采用针对工业文本分类中普遍存在的高度多类、稀疏标注数据集量身定制的周期性、性能感知方法,PGKD有效解决了训练挑战,并在多个多类分类数据集上优于传统的BERT-base模型和其他知识蒸馏方法。此外,成本和延迟基准测试显示,使用PGKD微调的模型在同一分类任务的推理速度比大语言模型快高达130倍,成本低25倍。虽然PGKD在文本分类任务中展示,但其通用框架可扩展到任何大语言模型蒸馏任务,包括语言生成,使其成为优化广泛AI应用性能的强大工具。

英文摘要:

Large Language Models (LLMs) face significant challenges at inference time due to their high computational demands. To address this, we present Performance-Guided Knowledge Distillation (PGKD), a cost-effective and high-throughput solution for production text classification applications. PGKD utilizes teacher-student Knowledge Distillation to distill the knowledge of LLMs into smaller, task-specific models. PGKD establishes an active learning routine between the student model and the LLM; the LLM continuously generates new training data leveraging hard-negative mining, student model validation performance, and early-stopping protocols to inform the data generation. By employing a cyclical, performance-aware approach tailored for highly multi-class, sparsely annotated datasets prevalent in industrial text classification, PGKD effectively addresses training challenges and outperforms traditional BERT-base models and other knowledge distillation methods on several multi-class classification datasets. Additionally, cost and latency benchmarking reveals that models fine-tuned with PGKD are up to 130X faster and 25X less expensive than LLMs for inference on the same classification task. While PGKD is showcased for text classification tasks, its versatile framework can be extended to any LLM distillation task, including language generation, making it a powerful tool for optimizing performance across a wide range of AI applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑