2504.14772
2026-01-06
cs.CL
cs.LG
stat.ML
62%
Knowledge Distillation and Dataset Distillation of Large Language Models: Emerging Trends, Challenges, and Future Directions
大型语言模型的知识蒸馏与数据集蒸馏:新兴趋势、挑战与未来方向
Luyang Fang, Xiaowei Yu, Jiazhang Cai, Yongkai Chen, Shushan Wu, Zhengliang Liu, Zhenyuan Yang, Haoran Lu, Xilin Gong, Yufang Liu, Terry Ma, Wei Ruan, Ali Abbasi, Jing Zhang, Tao Wang, Ehsan Latif, Weihang You, Hanqi Jiang, Wei Liu, Wei Zhang, Soheil Kolouri, Xiaoming Zhai, Dajiang Zhu, Wenxuan Zhong, Tianming Liu, Ping Ma
专题命中
推理评测
:reasoning(abstract);分类 cs.CL、cs.LG
AI总结
本文综述了大型语言模型的知识蒸馏与数据集蒸馏技术,探讨了其在压缩模型、保持推理能力及语言多样性方面的挑战与未来发展方向。