Every Coin Has Two Sides: On the Dual Nature of Generalization in On-Policy Distillation of Large Language Models
硬币皆有两面:关于大型语言模型的策略内蒸馏中泛化的双重性质
Zhaoyi Li, Deyang Kong, Yuan Wei, Evan Yang, Ranran Shen, Mahardika Krisna Ihsani, Ming Yang, Wei Zhang, Chuan Hao, Jian Yang, Ran Tao, Bryan Dai, Shikun Zhang, Wei Ye, Ying Wei, Defu Lian
机构
*
University of Science and Technology of China(中国科学技术大学)
;
Peking University(北京大学)
;
IQuest Research(IQuest研究院)
;
MBZUAI(穆罕默德·本·扎耶德人工智能大学)
;
Zhejiang University(浙江大学)