What Makes Low-Bit Quantization-Aware Training Work for Reasoning LLMs? A Systematic Study
是什么使低比特量化感知训练在推理大语言模型中有效?一项系统研究
Keyu Lv, Manyi Zhang, Xiaobo Xia, Jingchen Ni, Shannan Yan, Xianzhi Yu, Lu Hou, Chun Yuan, Haoli Bai
机构
*
Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院)
;
Huawei Technologies(华为技术有限公司)
;
National University of Singapore(新加坡国立大学)
机构
*
CFCS, School of Computer Science, Peking University(计算机学院,北京大学)
;
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(多媒体信息处理国家重点实验室,计算机学院,北京大学)
;
LLM-Core Xiaomi(小米LLM-Core)
CI4A: Semantic Component Interfaces for Agents Empowering Web Automation
CI4A: 为网络自动化赋能的语义组件接口
Zhi Qiu, Jiazheng Sun, Chenxiao Xia, Jun Zheng, Xin Peng
机构
*
School of Cyberspace Science and Technology, Beijing Institute of Technology(电子信息学院,北京理工大学)
;
College of Computer Science and Artificial Intelligence, Fudan University(计算机科学与人工智能学院,复旦大学)
Rewarding How Models Think Pedagogically: Integrating Pedagogical Reasoning and Thinking Rewards for LLMs in Education
奖励模型如何思考:在教育中整合教学推理和思考奖励
Unggi Lee, Jiyeong Bae, Jaehyeon Park, Haeun Park, Taejun Park, Younghoon Jeon, Sungmin Cho, Junbo Koh, Yeil Jeong, Gyeonggeon Lee
机构
*
Chosun University(昌原大学)
;
Korea University(韩国大学)
;
Seoul National University(首尔国立大学)
;
Korea Institute for Curriculum and Evaluation(韩国课程与评价研究院)
;
Upstage
;
Indiana University Bloomington(印第安纳大学布卢明顿分校)
;
Nanyang Technological University(南洋理工大学)
机构
*
College of Systems Engineering, National University of Defense Technology(系统工程学院,国防科技大学)
;
State Key Laboratory of Digital Intelligent Modeling and Simulation(数字智能建模与仿真国家重点实验室)
;
BNRist, Tsinghua University(清华大学北京研究院)
;
Department of Electronic Engineering, Tsinghua University(电子工程系,清华大学)
Comments47 pages, 3 figures, 4 tables, preliminary version published in ICML 2024 (Workshop on Theoretical Foundations of Foundation Models) and , see https://openreview.net/pdf?id=WMaFRiggwV