Albina Klepach, Alexander Nikulin, Ilya Zisman, Denis Tarasov, Alexander Derevyagin, Andrei Polubarov, Nikita Lyubaykin, Igor Kiselev, Vladislav Kurenkov
AGGC: Adaptive Group Gradient Clipping for Stabilizing Large Language Model Training
AGGC:自适应分组梯度裁剪用于稳定大语言模型训练
Zhiyuan Li, Yuan Wu, Yi Chang
机构
*
School of Artificial intelligence, JiLin University(人工智能学院,吉林大学)
;
Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, JiLin University(知识驱动人机智能工程研究中心,吉林大学)
;
International Center of Future Science, JiLin University(未来科学国际中心,吉林大学)
专题命中
指令微调
:large language model(title,abstract);language model(title,abstract);post-training(abstract);分类 cs.CL、cs.LG
A self-evolving multi-role collaborative framework with fine-grained difficulty guidance for innovative mathematical problem generation
具有细粒度难度指导的自进化多角色协作框架用于创新数学问题生成
Yifei Sun, Yongan Li, A. K. Qin, Sicheng Hou, Tamas Pflanzner
机构
*
School of Physics and Information Technology, Shaanxi Normal University(陕西师范大学物理与信息技术学院)
;
Department of Computing Technologies, Swinburne University of Technology(斯威本科技大学计算技术系)
;
Department of Software Engineering, University of Szeged(塞格德大学软件工程系)
专题命中
指令微调
:large language model(abstract);language model(abstract);SFT(abstract);分类 cs.CL、cs.AI
机构
*
MetaPheno Laboratory(MetaPheno实验室)
;
PhenoTrait Technology Co., Ltd.(PhenoTrait技术有限公司)
;
Wuhan Digital Engineering Institute(武汉数字工程研究院)
;
National Key Laboratory of Multispectral Information Intelligent Processing Technology(国家多光谱信息智能处理技术重点实验室)
;
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology(华中科技大学人工智能与自动化学院)
;
Hubei Key Laboratory of Intelligent Robot(湖北省智能机器人重点实验室)
;
School of Computer Science and Engineering, School of Artificial Intelligence, Wuhan Institute of Technology(武汉理工大学计算机科学与工程学院、人工智能学院)
机构
*
USTC(University of Science and Technology of China)
;
Alibaba Group(阿里巴巴集团)
;
CASIA(Chinese Academy of Sciences Institute of Automation)
;
SIAT(Institute of Automation, Chinese Academy of Sciences)