AlphaDecay: Module-wise Weight Decay for Heavy-Tailed Balancing in LLMs
Di He, Songjun Tu, Ajay Jaiswal, Li Shen, Ganzhao Yuan, Shiwei Liu, Lu Yin
机构
*
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究所)
;
Peng Cheng Laboratory(鹏城实验室)
;
University of Chinese Academy of Sciences(中国科学院大学)
;
University of Texas at Austin(德克萨斯大学奥斯汀分校)
;
Shenzhen Campus of Sun Yat-sen University(中山大学深圳校区)
;
Shenzhen University of Advanced Technology(深圳大学)
;
University of Oxford(牛津大学)
;
University of Surrey(萨里大学)
专题命中
其他LLM
:LLM(abstract);large language model(abstract);language model(abstract);分类 cs.CL、cs.AI、cs.LG
Learning-based Cooperative Robotic Paper Wrapping: A Unified Control Policy with Residual Force Control
Rewida Ali, Cristian C. Beltran-Hernandez, Weiwei Wan, Kensuke Harada
机构
*
Department of Systems Innovation, Graduate School of Engineering Science, Osaka University(大阪大学系统创新部门,工学研究科)
;
OMRON SINIC X Corporation(OMRON SINIC X公司)
;
The National Institute of Advanced Industrial Science and Technology (AIST)(国家先进工业科学与技术研究院)
专题命中
其他LLM
:LLM(abstract);large language model(abstract);language model(abstract);分类 cs.LG
机构
*
The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州))
;
University of Oxford(牛津大学)
;
Xi’an Jiaotong University(西安交通大学)
;
The Hong Kong University of Science and Technology(香港科学与技术大学)
;
City University of Hong Kong(香港城市大学)
;
Beijing University of Technology(北京理工大学)
;
Duke University(杜克大学)
;
X-Humanoid Project(X-Humanoid 项目)
专题命中
其他LLM
:large language model(abstract);language model(abstract)
Cosmin-Constantin Popescu, Maarten Robbert Anton Peters, Oleg Maksimov, Harish Bhandari, Rashi Sharma, Kathleen Richardson, Arka Majumdar, Hyun Jung Kim, Rui Chen, Khoi Phuong Dao, Luigi Ranno, Brian Mills, Dennis Calahan, Tian Gu, Juejun Hu