AdamHD: Decoupled Huber Decay Regularization for Language Model Pre-Training
机构 * Stanford University(斯坦福大学) ; Harvard Medical School(哈佛医学院)
Comments 39th Conference on Neural Information Processing Systems (NeurIPS 2025) Workshop: GPU-Accelerated and Scalable Optimization (ScaleOpt)
Journal ref 39th Conference on Neural Information Processing Systems (NeurIPS 2025) Workshop: GPU-Accelerated and Scalable Optimization (ScaleOpt)