Bilevel ZOFO: Efficient LLM Fine-Tuning and Meta-Training
双层零阶优化:高效的LLM微调与元训练
机构 * Department of Computer Science University of Maryland - College Park(计算机科学系马里兰大学- College Park) ; Department of Computer Science and Engineering University of Texas at Arlington(计算机科学与工程系德克萨斯理工大学阿灵顿分校)
AI总结 Bilevel-ZOFO结合了快速的FO-PEFT内循环和稳定的ZO外循环,实现高效LLM微调与元训练,提升训练速度和泛化能力。