Improving the Downstream Performance of Mixture-of-Experts Transformers via Weak Vanilla Transformers
机构 * Research Center for Social Computing and Interactive Robotics(社会计算与交互机器人研究中心) ; Harbin Institute of Technology(哈尔滨工业大学)
高校专区
机构 * Research Center for Social Computing and Interactive Robotics(社会计算与交互机器人研究中心) ; Harbin Institute of Technology(哈尔滨工业大学)
机构 * School of Computer Science and Technology, Harbin Institute of Technology(计算机科学与技术学院,哈尔滨工业大学)
Comments 15 pages
机构 * Department of Computer Science and Technology, Harbin Institute of Technology(哈尔滨工业大学计算机科学与技术系) ; Harbin Institute of Technology Suzhou Research Institute(哈尔滨工业大学苏州研究所) ; School of AI for Science, the Shenzhen Graduate School, Peking University(北京大学人工智能科学学院、深圳研究生院) ; Peng Cheng Laboratory(鹏城实验室) ; School of Computer Science, Peking University(北京大学计算机学院)
Comments Accepted by IEEE TPAMI