How Does Sequence Modeling Architecture Influence Base Capabilities of Pre-trained Language Models? Exploring Key Architecture Design Principles to Avoid Base Capabilities Degradation
机构 * Research Center for Social Computing and Interactive Robotics(社会计算与交互机器人研究中心) ; Harbin Institute of Technology(哈尔滨工业大学) ; iFLYTEK Co., Ltd(iFLYTEK公司)
Comments Accepted by NeurIPS 2025