arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.11422cs.CL
收藏

通过多阶段知识迁移框架缓解多领域中文拼写校正中的灾难性遗忘

Mitigating Catastrophic Forgetting in Multi-domain Chinese Spelling Correction by Multi-stage Knowledge Transfer Framework

Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);State Key Lab of Software Development Environment, Beihang University(北京航空航天大学软件开发环境国家重点实验室);School of Intelligent Systems Engineering, Sun-Yat Sen University(中山大学智能工程学院);Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)(广东省人工智能与数字经济实验室(深圳))

Peng Xing, Yinghui Li, Shirong Ma, Xinnian Liang, Haojing Huang, Yangning Li, Hai-Tao Zheng, Wenhao Jiang, Ying Shen

AI总结:

针对多领域中文拼写校正中的灾难性遗忘问题,提出模型无关的多阶段知识迁移(MKT)框架,利用持续演进的教师模型进行跨领域知识迁移,首次将持续学习应用于该任务并验证了有效性。

摘要:

中文拼写校正(CSC)旨在检测并纠正给定句子中的拼写错误。近期,多领域CSC因更具实用性而逐渐引起研究者的关注。本文聚焦于CSC模型在适应多领域场景时的关键缺陷:在学习新的领域特定知识时倾向于遗忘先前已获取的知识(即灾难性遗忘)。为解决此问题,我们提出了一种新颖的模型无关的多阶段知识迁移(MKT)框架,该框架在每个领域利用持续演进的教师模型进行知识迁移,而非仅仅关注新的领域知识。值得一提的是,我们是首个将持续学习方法应用于多领域CSC任务的研究。实验证明了所提方法的有效性,进一步分析表明克服灾难性遗忘对提升模型性能的重要性。

英文摘要:

Chinese Spelling Correction (CSC) aims to detect and correct spelling errors in given sentences. Recently, multi-domain CSC has gradually attracted the attention of researchers because it is more practicable. In this paper, we focus on the key flaw of the CSC model when adapting to multi-domain scenarios: the tendency to forget previously acquired knowledge upon learning new domain-specific knowledge (i.e., catastrophic forgetting). To address this, we propose a novel model-agnostic Multi-stage Knowledge Transfer (MKT) framework, which utilizes a continuously evolving teacher model for knowledge transfer in each domain, rather than focusing solely on new domain knowledge. It deserves to be mentioned that we are the first to apply continual learning methods to the multi-domain CSC task. Experiments prove the effectiveness of our proposed method, and further analyses demonstrate the importance of overcoming catastrophic forgetting for improving the model performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑