衡量跨语言迁移范式中的灾难性遗忘：探索调优策略
Measuring Catastrophic Forgetting in Cross-Lingual Transfer Paradigms: Exploring Tuning Strategies
AI总结：
本研究比较大语言模型在跨语言零样本与全样本设置下的 adapter/全参数微调及 IT、CLV 迁移策略，发现 IT 更利于目标语言，CLV 更能保留英语知识。
摘要：
跨语言迁移是一项很有前景的技术，可用于解决资源较少语言中的任务。在这项实证研究中，我们在跨语言场景下比较了两种与大语言模型的零样本和全样本学习方法相结合的微调方法。作为微调策略，我们将参数高效的 adapter 方法与全参数微调进行比较。作为跨语言迁移策略，我们比较了依次使用每种语言的中间训练（IT）和在微调验证阶段就已使用目标语言的跨语言验证（CLV）。我们评估迁移的成功程度，以及跨语言迁移导致源语言出现灾难性遗忘的程度，即当我们以另一种语言学习新信息时，先前获得的知识损失了多少。在两个不同分类问题（仇恨言论检测和产品评论）上的结果均包含多种语言的数据集，结果表明 IT 跨语言策略在目标语言上的表现优于 CLV。我们的研究结果表明，在大多数情况下，当评估多次跨语言迁移中的灾难性遗忘时，CLV 策略相比 IT 策略在基础语言（英语）上表现出更优的知识保留能力。
英文摘要：
The cross-lingual transfer is a promising technique to solve tasks in less-resourced languages. In this empirical study, we compare two fine-tuning approaches combined with zero-shot and full-shot learning approaches for large language models in a cross-lingual setting. As fine-tuning strategies, we compare parameter-efficient adapter methods with fine-tuning of all parameters. As cross-lingual transfer strategies, we compare the intermediate-training (\textit{IT}) that uses each language sequentially and cross-lingual validation (\textit{CLV}) that uses a target language already in the validation phase of fine-tuning. We assess the success of transfer and the extent of catastrophic forgetting in a source language due to cross-lingual transfer, i.e., how much previously acquired knowledge is lost when we learn new information in a different language. The results on two different classification problems, hate speech detection and product reviews, each containing datasets in several languages, show that the \textit{IT} cross-lingual strategy outperforms \textit{CLV} for the target language. Our findings indicate that, in the majority of cases, the \textit{CLV} strategy demonstrates superior retention of knowledge in the base language (English) compared to the \textit{IT} strategy, when evaluating catastrophic forgetting in multiple cross-lingual transfers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
