斯拉夫语上下文学习的资源与少样本学习器
Resources and Few-shot Learners for In-context Learning in Slavic Languages
AI总结：
本研究构建了捷克语、波兰语和俄语的上下文学习数据集，训练并评估了少样本学习器，发现多语言指令微调及目标语言单任务训练能显著提升模型性能。
摘要：
尽管近期在创建准确且紧凑的上下文学习器方面取得了快速进展，但最近的大多数工作集中在英语任务的上下文学习（ICL）上。然而，与非英语用户交互的能力展现了将语言技术扩展至非英语使用者的巨大潜力。在本研究中，我们收集了部分斯拉夫语（捷克语、波兰语和俄语）ICL训练与评估所需的基础设施。我们链接了多种数据集，并通过一系列转换和纯目标语言编写的新模板，将其转换为统一的指令格式。使用新整理的数据集，我们评估了一组最新的上下文学习器，并将其结果与监督基线进行比较。最后，我们在收集的资源上训练、评估并发布了一组上下文学习模型，比较其性能与先前工作的差异。我们发现，在英语上调优的ICL模型也能从非英语上下文中学习部分任务，但多语言指令微调能持续提升ICL能力。我们还发现，在目标语言中，单任务训练可以超越大规模多任务训练，揭示了将上下文学习器专门应用于目标语言的潜力。
英文摘要：
Despite the rapid recent progress in creating accurate and compact in-context learners, most recent work focuses on in-context learning (ICL) for tasks in English. However, the ability to interact with users of languages outside English presents a great potential for broadening the applicability of language technologies to non-English speakers. In this work, we collect the infrastructure necessary for training and evaluation of ICL in a selection of Slavic languages: Czech, Polish, and Russian. We link a diverse set of datasets and cast these into a unified instructional format through a set of transformations and newly-crafted templates written purely in target languages. Using the newly-curated dataset, we evaluate a set of the most recent in-context learners and compare their results to the supervised baselines. Finally, we train, evaluate and publish a set of in-context learning models that we train on the collected resources and compare their performance to previous work. We find that ICL models tuned in English are also able to learn some tasks from non-English contexts, but multilingual instruction fine-tuning consistently improves the ICL ability. We also find that the massive multitask training can be outperformed by single-task training in the target language, uncovering the potential for specializing in-context learners to the language(s) of their application.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
