面向非洲语言的探针设计与语境化研究
Designing and Contextualising Probes for African Languages
AI总结：
该研究首次系统探究非洲语言预训练语言模型的语言知识探针，通过分层探针与控制任务验证适配型PLMs的知识编码优势，明确语言特征分布规律，为相关模型优化提供可解释性依据。
摘要：
面向非洲语言的预训练语言模型（PLMs）性能持续提升，但这些进展背后的原因仍不明确。本文首次对非洲语言预训练语言模型的语言知识探针开展系统性研究。我们针对六种类型学上多样化的非洲语言训练分层探针，以分析语言特征的分布情况。我们还为MasakhaPOS数据集设计了控制任务——一种用于解释探针性能的方法。研究发现，适配非洲语言的PLMs比大规模多语言PLMs编码了更多目标语言的语言信息。我们的结果印证了此前的发现：词元级句法信息集中在中后层，而句子级语义信息分布在所有层。通过控制任务和探针基线，我们证实性能反映的是PLMs的内部知识，而非探针的记忆效应。本研究将成熟的可解释性技术应用于非洲语言PLMs，借此揭示了主动学习、多语言适配等策略取得成功的内在机制。
英文摘要：
Pretrained language models (PLMs) for African languages are continually improving, but the reasons behind these advances remain unclear. This paper presents the first systematic investigation into probing PLMs for linguistic knowledge about African languages. We train layer-wise probes for six typologically diverse African languages to analyse how linguistic features are distributed. We also design control tasks, a way to interpret probe performance, for the MasakhaPOS dataset. We find PLMs adapted for African languages to encode more linguistic information about target languages than massively multilingual PLMs. Our results reaffirm previous findings that token-level syntactic information concentrates in middle-to-last layers, while sentence-level semantic information is distributed across all layers. Through control tasks and probing baselines, we confirm that performance reflects the internal knowledge of PLMs rather than probe memorisation. Our study applies established interpretability techniques to African-language PLMs. In doing so, we highlight the internal mechanisms underlying the success of strategies like active learning and multilingual adaptation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
