面向低资源菲律宾语言零样本跨语言命名实体识别的元预训练
Meta-Pretraining for Zero-Shot Cross-Lingual Named Entity Recognition in Low-Resource Philippine Languages
AI总结：
针对低资源语言NER微调大模型的落地限制，本研究用一阶MAML替换部分自回归目标预训练小型解码器LM，在他加禄语、宿务语测试中提升了零样本跨语言NER效果并缩短收敛时间。
摘要：
低资源语言的命名实体识别（NER）通常通过微调超大多语言语言模型（LM）来解决，这种方法在内存或延迟受限的场景中往往不可行。我们探究小型解码器LM能否通过预训练实现快速适配，并零样本迁移到预训练阶段未见过的语言。为此，我们用一阶模型无关元学习（MAML）替换了部分自回归目标。他加禄语（Tagalog）和宿务语（Cebuano）类型学上相似，但施事/非施事语态系统结构不同，因此可作为具有挑战性的测试平台。在四种模型规模（11M-570M）下，MAML在仅头部微调时将零样本微F1提升2-6个百分点，全量微调后提升1-3个百分点，同时最多将收敛时间缩短8%。对于与他加禄语格助词si/ni共现的单词级人名实体，提升效果最显著，凸显了表层锚点的重要性。
英文摘要：
Named-entity recognition (NER) in low-resource languages is usually tackled by finetuning very large multilingual LMs, an option that is often infeasible in memory- or latency-constrained settings. We ask whether small decoder LMs can be pretrained so that they adapt quickly and transfer zero-shot to languages unseen during pretraining. To this end we replace part of the autoregressive objective with first-order model-agnostic meta-learning (MAML). Tagalog and Cebuano are typologically similar yet structurally different in their actor/non-actor voice systems, and hence serve as a challenging test-bed. Across four model sizes (11 M - 570 M) MAML lifts zero-shot micro-F1 by 2-6 pp under head-only tuning and 1-3 pp after full tuning, while cutting convergence time by up to 8%. Gains are largest for single-token person entities that co-occur with Tagalog case particles si/ni, highlighting the importance of surface anchors.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
