低资源语言下金融交易数据的跨语言命名实体识别
Cross-Lingual NER for Financial Transaction Data in Low-Resource Languages
AI总结：
本研究提出结合知识蒸馏与一致性训练的跨语言NER框架，借助英文预训练XLMRoBERTa，仅用30个阿拉伯文标注样本就实现交易实体识别迁移，效果优于现有最优方法，为低资源多语言金融应用提供方案。
摘要：
我们提出了一种适用于半结构化文本数据跨语言命名实体识别的高效建模框架。该方法同时依托知识蒸馏与一致性训练技术。框架利用在源语言上预训练的大语言模型XLMRoBERTa的知识，采用师生架构实现知识蒸馏，学生模型在低资源目标语言上引入基于KL散度损失的无监督一致性训练。\n 我们使用两个独立的英文和阿拉伯文短信数据集，二者均包含半结构化银行交易信息，重点验证从英文到阿拉伯文的知识迁移效果。仅需30个标注样本，我们的模型就能将商户、金额及其他字段的识别能力从英文泛化到阿拉伯文。实验表明，该高效方法的整体表现优于现有最优方案，包括在目标语言上预训练的DistilBERT，以及直接用目标语言标注数据训练的监督模型。\n 实验证实，只要掌握英文实体识别能力，在少量半结构化标注样本的辅助下，就能在低资源语言上取得合理的识别效果。该框架对多语言应用开发具有参考价值，尤其适用于同时使用英文和一种或多种低资源语言（可与英文混用或单独使用）的数字化应用场景。
英文摘要：
We propose an efficient modeling framework for cross-lingual named entity recognition in semi-structured text data. Our approach relies on both knowledge distillation and consistency training. The modeling framework leverages knowledge from a large language model (XLMRoBERTa) pre-trained on the source language, with a student-teacher relationship (knowledge distillation). The student model incorporates unsupervised consistency training (with KL divergence loss) on the low-resource target language. We employ two independent datasets of SMSs in English and Arabic, each carrying semi-structured banking transaction information, and focus on exhibiting the transfer of knowledge from English to Arabic. With access to only 30 labeled samples, our model can generalize the recognition of merchants, amounts, and other fields from English to Arabic. We show that our modeling approach, while efficient, performs best overall when compared to state-of-the-art approaches like DistilBERT pre-trained on the target language or a supervised model directly trained on labeled data in the target language. Our experiments show that it is enough to learn to recognize entities in English to reach reasonable performance in a low-resource language in the presence of a few labeled samples of semi-structured data. The proposed framework has implications for developing multi-lingual applications, especially in geographies where digital endeavors rely on both English and one or more low-resource language(s), sometimes mixed with English or employed singly.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
