发表机构
MWire Labs（MWire实验室）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究针对印度东北部9种低资源语言的代表性不足问题，提出NE-BERT多语言语言模型，通过加权采样等技术提升性能，解决词汇碎片化，发布资源推动当地NLP研究与数字包容。
AI 中文摘要
大型预训练语言模型在多种语言上展现出卓越能力，但资源匮乏的语言代表性严重不足，仍被边缘化。我们提出NE-BERT，这是一种面向领域的多语言编码器模型，在约830万条句子上训练，涵盖9种印度东北部语言和2种锚定语言（印地语、英语），该语言多样的区域在现有多语言模型中代表性极低。通过采用加权数据采样和自定义SentencePiece Unigram分词器，NE-BERT在全部9种印度东北部语言上的表现优于IndicBERT-V2和MuRIL，平均困惑度分别降低15.97倍和7.64倍，分词 fertility比mBERT高1.50倍。我们通过激进的上采样策略解决了Pnar（1002条句子）和Kokborok（2463条句子）等极资源匮乏语言中严重的词汇碎片化问题。对三种印度东北部语言的词性标注下游评估验证了其实际实用性。我们在CC-BY-4.0许可下发布NE-BERT、测试集和训练语料库，以支持印度东北部社区的NLP研究和数字包容。
英文摘要
Large pretrained language models have demonstrated remarkable capabilities across diverse languages, yet critically underrepresented low-resource languages remain marginalized. We present NE-BERT, a domain-specific multilingual encoder model trained on approximately 8.3 million sentences spanning 9 Northeast Indian languages and 2 anchor languages (Hindi, English), a linguistically diverse region with minimal representation in existing multilingual models. By employing weighted data sampling and a custom SentencePiece Unigram tokenizer, NE-BERT outperforms IndicBERT-V2 and MuRIL across all 9 Northeast Indian languages, achieving 15.97X and 7.64X lower average perplexity respectively, with 1.50X better tokenization fertility than mBERT. We address critical vocabulary fragmentation issues in extremely low-resource languages such as Pnar (1,002 sentences) and Kokborok (2,463 sentences) through aggressive upsampling strategies. Downstream evaluation on part-of-speech tagging validates practical utility on three Northeast Indian languages. We release NE-BERT, test sets, and training corpus under CC-BY-4.0 to support NLP research and digital inclusion for Northeast Indian communities.
DOI：10.18653/v1/2026.loreslm-1.1
