arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.00745cs.CL
收藏

FastPOS:一种语言无关的可扩展词性标注框架(低资源语言用例)

FastPOS: Language-Agnostic Scalable POS Tagging Framework Low-Resource Use Case

Daffodil International University(达福尔国际大学);Alliance University(联盟大学)

Md Abdullah Al Kafi, Sumit Kumar Banshal

AI总结:

FastPOS提出了一种基于变换器的词性标注框架,通过少量代码实现多语言适应,显著提升低资源语言的词性标注性能。

摘要:

本研究提出了一种基于变换器的、语言无关的词性标注框架,专门针对低资源语言,以孟加拉语和印地语为例进行研究。仅需三行框架特定代码,模型即可从孟加拉语适应到印地语,展示了在最小修改的情况下有效的可移植性。该框架在孟加拉语和印地语的词性标注类别上分别实现了96.85%和97%的令牌级准确率,同时在数据集不平衡和语言重叠的情况下仍能保持强劲的F1分数。在特定词性标注类别上存在性能差异,凸显出数据集编纂仍面临持续挑战。强劲的结果源于底层的变换器架构,该架构可以通过有限的代码调整进行替换。其模块化和开源设计使跨语言适应变得迅速,同时减少了模型设计和调优的开销,使研究人员能够专注于语言预处理和数据集优化,这对于推动低资源语言的自然语言处理发展至关重要。

英文摘要:

This study proposes a language-agnostic transformer-based POS tagging framework designed for low-resource languages, using Bangla and Hindi as case studies. With only three lines of framework-specific code, the model was adapted from Bangla to Hindi, demonstrating effective portability with minimal modification. The framework achieves 96.85 percent and 97 percent token-level accuracy across POS categories in Bangla and Hindi while sustaining strong F1 scores despite dataset imbalance and linguistic overlap. A performance discrepancy in a specific POS category underscores ongoing challenges in dataset curation. The strong results stem from the underlying transformer architecture, which can be replaced with limited code adjustments. Its modular and open-source design enables rapid cross-lingual adaptation while reducing model design and tuning overhead, allowing researchers to focus on linguistic preprocessing and dataset refinement, which are essential for advancing NLP in underrepresented languages.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑