arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2512.00829cs.CLcs.AI
收藏

通过自动混合精度加速孟加拉语NLP任务:资源高效训练保持模型效能

Accelerating Bangla NLP Tasks with Automatic Mixed Precision: Resource-Efficient Training Preserving Model Efficacy

Department of Computer Science;Engineering Khulna University of Engineering \& Technology Khulna, Bangladesh Email

Md Mehrab Hossain Opi, Sumaiya Khan, Moshammad Farzana Rahman

AI总结:

本文通过自动混合精度技术提升孟加拉语NLP任务的训练效率,减少内存消耗并保持模型性能。

摘要:

训练自然语言处理(NLP)模型需要大量的计算资源和时间,尤其是在孟加拉语NLP开发中,高端硬件的获取往往受限。本文探讨了自动混合精度(AMP)训练作为提高计算效率的方法,不牺牲模型性能。通过动态混合16位和32位浮点计算,AMP降低了GPU内存需求并加快了训练速度,而不会降低模型性能。我们评估了AMP在四个标准孟加拉语NLP任务上的表现:情感分析、命名实体识别、错误分类和问答,使用四个基于Transformer的模型:BanglaBERT、BanglishBERT、XLM-R和mBERT。我们的结果表明,AMP将训练速度提高了44.5%,减少了17.6%的内存消耗,同时保持F-1分数在全精度基线的99.7%以内。这项实证研究突显了AMP在硬件受限环境中普及前沿NLP能力的潜力,通过降低计算障碍。

英文摘要:

Training models for Natural Language Processing (NLP) requires substantial computational resources and time, posing significant challenges, especially for NLP development in Bangla, where access to high-end hardware is often limited. In this work, we explore automatic mixed precision (AMP) training as a means to improve computational efficiency without sacrificing model performance. By leveraging a dynamic mix of 16-bit and 32-bit floating-point computations, AMP lowers GPU memory requirements and speeds up training without degrading model performance. We evaluate AMP across four standard Bangla NLP tasks, namely sentiment analysis, named entity recognition, error classification, and question answering, using four transformer-based models: BanglaBERT, BanglishBERT, XLM-R, and mBERT. Our results demonstrate that AMP accelerates training by 44.5% and reduces memory consumption by 17.6%, while maintaining F-1 score within 99.7% of the full-precision baselines. This empirical study highlights AMP's potential to democratize access to state-of-the-art NLP capabilities in hardware-constrained settings by lowering computational barriers.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑