Diffutron：一种针对土耳其语言的掩码扩散语言模型
Diffutron: A Masked Diffusion Language Model for Turkish Language
AI总结：
本文提出Diffutron，一种专为土耳其语言设计的掩码扩散语言模型，通过高效训练流程和分阶段指令微调，实现了紧凑模型在土耳其语言生成任务中的竞争力表现。
摘要：
掩码扩散语言模型（MDLMs）作为一种非自回归的替代方案，已经展现出对标准大型语言模型的竞争力；然而，其在形态丰富的语言中的应用仍显有限。本文我们介绍Diffutron，一种专门针对土耳其语言设计的掩码扩散语言模型。我们的方法利用了资源高效的训练流程，从基于LoRA的连续预训练多语言编码器开始，在大规模语料库上进行预训练。为了实现生成能力，我们采用了一种逐步的指令微调策略，依次在通用和任务特定的指令集上调整模型。在全面的基准测试中，实验结果表明，尽管模型体积紧凑，但其性能仍可与现有多十亿参数基线相媲美。这些发现验证了在土耳其语言中非自回归文本生成中，结合多阶段微调的掩码扩散建模的有效性。
英文摘要：
Masked Diffusion Language Models (MDLMs) have emerged as a compelling non-autoregressive alternative to standard large language models; however, their application to morphologically rich languages remains limited. In this paper, we introduce $\textit{Diffutron}$, a masked diffusion language model specifically designed for Turkish. Our approach leverages a resource-efficient training pipeline, starting with LoRA-based continual pre-training of a multilingual encoder on a large-scale corpus. To enable generative capabilities, we employ a progressive instruction-tuning strategy, sequentially adapting the model on general and task-specific instruction sets. Experimental results across comprehensive benchmarks demonstrate that, despite its compact size, our model achieves competitive performance compared to existing multi-billion-parameter baselines. These findings validate the effectiveness of masked diffusion modeling combined with multi-stage tuning for non-autoregressive text generation in Turkish.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
