TharuChat：通过合成数据和人类验证提升低资源语言的大型语言模型
TharuChat: Bootstrapping Large Language Models for a Low-Resource Language via Synthetic Data and Human Validation
AI总结：
本文提出Tharu-LLaMA(3B)模型，通过合成数据和人类验证构建TharuChat数据集，解决低资源语言在AI中的代表性问题，提升模型性能并证明生成式AI在保护濒危语言中的可行性。
摘要：
大型语言模型的快速普及加剧了数字鸿沟，使全球南方原生语言在AI革命中被排除。Tharu语言作为印欧语系的一种方言，面临严重数据稀缺和语言碎片化问题。本文提出Tharu-LLaMA(3B)模型，通过LLM到人类的 bootstrap 管道构建TharuChat数据集，利用提示工程的Gemini模型生成训练数据，整合Rana Tharu语法和民间故事，反映区域语言的噪声和异质性。通过严谨的消融研究，证明小规模合成数据有效，使数据量从25%提升至100%，使困惑度从6.42降至2.88。该模型证明了生成式AI在保护濒危喜马拉雅语言中的可行性，可在消费级硬件上实现。
英文摘要：
The rapid proliferation of Large Language Models (LLMs) has created a profound digital divide, effectively excluding indigenous languages of the Global South from the AI revolution. The Tharu language, an Indo-Aryan vernacular spoken by approximately 1.7 million people across the Terai belt of Nepal and India, exemplifies this crisis. Despite a rich oral tradition, Tharu suffers from severe data scarcity and linguistic fragmentation, causing state-of-the-art multilingual models to routinely "hallucinate" or default to dominant high-resource neighbors like Hindi and Nepali due to contamination in pre-training corpora. This paper presents Tharu-LLaMA (3B), a specialized instruction-following model designed to address this exclusion. We introduce TharuChat, a novel dataset constructed via a LLM-to-Human bootstrapping pipeline. We utilized prompt-engineered Gemini models, fed with Rana Tharu grammar and folklore, to synthesize training data. Unlike curated gold-standard corpora, TharuChat reflects the noisy, heterogeneous linguistic reality of the region: it is predominantly anchored in Rana Tharu (~70%) while integrating elements of Dangaura and Kochila dialects. We provide a transparent analysis of the dataset's limitations, including dialectal code-mixing and residual Awadhi/Hindi influence. Through a rigorous empirical ablation study, we demonstrate that despite these imperfections, small-scale synthetic data is highly effective, increasing the dataset volume from 25% to 100% results in a linear reduction in perplexity from 6.42 to 2.88. The resulting model serves as a proof-of-concept for the preservation of under-resourced Himalayan languages via generative AI, achievable on consumer-grade hardware.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
