推进波斯语检索增强生成：语言模型开发、综合基准与优化最佳实践
Advancing Retrieval-Augmented Generation for Persian: Development of Language Models, Comprehensive Benchmarks, and Best Practices for Optimization
AI总结：
针对波斯语 RAG 低资源挑战，本文训练 MatinaRoberta 与 MatinaSRoberta，构建多领域基准并评估检索、生成及优化策略。
摘要：
本文考察了在低资源语言中构建检索增强生成（RAG）系统所面临的具体障碍，重点关注波斯语复杂的形态和灵活多变的句法。研究旨在通过引入面向波斯语的模型来提升检索与生成准确性，即 MatinaRoberta（一种掩码语言模型）和 MatinaSRoberta（一种经过微调的 Sentence-BERT），并提出一个综合基准测试框架。这些模型在包含731.1亿个波斯语 token 的多样化语料库上训练后，研究使用三类数据集对其进行评估：通用知识（PQuad）、科学专业文本和组织报告。方法包括大规模预训练、采用定制损失函数的微调，以及使用传统指标和 Retrieval-Augmented Generation Assessment 框架进行系统评估。结果表明，MatinaSRoberta 优于以往的嵌入模型，在各数据集上实现了更出色的上下文相关性和检索准确率。研究还探索了温度调节、分块大小修改和文档摘要索引，以增强 RAG 配置。诸如 Llama-3.1（70B）等较大模型始终表现出最高的生成准确率，而较小模型在处理领域专用和正式语境时面临困难。研究结果强调了通过定制嵌入和检索—生成设置开发波斯语 RAG 系统的潜力，并凸显了其对提升低资源语言中搜索引擎和法律文件分析等 NLP 应用的意义。
英文摘要：
This paper examines the specific obstacles of constructing Retrieval-Augmented Generation(RAG) systems in low-resource languages, with a focus on Persian's complicated morphology and versatile syntax. The research aims to improve retrieval and generation accuracy by introducing Persian-specific models, namely MatinaRoberta(a masked language model) and MatinaSRoberta(a fine-tuned Sentence-BERT), along with a comprehensive benchmarking framework. Three datasets-general knowledge(PQuad), scientifically specialized texts, and organizational reports, were used to assess these models after they were trained on a varied corpus of 73.11 billion Persian tokens. The methodology involved extensive pretraining, fine-tuning with tailored loss functions, and systematic evaluations using both traditional metrics and the Retrieval-Augmented Generation Assessment framework. The results show that MatinaSRoberta outperformed previous embeddings, achieving superior contextual relevance and retrieval accuracy across datasets. Temperature tweaking, chunk size modifications, and document summary indexing were explored to enhance RAG setups. Larger models like Llama-3.1 (70B) consistently demonstrated the highest generation accuracy, while smaller models faced challenges with domain-specific and formal contexts. The findings underscore the potential for developing RAG systems in Persian through customized embeddings and retrieval-generation settings and highlight the enhancement of NLP applications such as search engines and legal document analysis in low-resource languages.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
