KinyaColBERT：一种面向低资源检索增强生成的词汇基础检索模型
KinyaColBERT: A Lexically Grounded Retrieval Model for Low-Resource Retrieval-Augmented Generation
AI总结：
本文提出KinyaColBERT检索模型，通过词级后期交互和基于形态学的分词结合两层Transformer编码，解决低资源语言RAG检索准确率低的问题，并在基尼亚卢旺达语农业检索基准上优于强基线和商业API。
摘要：
大型语言模型（LLM）技术近期的主流采用正在许多领域催生聊天机器人和虚拟助手等新型应用。为了将LLM锚定在可信领域并避免幻觉问题，检索增强生成（RAG）已成为一种可行的解决方案。为了在低资源环境中部署可持续的RAG系统，实现高检索准确率不仅是一项可用性要求，也是一种节约成本的策略。通过对基尼亚卢旺达语数据集的实证评估，我们发现实现高检索准确率的最大限制因素是预训练语言模型的语言覆盖范围有限以及子词分词不充分。我们提出了一种新的检索器模型KinyaColBERT，该模型整合了两个关键概念：查询与文档之间的词级后期交互，以及基于形态学的分词与两层Transformer编码相结合。这一方法产生了词汇基础的上下文嵌入，既细粒度又自包含。我们的评估结果表明，在基尼亚卢旺达语农业检索基准上，KinyaColBERT优于强基线模型和领先的商业文本嵌入API。通过采用这种检索策略，我们相信其他低资源环境中的实践者不仅能够实现可靠的RAG系统，还能部署更具成本效益的解决方案。
英文摘要：
The recent mainstream adoption of large language model (LLM) technology is enabling novel applications in the form of chatbots and virtual assistants across many domains. With the aim of grounding LLMs in trusted domains and avoiding the problem of hallucinations, retrieval-augmented generation (RAG) has emerged as a viable solution. In order to deploy sustainable RAG systems in low-resource settings, achieving high retrieval accuracy is not only a usability requirement but also a cost-saving strategy. Through empirical evaluations on a Kinyarwanda-language dataset, we find that the most limiting factors in achieving high retrieval accuracy are limited language coverage and inadequate sub-word tokenization in pre-trained language models. We propose a new retriever model, KinyaColBERT, which integrates two key concepts: late word-level interactions between queries and documents, and a morphology-based tokenization coupled with two-tier transformer encoding. This methodology results in lexically grounded contextual embeddings that are both fine-grained and self-contained. Our evaluation results indicate that KinyaColBERT outperforms strong baselines and leading commercial text embedding APIs on a Kinyarwanda agricultural retrieval benchmark. By adopting this retrieval strategy, we believe that practitioners in other low-resource settings can not only achieve reliable RAG systems but also deploy solutions that are more cost-effective.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
