多语言检索增强生成用于文化敏感任务：跨语言鲁棒性基准
Multilingual Retrieval Augmented Generation for Culturally-Sensitive Tasks: A Benchmark for Cross-lingual Robustness
AI总结：
本文提出BordIRLines基准，包含49种语言的领土争端数据集，评估多语言检索增强生成的跨语言鲁棒性，发现多语言检索能提高响应一致性并减少地缘政治偏见。
摘要：
检索增强生成（RAG）范式有助于缓解大型语言模型（LLMs）的幻觉问题。然而，RAG也引入了检索文档中包含的偏见。在多语言和文化敏感的背景下，如领土争端，这些偏见可能被放大。因此，我们引入了BordIRLines，一个包含49种语言、与检索到的维基百科文档配对的领土争端数据集。我们通过形式化多语言检索的几种模式来评估这种RAG设置的跨语言鲁棒性。我们在多个LLM上的实验表明，纳入不同语言的观点实际上可以提高鲁棒性；与纯语言内文档的RAG相比，检索多语言文档最能提高响应一致性并减少地缘政治偏见。我们还考虑了RAG响应如何利用所提供的文档，发现在使用低资源语言查询时，响应引用的语言分布存在更大的方差。我们的进一步分析调查了跨语言RAG管道的各个方面，从检索到文档内容。我们发布了我们的基准和代码，以支持在https://huggingface.co/datasets/borderlines/bordirlines上继续研究跨语言的公平信息获取。
英文摘要：
The paradigm of retrieval-augmented generated (RAG) helps mitigate hallucinations of large language models (LLMs). However, RAG also introduces biases contained within the retrieved documents. These biases can be amplified in scenarios which are multilingual and culturally-sensitive, such as territorial disputes. We thus introduce BordIRLines, a dataset of territorial disputes paired with retrieved Wikipedia documents, across 49 languages. We evaluate the cross-lingual robustness of this RAG setting by formalizing several modes for multilingual retrieval. Our experiments on several LLMs show that incorporating perspectives from diverse languages can in fact improve robustness; retrieving multilingual documents best improves response consistency and decreases geopolitical bias over RAG with purely in-language documents. We also consider how RAG responses utilize presented documents, finding a much wider variance in the linguistic distribution of response citations, when querying in low-resource languages. Our further analyses investigate the various aspects of a cross-lingual RAG pipeline, from retrieval to document contents. We release our benchmark and code to support continued research towards equitable information access across languages at https://huggingface.co/datasets/borderlines/bordirlines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
