LoFTI：面向印度本地的定位与事实性迁移
LoFTI: Localization and Factuality Transfer to Indian Locales
AI总结：
针对LLM地理偏见问题，提出LoFTI基准评估模型在印度本地的事实迁移能力，发现包括GPT-4在内的模型在不同本地化层级下结果有偏。
摘要：
大型语言模型（LLMs）通过训练从互联网抓取的大规模网络级数据集，编码了海量的世界知识。然而，这些数据集通常表现出对讲英语的西方国家的地理偏见。这导致LLMs在回答需要针对其他地理区域进行本地化的查询时，产生有偏见或幻觉性的响应。在这项工作中，我们引入了一个名为LoFTI（面向印度本地的定位与事实性迁移）的新基准，可用于评估LLM的本地化和事实性文本迁移能力。LoFTI包含关于源位置和目标位置实体的事实性陈述；源位置分布在全球各地，而目标位置均在印度境内，具有不同程度的超本地性（国家、邦、城市）。这些实体涵盖了广泛的类别。我们使用LoFTI来评估Mixtral、GPT-4以及另外两种基于Mixtral的、非常适合本地化事实迁移任务的方法。我们证明LoFTI是一个高质量的评估基准，并且所有模型（包括GPT-4）在不同程度的超本地性水平下都产生了偏差性结果。
英文摘要：
Large language models (LLMs) encode vast amounts of world knowledge acquired via training on large web-scale datasets crawled from the internet. However, these datasets typically exhibit a geographical bias towards English-speaking Western countries. This results in LLMs producing biased or hallucinated responses to queries that require answers localized to other geographical regions. In this work, we introduce a new benchmark named LoFTI (Localization and Factuality Transfer to Indian Locales) that can be used to evaluate an LLM's localization and factual text transfer capabilities. LoFTI consists of factual statements about entities in source and target locations; the source locations are spread across the globe and the target locations are all within India with varying degrees of hyperlocality (country, states, cities). The entities span a wide variety of categories. We use LoFTI to evaluate Mixtral, GPT-4 and two other Mixtral-based approaches well-suited to the task of localized factual transfer. We demonstrate that LoFTI is a high-quality evaluation benchmark and all the models, including GPT-4, produce skewed results across varying levels of hyperlocality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
