预训练语言模型对某些地理人群的表征优于其他人群
Pre-Trained Language Models Represent Some Geographic Populations Better Than Others
AI总结:
本文通过空间探测任务衡量OPT和BLOOM系列预训练语言模型对不同地理人群的表征偏差,发现模型对美英人群表征较好,对南亚和东南亚人群表征较差,且偏差无法由社会语言学、经济或地理因素完全解释,挑战了单一模型适用于所有人群的观点。
摘要:
本文衡量了两类大型语言模型(LLM)在表征不同地理人群方面的偏差程度。我们使用带有地理参考的语料库进行空间探测任务,以衡量来自OPT和BLOOM系列的预训练语言模型在多大程度上表征了世界各地不同的人群。结果表明,这些模型对某些人群的表现远优于其他人群。特别是,美国和英国的人群被表征得相当好,而南亚和东南亚的人群则被表征得较差。分析显示,这两类模型在不同人群上的偏差在很大程度上是相同的。同时,这种偏差不能完全由社会语言学因素、经济因素或地理因素来解释。这项分析的基本结论是,预训练模型并未平等地表征世界人口:存在对特定地理人群的强烈偏差。这一发现挑战了单一模型可以用于所有人群的观点。
英文摘要:
This paper measures the skew in how well two families of LLMs represent diverse geographic populations. A spatial probing task is used with geo-referenced corpora to measure the degree to which pre-trained language models from the OPT and BLOOM series represent diverse populations around the world. Results show that these models perform much better for some populations than others. In particular, populations across the US and the UK are represented quite well while those in South and Southeast Asia are poorly represented. Analysis shows that both families of models largely share the same skew across populations. At the same time, this skew cannot be fully explained by sociolinguistic factors, economic factors, or geographic factors. The basic conclusion from this analysis is that pre-trained models do not equally represent the world's population: there is a strong skew towards specific geographic populations. This finding challenges the idea that a single model can be used for all populations.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。