arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.14835stat.MEcs.CYstat.AP
收藏

全球代表性指数:一种总变差距离框架,用于测量调查研究中的人口保真度

The Global Representativeness Index: A Total Variation Distance Framework for Measuring Demographic Fidelity in Survey Research

Evan Hadfield, Andrew Konya

AI总结:

本文提出全球代表性指数(GRI),通过总变差距离框架量化调查样本与目标人群的匹配程度,为样本质量评估提供标准化指标。

摘要:

在全球调查研究日益影响人工智能治理和跨文化交流政策的高风险决策中,却没有一个标准化的指标能够量化样本的人口构成与目标人群的匹配程度。响应率和人口配额——目前样本质量的常用代理指标——衡量的是努力和覆盖范围,而不是分布的保真度。本文介绍了全球代表性指数(GRI),该框架基于总变差距离,在多个人口维度上对调查样本与人口基准进行评分,评分范围为[0, 1]。对全球对话调查七轮数据(N=7500,覆盖60多个国家)的验证发现,细粒度的人口GRI评分仅为0.33-0.36,大约在该样本量下的理论最大值的43%。对世界价值观调查(七轮,N=403,000)、非洲民调第9轮(N=53,000)和拉丁民调(N=19,000)的交叉验证表明,即使是在大规模概率调查中,当国家覆盖有限时,细粒度的全球人口评分仍低于0.22。GRI通过设计效应与经典调查统计学连接;两者都被推荐作为样本质量的最低总结指标,因为GRI对人口距离进行了对称量化,而有效N捕捉了代表性不足的不对称推断成本。该框架作为开源Python库发布,附带联合国和皮尤研究中心的人口基准,适用于调查研究、机器学习数据集审核和AI评估基准。

英文摘要:

Global survey research increasingly informs high-stakes decisions in AI governance and cross-cultural policy, yet no standardized metric quantifies how well a sample's demographic composition matches its target population. Response rates and demographic quotas -- the prevailing proxies for sample quality -- measure effort and coverage but not distributional fidelity. This paper introduces the Global Representativeness Index (GRI), a framework grounded in Total Variation Distance that scores any survey sample against population benchmarks across multiple demographic dimensions on a [0, 1] scale. Validation on seven waves of the Global Dialogues survey (N = 7,500 across 60+ countries) finds fine-grained demographic GRI scores of only 0.33--0.36 -- roughly 43% of the theoretical maximum at that sample size. Cross-validation on the World Values Survey (seven waves, N = 403,000), Afrobarometer Round 9 (N = 53,000), and Latinobarometro (N = 19,000) reveals that even large probability surveys score below 0.22 on fine-grained global demographics when country coverage is limited. The GRI connects to classical survey statistics through the design effect; both metrics are recommended as a minimum summary of sample quality, since GRI quantifies demographic distance symmetrically while effective N captures the asymmetric inferential cost of underrepresentation. The framework is released as an open-source Python library with UN and Pew Research Center population benchmarks, applicable to survey research, machine learning dataset auditing, and AI evaluation benchmarks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑