RADIUS：排名、分布与显著性——面向调查模拟的全面对齐套件
RADIUS: Ranking, Distribution, and Significance - A Comprehensive Alignment Suite for Survey Simulation
AI总结：
本文提出RADIUS，一种用于调查模拟的二维对齐套件，通过排名对齐和分布对齐及显著性测试，改进了现有评估指标的不足，提供可重复的开源实现。
摘要：
使用LLMs进行调查模拟正成为生成大规模人类样本人类响应的强大应用。先前工作使用其他领域的指标评估调查模拟，这些指标通常是随意的、碎片化的和非标准化的，导致结果难以比较。此外，现有指标主要关注准确性或分布性措施，忽略了排名对齐这一关键维度。在实践中，模拟可以实现高准确性，但仍可能无法捕捉人类最偏好的选项——这一区别在决策应用中至关重要。我们引入RADIUS，一种面向调查模拟的全面二维对齐套件，能够捕捉：1）排名对齐和2）分布对齐，每种都辅以统计显著性测试。RADIUS突显了现有指标的局限性，使调查模拟的评估更加有意义，并提供开源实现以实现可重复和可比较的评估。
英文摘要：
Simulating survey responses with LLMs offers a scalable alternative to costly human data collection, yet evaluation methods remain ad hoc and inconsistent. Current metrics, typically borrowed from adjacent domains, focus on accuracy or distributional similarity while overlooking ranking alignment-whether simulations preserve the relative preference orderings observed in human populations. A simulator may achieve low distributional divergence yet fail to identify the option humans most prefer, a critical gap for decision-making applications. We propose RADIUS, a two-dimensional alignment suite that evaluates survey simulation along complementary axes: 1) RA alignment and 2) DIstribUtion alignment, accompanied by statistical Significance testing. Experiments across multiple social surveys and persona sources demonstrate that RADIUS exposes limitations of single-metric evaluation and offers stronger discriminative power. On an independently reproduced simulator, we further show that distributional improvement often coincides with misidentifying the option humans most prefer - a decision-relevant error RADIUS surfaces but distributional metrics miss. We open-source RADIUS to support reproducible and comparable assessment of survey simulators.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
