人们几乎做了什么：超越行为契合度的LLM社会模拟评估

中文摘要 AI 辅助

基于LLM的社会模拟主要评估行为契合度，即测试智能体是否重现其所模拟人群的行为或响应分布。然而，模拟的承诺远不止于行为契合度。模拟可以解释人类行为、诊断障碍，并比较大规模干预措施。这些用例依赖于理解人们为何以某种方式行事，而不仅仅关注他们做了什么。因此，行为契合度不足以支撑这类论断，因为行为无法充分决定其背后的推理过程。例如，保持沉默的行为可能源于不感兴趣或言论受压制，而不接听电话可能源于对来电者的不信任或手机使用受限。在本文中，我们提出“表征充分性”作为基于LLM的社会模拟的新评估目标。通过利用LLM推理轨迹，表征充分性衡量模拟的情景-推理-行动三元组是否以忠实于所模拟人群和情景的方式保留了行为背后的推理过程。我们将表征充分性与可解释性和对齐指标区分开来，提出将其整合到模拟研究中的方法，并将其测量视为一个开放问题。

英文摘要

LLM-based social simulations are primarily evaluated for behavioral fit, testing whether agents reproduce the actions or response distributions of the people they are simulating. However, the promise of simulation extends beyond behavioral fit. Simulations can explain human behavior, diagnose barriers, and compare large-scale interventions. These use cases depend on understanding \textit{why} people acted a certain way, not just \textit{what} they did. As a result, behavioral fit is insufficient for these types of claims because behavior underdetermines the reasoning process behind it. For instance, the behavior of staying silent may be due to disinterest or suppressed speech, and not answering a call may be due to distrust of the caller or limited phone access. In this paper, we propose \textit{representational adequacy} as a new evaluation target for LLM-based social simulations. By leveraging LLM reasoning traces, representational adequacy measures whether a simulation's scenario--reasoning--action triples preserve the reasoning process behind the behavior in a way that is faithful to the population and scenarios being simulated. We distinguish representational adequacy from interpretability and alignment metrics, propose ways to integrate it into simulation research, and pose its measurement as an open problem.

发表机构

• The Information School, University of Washington（华盛顿大学信息学院）
• MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）

机构由 AI 辅助整理，请以论文原文为准。