秩序点:面向行动的LLM角色建模用于真实市政模拟
Point of Order: Action-Aware LLM Persona Modeling for Data-Grounded Civic Deliberation
AI总结:
本文提出一种可复现的流程,将公开Zoom录像转换为带有角色档案和行动标签的发言转录,通过细调LLM提升角色建模效果,显著降低困惑度并提高模拟真实性和一致性。
摘要:
大型语言模型提供了多方辩论模拟的机会,但真实建模受限于缺乏带有发言者属性的数据。通过自动语音识别(ASR)生成的转录文件分配匿名发言者标签(如Speaker_1),阻碍了模型对一致人类行为的捕捉。本文介绍了一种可复现的流程,将公共Zoom录像转换为带有角色档案和实用行动标签(如[propose_motion])的发言转录。我们发布了三个地方政府辩论数据集:上诉法院听证会、学校董事会会议和市政委员会会议。通过使用这种“行动感知”数据对LLM进行微调,可将困惑度降低67%,并使基于分类器的发言真实性和现实性指标几乎翻倍。图灵式的人类评估显示,我们的模拟往往无法区分于真实辩论,提供了一种实用且可扩展的复杂真实市政模拟方法。
英文摘要:
LLM-based simulations can enable controlled studies of civic deliberation, but current systems lack speaker-attributed data and methods for evaluating long-form institutional behavior. ASR transcripts typically use anonymous labels such as $Speaker\_1$, preventing models from learning stable participant behavior across meetings. We present a reproducible pipeline that converts public Zoom recordings into speaker-attributed transcripts enriched with persona profiles, topics, and pragmatic "action tags" such as $[propose\_motion]$. Using this pipeline, we release three public datasets of government deliberation (Appellate Court hearings, School Board meetings, and Municipal Council sessions) and fine-tune LLM personas on this action-aware data. We evaluate simulations along four dimensions: persona fidelity, persona consistency, institutional fidelity, and behavioral coherence. Action-aware fine-tuning cuts perplexity by 67%, doubles classifier-based persona fidelity, increases vote attempts by up to $3.6\times$, and improves deliberative responsiveness by up to 70%. Human evaluations show that simulated excerpts are often hard to distinguish from real deliberations, indicating a practical foundation for data-grounded civic simulation studies.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。