arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.04479cs.HC
收藏

大型语言模型如何支持政策研究?评估一个用于大规模非结构化数据的LLM辅助工作流

How can LLMs Support Policy Researchers? Evaluating an LLM-Assisted Workflow for Large-Scale Unstructured Data

Princeton University(普林斯顿大学);Nuremberg Institute for Market Decisions(纽伦堡市场决策研究所)

Yuhan Liu, Shuyao Zhou, Jakob Kaiser, Ella Colby, Jennifer Okwara, Maggie Wang, Varun Nagaraj Rao, Andrés Monroy-Hernández

AI总结:

本文评估了LLM辅助工作流在处理大规模非结构化数据中的应用,探讨了其在政策研究中的可行性和有效性。

摘要:

政策研究者需要可扩展的方法来获取公众观点,但通常依赖于访谈、听证会和调查等分析方法,这些方法效率低、成本高且规模有限。LLM为非结构化文本的主题分析提供了新可能,但关于LLM辅助工作流在政策研究中的表现知之甚少。基于一个用于LLM辅助分析在线论坛的主题工作流,我们与11名政策研究者进行了研究,他们使用早期原型并认为其是快速、粗略的输入。然后我们扩展并扩展了该工作流,以分析数百万Reddit帖子和1,058个由聊天机器人引导的访谈记录,将这些来源视为丰富的政策话语数据。我们比较了合成主题与权威政策报告的主题,识别了一致和分歧点,并讨论了这对采用LLM辅助工作流的政策研究者的意义。

英文摘要:

Policy researchers need scalable ways to surface public views, yet they often rely on interviews, listening sessions, and surveys-analyzed thematically-that are slow, expensive, and limited in scale and diversity. LLMs offer new possibilities for thematic analysis of unstructured text, yet we know little about how LLM-assisted workflows perform for policy research. Building on a workflow for LLM-assisted thematic analysis of online forums, we conduct a study with 11 policy researchers, who use an early prototype and see it as a quick, rough-and-ready input to their research. We then extend and scale the workflow to analyze millions of Reddit posts and 1,058 chatbot-led interview transcripts on a policy-relevant topic, treating these sources as rich and scalable data for policy discourse. We compare the synthesized themes to those from authoritative policy reports, identify points of alignment and divergence, and discuss what this implies for policy researchers adopting LLM-assisted workflows.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑