arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.19271cs.CLcs.AI
收藏

面向人类的大型语言模型在内容分析中的工作流程

A Human-Centered Workflow for Using Large Language Models in Content Analysis

Ivan Zupic

AI总结:

本文提出了一种面向人类的大型语言模型在内容分析中的工作流程,涵盖注释、摘要和信息提取三个任务,强调通过API利用LLM的潜力,并提供验证方法和最佳实践以解决黑盒、提示敏感和幻觉等问题。

摘要:

尽管许多研究者通过聊天式接口使用大型语言模型(LLMs),其真实潜力在于通过应用程序编程接口(API)利用LLMs。本文将LLMs视为通用文本处理机器,并提出一个全面的工作流程,用于在三种定性和定量内容分析任务中使用LLMs:(1)注释(涵盖定性编码、标注和文本分类),(2)摘要,以及(3)信息提取。该工作流程明确以人类为中心。研究者设计、监督和验证LLM过程的每个阶段,以确保严谨性和透明性。我们的方法综合了多个学科(政治学、社会学、计算机科学、心理学和管理学)的广泛方法论文献的见解。我们概述了验证程序和最佳实践,以解决LLMs的关键限制,如黑盒性质、提示敏感性和倾向于幻觉。为了促进实际实施,我们提供了补充材料,包括提示库和Jupyter Notebook格式的Python代码,并附有详细的使用说明。

英文摘要:

While many researchers use Large Language Models (LLMs) through chat-based access, their real potential lies in leveraging LLMs via application programming interfaces (APIs). This paper conceptualizes LLMs as universal text processing machines and presents a comprehensive workflow for employing LLMs in three qualitative and quantitative content analysis tasks: (1) annotation (an umbrella term for qualitative coding, labeling and text classification), (2) summarization, and (3) information extraction. The workflow is explicitly human-centered. Researchers design, supervise, and validate each stage of the LLM process to ensure rigor and transparency. Our approach synthesizes insights from extensive methodological literature across multiple disciplines: political science, sociology, computer science, psychology, and management. We outline validation procedures and best practices to address key limitations of LLMs, such as their black-box nature, prompt sensitivity, and tendency to hallucinate. To facilitate practical implementation, we provide supplementary materials, including a prompt library and Python code in Jupyter Notebook format, accompanied by detailed usage instructions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑