arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2304.07366cs.HC
收藏

CollabCoder:一种利用大语言模型开展归纳式协作质性分析的低门槛、严谨工作流

CollabCoder: A Lower-barrier, Rigorous Workflow for Inductive Collaborative Qualitative Analysis with Large Language Models

Jie Gao, Yuchen Guo, Gionnieve Lim, Tianqin Zhang, Zheng Zhang, Toby Jia-Jun Li, Simon Tangi Perrault

AI总结:

本文提出 CollabCoder 工作流,将 LLMs 融入归纳式协作质性分析的开放编码、讨论与编码簿构建环节,并经 16 名用户评估验证其低门槛、有效性及相对现有软件的优势。

摘要:

协作质性分析(Collaborative Qualitative Analysis, CQA)能够通过纳入多元观点来提升质性分析的严谨性与深度。然而,确保严谨的 CQA 过程本身既费力又昂贵。为降低这一门槛,我们从理论视角出发设计了 CollabCoder 工作流,将大语言模型(Large Language Models, LLMs)整合到归纳式 CQA 的关键阶段:独立开放编码、迭代讨论以及最终编码簿创建。在开放编码阶段,CollabCoder 提供 AI 生成的代码建议,并记录决策数据。在讨论过程中,它通过在编码团队内共享这些数据,并使用量化指标识别编码的一致与分歧,促进相互理解,从而协助达成共识。在代码分组阶段,CollabCoder 提供初步代码分组建议,减轻最终确定编码簿的认知负荷。一项包含 16 名用户的评估证实了 CollabCoder 的有效性,展示了其相较现有软件的优势,并提供了关于 LLMs 在 CQA 实践中作用的经验性洞见。

英文摘要:

Collaborative Qualitative Analysis (CQA) can enhance qualitative analysis rigor and depth by incorporating varied viewpoints. Nevertheless, ensuring a rigorous CQA procedure itself can be both demanding and costly. To lower this bar, we take a theoretical perspective to design the CollabCoder workflow, that integrates Large Language Models (LLMs) into key inductive CQA stages: independent open coding, iterative discussions, and final codebook creation. In the open coding phase, CollabCoder offers AI-generated code suggestions and records decision-making data. During discussions, it promotes mutual understanding by sharing this data within the coding team and using quantitative metrics to identify coding (dis)agreements, aiding in consensus-building. In the code grouping stage, CollabCoder provides primary code group suggestions, lightening the cognitive load of finalizing the codebook. A 16-user evaluation confirmed the effectiveness of CollabCoder, demonstrating its advantages over existing software and providing empirical insights into the role of LLMs in the CQA practice.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑