CoAIcoder:检验AI辅助的人际协作在定性分析中的有效性
CoAIcoder: Examining the Effectiveness of AI-assisted Human-to-Human Collaboration in Qualitative Analysis
AI总结:
本文针对AI辅助协作式定性分析,设计CoAIcoder并通过32对参与者的被试间实验,揭示共享AI中介提升效率与早期一致性但可能降低代码多样性。
摘要:
尽管AI辅助的个人定性分析已得到大量研究,但AI辅助的协作式定性分析(CQA)——一个涉及多名研究者共同解释数据的过程——仍相对未被探索。在通过形成性访谈识别CQA实践和设计机会后,我们设计并实现了CoAIcoder,这是一款利用AI通过四种不同协作方法增强CQA中人际协作的工具。我们采用被试间设计,在每种协作方法下,让32对接受过CQA训练的参与者跨越常见CQA阶段评估CoAIcoder。研究结果表明,虽然将共享AI模型作为编码者之间的中介可以提高CQA效率,并在早期编码阶段更快促成一致,但它可能影响最终代码的多样性。我们还强调,在各种CQA场景中使用AI辅助人际协作时,需要考虑独立性水平。最后,我们为未来的AI辅助CQA系统提出了设计启示。
英文摘要:
While AI-assisted individual qualitative analysis has been substantially studied, AI-assisted collaborative qualitative analysis (CQA)-a process that involves multiple researchers working together to interpret data-remains relatively unexplored. After identifying CQA practices and design opportunities through formative interviews, we designed and implemented CoAIcoder, a tool leveraging AI to enhance human-to-human collaboration within CQA through four distinct collaboration methods. With a between-subject design, we evaluated CoAIcoder with 32 pairs of CQA-trained participants across common CQA phases under each collaboration method. Our findings suggest that while using a shared AI model as a mediator among coders could improve CQA efficiency and foster agreement more quickly in the early coding stage, it might affect the final code diversity. We also emphasize the need to consider the independence level when using AI to assist human-to-human collaboration in various CQA scenarios. Lastly, we suggest design implications for future AI-assisted CQA systems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。