LLM辅助的对话数据自动演绎编码：利用对话特定特征增强上下文理解
LLM-Assisted Automated Deductive Coding of Dialogue Data: Leveraging Dialogue-Specific Characteristics to Enhance Contextual Understanding
AI总结：
本研究提出一种LLM辅助的对话数据自动编码框架，利用交际行为与事件特征及多模型协作与一致性检查，显著提升了编码精度并有效应对上下文挑战。
摘要：
对话数据一直是理解学习过程的关键来源，为揭示学生如何参与协作讨论以及这些互动如何塑造其知识建构提供了重要见解。大型语言模型（LLMs）的出现为推进定性研究带来了机遇，尤其是在对话数据的自动编码方面。然而，对话固有的上下文复杂性给这些模型带来了独特挑战，特别是在理解和解释复杂的上下文信息时。本研究通过开发一种新型的LLM辅助对话数据自动编码方法来应对这些挑战。我们提出的框架的新颖性体现在三个方面：1）我们遵循角色提示和思维链方法，使用独立的提示词，基于对话特定特征——即交际行为和交际事件——来预测话语的代码；2）我们让包括GPT-4-turbo、GPT-4o和DeepSeek在内的多个LLM参与协作的代码预测；3）我们利用事件与行为之间的相互关联，使用GPT-4o实现一致性检查。特别是，我们的上下文一致性检查提供了显著的准确率提升。我们还发现，行为预测的准确率始终高于事件预测的准确率。本研究贡献了一个新的方法框架，用于提高对话数据自动编码的精度，并为解决对话分析中固有的上下文挑战提供了可扩展的解决方案。
英文摘要：
Dialogue data has been a key source for understanding learning processes, offering critical insights into how students engage in collaborative discussions and how these interactions shape their knowledge construction. The advent of Large Language Models (LLMs) has introduced promising opportunities for advancing qualitative research, particularly in the automated coding of dialogue data. However, the inherent contextual complexity of dialogue presents unique challenges for these models, especially in understanding and interpreting complex contextual information. This study addresses these challenges by developing a novel LLM-assisted automated coding approach for dialogue data. The novelty of our proposed framework is threefold: 1) We predict the code for an utterance based on dialogue-specific characteristics -- communicative acts and communicative events -- using separate prompts following the role prompts and chain-of-thoughts methods; 2) We engaged multiple LLMs including GPT-4-turbo, GPT-4o, DeepSeek in collaborative code prediction; 3) We leveraged the interrelation between events and acts to implement consistency checking using GPT-4o. In particular, our contextual consistency checking provided a substantial accuracy improvement. We also found the accuracy of act predictions was consistently higher than that of event predictions. This study contributes a new methodological framework for enhancing the precision of automated coding of dialogue data as well as offers a scalable solution for addressing the contextual challenges inherent in dialogue analysis.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
