使用生成式AI自动分类导师对话行为：基于CIMA语料库的案例研究
Automated Classification of Tutors' Dialogue Acts Using Generative AI: A Case Study Using the CIMA Corpus
AI总结：
本研究利用生成式AI（GPT-4）自动分类导师对话行为，在CIMA语料库上达到80%准确率，验证了其高效性与应用潜力。
摘要：
本研究探索使用生成式人工智能自动分类导师的对话行为（DAs），旨在减少传统人工编码所需的时间和精力。本案例研究使用开源的CIMA语料库，其中导师的回应被预先标注为四个对话行为类别。研究测试了GPT-3.5-turbo和GPT-4两种模型，均使用定制提示词。结果显示，GPT-4达到了80%的准确率，加权F1分数为0.81，Cohen's Kappa为0.74，超过了基线性能，并与人工标注具有实质性一致性。这些发现表明，生成式AI在提供高效且易用的对话行为分类方法方面具有巨大潜力，对教育对话分析具有深远意义。研究还强调了任务特定标签定义和上下文信息在提高自动标注质量中的重要性。最后，研究强调了使用生成式AI相关的伦理考量，以及负责任和透明研究实践的必要性。本研究的脚本公开于https://github.com/liqunhe27/Generative-AI-for-educational-dialogue-act-tagging。
英文摘要：
This study explores the use of generative AI for automating the classification of tutors' Dialogue Acts (DAs), aiming to reduce the time and effort required by traditional manual coding. This case study uses the open-source CIMA corpus, in which tutors' responses are pre-annotated into four DA categories. Both GPT-3.5-turbo and GPT-4 models were tested using tailored prompts. Results show that GPT-4 achieved 80% accuracy, a weighted F1-score of 0.81, and a Cohen's Kappa of 0.74, surpassing baseline performance and indicating substantial agreement with human annotations. These findings suggest that generative AI has strong potential to provide an efficient and accessible approach to DA classification, with meaningful implications for educational dialogue analysis. The study also highlights the importance of task-specific label definitions and contextual information in enhancing the quality of automated annotation. Finally, it underscores the ethical considerations associated with the use of generative AI and the need for responsible and transparent research practices. The script of this research is publicly available at https://github.com/liqunhe27/Generative-AI-for-educational-dialogue-act-tagging.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
