话语分析中的自动演绎编码：大语言模型在学习分析中的应用
Automatic deductive coding in discourse analysis: an application of large language models in learning analytics
AI总结：
针对话语演绎编码耗时耗力的问题，本文比较传统文本分类、类BERT模型与结合提示词工程的GPT，证明后者在少量训练样本下于两个数据集均取得更优准确率和Kappa值。
摘要：
演绎编码是学习科学与学习分析研究者广泛使用的一种常见话语分析方法，用于理解教学与学习互动。它通常要求研究者依据理论指导的编码方案，手动标注所有待分析话语，既耗时又耗费人力。GPT等大语言模型的出现，为克服传统演绎编码的局限、实现自动演绎编码开辟了新途径。为评估大语言模型在自动演绎编码中的效用，我们采用了由不同人工智能技术驱动的三种分类方法，包括结合文本特征工程的传统文本分类方法、类BERT预训练语言模型，以及类GPT预训练大语言模型（LLM）。我们将这些方法应用于两个不同的数据集，并探究了GPT与提示词工程在自动演绎编码中的潜力。通过分析和比较这三种分类方法的准确率和Kappa值，我们发现，在训练样本数量有限的两个数据集上，结合提示词工程的GPT均优于另外两种方法。通过提供详细的提示词结构，所报告的工作展示了如何在自动演绎编码的实施中使用大语言模型。
英文摘要：
Deductive coding is a common discourse analysis method widely used by learning science and learning analytics researchers for understanding teaching and learning interactions. It often requires researchers to manually label all discourses to be analyzed according to a theoretically guided coding scheme, which is time-consuming and labor-intensive. The emergence of large language models such as GPT has opened a new avenue for automatic deductive coding to overcome the limitations of traditional deductive coding. To evaluate the usefulness of large language models in automatic deductive coding, we employed three different classification methods driven by different artificial intelligence technologies, including the traditional text classification method with text feature engineering, BERT-like pretrained language model and GPT-like pretrained large language model (LLM). We applied these methods to two different datasets and explored the potential of GPT and prompt engineering in automatic deductive coding. By analyzing and comparing the accuracy and Kappa values of these three classification methods, we found that GPT with prompt engineering outperformed the other two methods on both datasets with limited number of training samples. By providing detailed prompt structures, the reported work demonstrated how large language models can be used in the implementation of automatic deductive coding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
