从学生话语中预测挑战时刻：GPT-4与两种传统自然语言处理方法的比较
Predicting challenge moments from students' discourse: A comparison of GPT-4 to two traditional natural language processing approaches
AI总结：
本研究比较了规则模型、监督机器学习模型和LLM在学生话语中检测挑战及识别维度的表现，发现监督ML和LLM效果更优，并指出LLM虽具优势但存在可靠性等问题，需结合模型透明度以提供有效反馈。
摘要：
有效的协作要求小组进行策略性的自我调节以克服挑战。研究表明，由于成员对挑战的认知存在差异，小组可能无法进行调节，这可能需要外部支持。在本研究中，我们探讨了利用三种不同的自然语言处理模型：专家知识规则模型、监督机器学习（ML）模型和大语言模型（LLM），从学生话语中进行挑战检测和挑战维度识别（认知、元认知、情感和技术/其他挑战）的潜力。结果显示，监督ML和LLM方法在两项任务中均表现相当出色，而规则方法的效果则严重依赖专家设计的特征。本文对这三种方法在协作学习活动中自动检测和支持学生挑战时刻的表现进行了广泛讨论。文章指出，尽管LLM具有诸多优势，但由于其缺乏可靠性，以及有效性评估、隐私和虚构等问题，它们不太可能成为社会共享学习调节的检测和反馈提供问题的万能药。我们在文末讨论了其他考量因素，包括模型透明度，以探索为使用LLM的学生和教育者提供可行且有意义的分析反馈。
英文摘要：
Effective collaboration requires groups to strategically regulate themselves to overcome challenges. Research has shown that groups may fail to regulate due to differences in members' perceptions of challenges which may benefit from external support. In this study, we investigated the potential of leveraging three distinct natural language processing models: an expert knowledge rule-based model, a supervised machine learning (ML) model and a Large Language model (LLM), in challenge detection and challenge dimension identification (cognitive, metacognitive, emotional and technical/other challenges) from student discourse, was investigated. The results show that the supervised ML and the LLM approaches performed considerably well in both tasks, in contrast to the rule-based approach, whose efficacy heavily relies on the engineered features by experts. The paper provides an extensive discussion of the three approaches' performance for automated detection and support of students' challenge moments in collaborative learning activities. It argues that, although LLMs provide many advantages, they are unlikely to be the panacea to issues of the detection and feedback provision of socially shared regulation of learning due to their lack of reliability, as well as issues of validity evaluation, privacy and confabulation. We conclude the paper with a discussion on additional considerations, including model transparency to explore feasible and meaningful analytical feedback for students and educators using LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
