提示学习模型在在线任务中识别协作问题解决技能的应用
Application of Prompt Learning Models in Identifying the Collaborative Problem Solving Skills in an Online Task
AI总结:
针对CPS行为编码依赖人工且现有自动模型需大量数据、准确率低的问题,本文提出基于提示学习的预训练模型,在有限数据下实现高性能,实验证明其在大小训练集上均最优,可替代人工编码。
摘要:
协作问题解决(CPS)能力被认为是21世纪必备技能之一。为了促进CPS能力的评估与学习,研究者提出了一系列框架来概念化CPS,并探索理解协作问题解决中复杂过程的方法。然而,在CPS技能框架内将显性行为编码为子技能仍然是一项具有挑战性的任务。传统研究依赖人工编码来解读CPS的行为数据,但这种编码方法非常耗时,且无法支持实时分析。学者们已开始探索构建自动编码模型的方法。尽管如此,现有基于机器学习或深度学习技术构建的模型依赖于大量训练数据,且准确率相对较低。为解决这些问题,本文提出了一种基于提示学习的预训练模型。该模型即使在训练数据有限的情况下也能实现高性能。在本研究中,进行了三项实验,结果表明,我们的模型不仅在大训练集上产生了最高的准确率、宏F1分数和kappa值,而且在CPS行为数据的小训练集上也表现最佳。所提出的基于提示学习的预训练模型的应用有助于CPS技能编码任务,也可用于其他CSCW编码任务,以替代人工编码。
英文摘要:
Collaborative problem solving (CPS) competence is considered one of the essential 21st-century skills. To facilitate the assessment and learning of CPS competence, researchers have proposed a series of frameworks to conceptualize CPS and explored ways to make sense of the complex processes involved in collaborative problem solving. However, encoding explicit behaviors into subskills within the frameworks of CPS skills is still a challenging task. Traditional studies have relied on manual coding to decipher behavioral data for CPS, but such coding methods can be very time-consuming and cannot support real-time analyses. Scholars have begun to explore approaches for constructing automatic coding models. Nevertheless, the existing models built using machine learning or deep learning techniques depend on a large amount of training data and have relatively low accuracy. To address these problems, this paper proposes a prompt-based learning pre-trained model. The model can achieve high performance even with limited training data. In this study, three experiments were conducted, and the results showed that our model not only produced the highest accuracy, macro F1 score, and kappa values on large training sets, but also performed the best on small training sets of the CPS behavioral data. The application of the proposed prompt-based learning pre-trained model contributes to the CPS skills coding task and can also be used for other CSCW coding tasks to replace manual coding.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。