arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2208.04548cs.HCcs.CV
收藏

虚拟学习数据集中学生参与度的定义与诠释

Inconsistencies in the Definition and Annotation of Student Engagement in Virtual Learning Datasets: A Critical Review

KITE Research Institute(KITE研究院);University Health Network(大学健康网络)

Shehroz S. Khan, Ali Abedi, Tracey Colella

摘要:

背景:学生在虚拟学习中的参与度(SE)可以对实现学习目标和计划辍学风险产生重大影响。开发用于自动SE测量的人工智能(AI)模型需要注释数据集。然而,现有的SE数据集遭受不一致的定义和注释协议大多不符合教育心理学中的SE定义。这个问题可能会在开发可推广的AI模型时产生误导,并且很难比较在不同数据集上开发的这些模型的性能。本次关键性审查的目的是探索现有的SE数据集,并强调不同的参与定义和注释协议方面的不一致性。方法:检索几个学术数据库,以查找介绍新SE数据集的出版物。包含学生在线或离线基于计算机的虚拟学习会话中的单模态或多模态数据的数据集被包括在内。基于我们定义的参与注释的七个维度:来源、数据形式、时间、时间分辨率、抽象层次、组合和量化,分析了现有数据集中SE的定义和注释。结果:30个SE测量数据集符合纳入标准。审查的SE数据集使用了非常多样化和不一致的定义和注释方案。出乎意料的是,很少有审查的数据集使用现有的心理测量学验证量表在他们的定义SE。讨论内容:不一致的定义和注释的SE是问题的研究开发可比的人工智能模型自动SE测量。一些现有的SE定义和协议的设置,而不是虚拟学习,有可能被用于虚拟学习。

英文摘要:

Background: Student engagement (SE) in virtual learning can have a major impact on meeting learning objectives and program dropout risks. Developing Artificial Intelligence (AI) models for automatic SE measurement requires annotated datasets. However, existing SE datasets suffer from inconsistent definitions and annotation protocols mostly unaligned with the definition of SE in educational psychology. This issue could be misleading in developing generalizable AI models and make it hard to compare the performance of these models developed on different datasets. The objective of this critical review was to explore the existing SE datasets and highlight inconsistencies in terms of differing engagement definitions and annotation protocols. Methods: Several academic databases were searched for publications introducing new SE datasets. The datasets containing students' single- or multi-modal data in online or offline computer-based virtual learning sessions were included. The definition and annotation of SE in the existing datasets were analyzed based on our defined seven dimensions of engagement annotation: sources, data modalities, timing, temporal resolution, level of abstraction, combination, and quantification. Results: Thirty SE measurement datasets met the inclusion criteria. The reviewed SE datasets used very diverse and inconsistent definitions and annotation protocols. Unexpectedly, very few of the reviewed datasets used existing psychometrically validated scales in their definition of SE. Discussion: The inconsistent definition and annotation of SE are problematic for research on developing comparable AI models for automatic SE measurement. Some of the existing SE definitions and protocols in settings other than virtual learning that have the potential to be used in virtual learning are introduced.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑