arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.02523cs.CV
收藏

一种基于时空注意力的学生课堂行为检测方法

A Spatio-Temporal Attention-Based Method for Detecting Student Classroom Behaviors

Jinan University(暨南大学);Chengdu University(成都大学);Chengdu Neusoft University(成都东软学院)

Fan Yang

AI总结:

针对学生课堂行为检测准确率低及数据长尾问题,提出融合 SlowFast 与时空注意力模块、采用改进 focal loss 的 BDSTA 方法,在 STSCB 数据集上较 SlowFast 平均准确率提升 8.94%。

摘要:

从课堂视频中准确检测学生行为,有助于分析学生的课堂状态并提高教学效率。然而,学生课堂行为检测准确率较低是一个普遍存在的问题。为解决该问题,我们提出了一种基于时空注意力的学生课堂行为检测方法(BDSTA)。首先,使用 SlowFast 网络从视频中生成运动信息和环境信息特征图。然后,将时空注意力模块应用于这些特征图,该模块包括信息聚合、压缩和激励过程。随后,获得时间、通道和空间维度上的注意力图,并基于这些注意力图执行多标签行为分类。为解决学生课堂行为数据集中存在的长尾数据问题,我们采用改进的 focal loss 函数,在训练过程中为尾类数据分配更大的权重。实验在名为 STSCB 的自制学生课堂行为数据集上进行。与 SlowFast 模型相比,使用 BDSTA 后,学生行为分类检测的平均准确率提高了 8.94%。

英文摘要:

Accurately detecting student behavior from classroom videos is beneficial for analyzing their classroom status and improving teaching efficiency. However, low accuracy in student classroom behavior detection is a prevalent issue. To address this issue, we propose a Spatio-Temporal Attention-Based Method for Detecting Student Classroom Behaviors (BDSTA). Firstly, the SlowFast network is used to generate motion and environmental information feature maps from the video. Then, the spatio-temporal attention module is applied to the feature maps, including information aggregation, compression and stimulation processes. Subsequently, attention maps in the time, channel and space dimensions are obtained, and multi-label behavior classification is performed based on these attention maps. To solve the long-tail data problem that exists in student classroom behavior datasets, we use an improved focal loss function to assign more weight to the tail class data during training. Experimental results are conducted on a self-made student classroom behavior dataset named STSCB. Compared with the SlowFast model, the average accuracy of student behavior classification detection improves by 8.94\% using BDSTA.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑