发表机构
Goethe University Frankfurt; The Hessian Center for Artificial Intelligence（法兰克福大学; 黑森人工智能中心）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究针对在线学习中自动参与度预测的多维构念与标注主观性等挑战，提出整合多模态特征与不确定性感知预测的Perceiver IO框架，在CASED数据集上取得具竞争力性能且提供校准良好的不确定性指标。
AI 中文摘要
从在线辅导视频中预测学生参与度颇具难度，因为参与度是包含不同行为、情绪和认知状态的多维构念，可靠的预测需融合不同类型的行为信号与表达线索。通过对CASED数据集的分析可发现，由于高人际变异性及参与度标注的主观性，参与度预测难度进一步提升。为应对这些挑战，我们开发了多模态框架，该框架整合了从预训练视频、音频和图像编码器中提取的隐式时空特征，以及结构化行为模态（如头部姿态、注视、面部动作单元、情绪和基于小波的音频特征）；我们通过Perceiver IO潜在瓶颈整合这些模态，还将学生与教师的人格建模为可学习嵌入上的变分后验，以实现跨参与者的部分池化。我们采用证据回归和谱归一化高斯过程分类头进行不确定性感知预测，以进一步提升鲁棒性与校准度。在CASED挑战测试集上的基准结果显示，所有参与方法均收敛至接近随机水平的性能，凸显该数据集的难度；在这种高度模糊的 regime 中，我们的框架取得了具有竞争力的性能，同时还提供了校准良好的不确定性指标，表明可靠的风险量化是将参与度模型部署到真实教育工具中的必要前提。
英文摘要
The prediction of student engagement from the online tutoring videos is difficult because engagement is a multidimensional construct comprising distinct behavioral, emotional, and cognitive states. A reliable prediction requires bringing together different types of behavioral signals as well as expressive cues. Through our analysis of the CASED dataset, it is clear that engagement prediction gets even harder due to the high inter-person variability as well as the subjectivity of the engagement annotation. To tackle these challenges, we develop a multimodal framework that integrates the implicit spatiotemporal features extracted from pretrained video, audio, and image encoders along with structured behavioral modalities like head pose, gaze, facial action units, emotion, and wavelet-based audio features. We integrate these modalities via a Perceiver IO latent bottleneck. Moreover, student and instructor personalities are modeled as variational posteriors over learnable embeddings to enable partial pooling across participants. We employ evidential regression and spectral-normalized Gaussian process classification heads for uncertainty-aware prediction to further improve robustness and calibration. Benchmark on the CASED challenge test set shows that all participating methods converge near random-chance performance, revealing the difficulty of the dataset. In this highly ambiguous regime, our framework achieves competitive performance while uniquely offering well-calibrated uncertainty metrics, demonstrating that reliable risk-quantification is an essential prerequisite for deploying engagement models in real-world educational tools.
CommentsAccepted to ICMI 2026 (International Conference on Multimodal Interaction), October 5-9, 2026, Napoli, Italy. 5 pages, 1 figures
