零样本视觉语言模型用于课堂投入度识别：提示敏感性与跨数据集泛化的基准研究
Zero-Shot Vision-Language Models for Classroom Engagement Recognition: A Benchmark Study of Prompt Sensitivity and Cross-Dataset Generalization
AI总结：
本研究系统评估五种视觉语言模型在零样本条件下识别课堂投入度的表现，发现三个主要失败模式：个体学生识别近乎随机、类别崩溃和极端提示敏感性，但场景级分类效果较好。
摘要：
自动化的课堂投入度识别对于可扩展的学习分析具有巨大潜力，然而现代视觉语言模型（VLM）在零样本条件下执行此任务的适用性仍 largely unexplored。我们提出了一个系统基准，评估五种广泛使用的VLM：CLIP、BLIP-VQA、GPT-4o、LLaVA-1.5-7B和Qwen2.5VL-7B-Instruct，在两个互补的教育数据集上：DAiSEE（个体学生视频数据集，300个采样测试片段）和学生课堂行为数据集（SCB，1168张场景级图像）。每个模型使用三种提示变体进行探测，涵盖最小、基于评分标准和思维链设计。我们的实验揭示了零样本VLM用于投入度识别的三个主要失败模式：（1）个体学生上的近乎随机表现，DAiSEE上Cohen's kappa从未超过0.10；（2）严重的类别崩溃，模型将85-100%的预测分配给单一投入度水平，无论视觉内容如何；（3）极端的提示敏感性，仅因提示措辞不同，相同图像上的准确率波动高达32个百分点。值得注意的是，SCB上的场景级分类要容易得多：当使用基于行为的评分标准提示时，CLIP和GPT-4o的kappa约为0.60。我们还记录了一个部署的实际障碍：GPT-4o的安全过滤器拒绝了98%涉及个体学生面部的思维链请求。我们的发现为在教育观察系统中使用VLM提供了校准基线，并揭示了关键的设计考虑因素。
英文摘要：
Automated classroom engagement recognition holds substantial promise for scalable learning analytics, yet the suitability of modern Vision-Language Models (VLMs) for this task under zero-shot conditions remains largely unexplored. We present a systematic benchmark that evaluates five widely-used VLMs: CLIP, BLIP-VQA, GPT-4o, LLaVA-1.5-7B, and Qwen2.5VL-7B-Instruct across two complementary educational datasets: DAiSEE, an individual-student video dataset (300 sampled test clips), and the Student Classroom Behaviour dataset (SCB, 1,168 scene-level images). Each model is probed with three prompt variants spanning minimal, rubric-anchored, and chain-of-thought designs. Our experiments reveal three primary failure modes of zero-shot VLMs for engagement recognition: (1) near-random performance on individual students, with Cohen's kappa never exceeding 0.10 on DAiSEE; (2) severe class collapse, where models assign 85-100% of predictions to a single engagement level regardless of visual content; and (3) extreme prompt sensitivity, with accuracy swings of up to 32 percentage points on identical images depending solely on prompt phrasing. Remarkably, scene-level classification on SCB is substantially more tractable: CLIP and GPT-4o achieve kappa approximately 0.60 when prompted with behaviorally-grounded rubrics. We also document a practical barrier for deployment: GPT-4o's safety filters reject 98% of chain-of-thought requests involving individual student faces. Our findings provide a calibrated baseline and surface critical design considerations for the use of VLMs in educational observation systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
