arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.08294cs.CVcs.AIcs.CL
收藏

视觉语言模型能否评判动作质量?一项实证评估

Can Vision Language Models Judge Action Quality? An Empirical Evaluation

Sword Health;Instituto Superior Técnico, Universidade de Lisboa(里斯本大学高等技术学院);INESC-ID(INESC-ID研究所)

Miguel Monte e Freitas, Rui Henriques, Ricardo Rei, Pedro Henrique Martins

AI总结:

本文评估了视觉语言模型在动作质量评估中的表现,发现现有模型在多个领域仅略高于随机水平,且存在预测偏差,提示细粒度动作质量评估存在根本性困难。

摘要:

动作质量评估(AQA)在物理治疗、体育指导和竞技评判中有广泛应用。尽管视觉语言模型(VLMs)在AQA中具有潜力,但其实际性能尚不明确。本文对最先进的VLMs在活动领域(如健身、花样滑冰、跳水)、任务、表示方法和提示策略上进行了全面评估。基线结果表明,Gemini 3.1 Pro、Qwen3-VL和InternVL3.5模型仅略微超过随机水平,尽管结合骨架信息、 grounding指令、推理结构和上下文学习等策略可带来局部提升,但无一致有效。预测分布分析揭示出两种系统性偏差:倾向于无论视觉证据都预测正确执行,以及对表面语言框架敏感。将任务对比化以缓解这些偏差仅带来最小改进,表明模型的限制不仅限于这些偏差,指向细粒度动作质量评估的根本困难。本文结果为未来基于VLMs的AQA研究建立了严格基准,并提供了需要在可靠现实部署前解决的失败模式路线图。

英文摘要:

Action Quality Assessment (AQA) has broad applications in physical therapy, sports coaching, and competitive judging. Although Vision Language Models (VLMs) hold considerable promise for AQA, their actual performance in this domain remains largely uncharacterised. We present a comprehensive evaluation of state-of-the-art VLMs across activity domains (e.g. fitness, figure skating, diving), tasks, representations, and prompting strategies. Baseline results reveal that Gemini 3.1 Pro, Qwen3-VL and InternVL3.5 models perform only marginally above random chance, and although strategies such as incorporation of skeleton information, grounding instructions, reasoning structures and in-context learning lead to isolated gains, none is consistently effective. Analysis of prediction distributions uncovers two systematic biases: a tendency to predict correct execution regardless of visual evidence, and a sensitivity to superficial linguistic framing. Reformulating tasks contrastively to mitigate these biases yields minimal improvement, suggesting that the models' limitations go beyond these biases, pointing to a fundamental difficulty with fine-grained movement quality assessment. Our findings establish a rigorous baseline for future VLM-based AQA research and provide an actionable outline for failure modes requiring mitigation prior to reliable real-world deployment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑