VLMs在视觉缺陷检测中能走多远？研究41小时游戏录像中的19,738个关键帧
How Far Can VLMs Go for Visual Bug Detection? Studying 19,738 Keyframes from 41 Hours of Gameplay Videos
AI总结：
研究通过分析41小时游戏录像中的19,738个关键帧，评估VLMs在实际场景中检测视觉缺陷的能力，发现其性能有限，需结合混合方法提升。
摘要：
基于视频的质量保证（QA）在长形式游戏视频中是劳动密集型且易出错的，但对评估游戏稳定性和长时间游戏中的视觉正确性具有价值。视觉语言模型（VLMs）承诺具备通用的视觉推理能力，因此在直接从视频帧中检测视觉错误方面显得有吸引力。最近的基准测试表明，VLMs在受控数据集上检测视觉故障方面可以取得有希望的结果。基于这些发现，我们利用工业QA游戏视频进行真实世界研究，以评估VLMs在实际场景中的表现。我们的研究从长游戏视频中采样关键帧，并让VLM判断每个关键帧是否包含错误。从单一提示基线开始，模型的精度为0.50，准确率为0.72。然后我们检查了两种常见的增强策略，用于在不微调的情况下提高VLM性能：（1）一个二次判断模型重新评估VLM的输出，以及（2）通过检索先前的bug报告进行元数据增强的提示。在跨100个视频，总计41小时和19,738个关键帧的情况下，这些策略在简单的基线之上只提供了微小的改进，同时引入了额外的计算成本和输出变化。我们的发现表明，现成的VLMs已经能够检测QA游戏视频中一定范围的视觉错误，但进一步的进步可能需要混合方法，以更好地分离文本和视觉异常检测。
英文摘要：
Video-based quality assurance (QA) for long-form gameplay video is labor-intensive and error-prone, yet valuable for assessing game stability and visual correctness over extended play sessions. Vision language models (VLMs) promise general-purpose visual reasoning capabilities and thus appear attractive for detecting visual bugs directly from video frames. Recent benchmarks suggest that VLMs can achieve promising results in detecting visual glitches on curated datasets. Building on these findings, we conduct a real-world study using industrial QA gameplay videos to evaluate how well VLMs perform in practical scenarios. Our study samples keyframes from long gameplay videos and asks a VLM whether each keyframe contains a bug. Starting from a single-prompt baseline, the model achieves a precision of 0.50 and an accuracy of 0.72. We then examine two common enhancement strategies used to improve VLM performance without fine-tuning: (1) a secondary judge model that re-evaluates VLM outputs, and (2) metadata-augmented prompting through the retrieval of prior bug reports. Across \textbf{100 videos} totaling \textbf{41 hours} and \textbf{19,738 keyframes}, these strategies provide only marginal improvements over the simple baseline, while introducing additional computational cost and output variance. Our findings indicate that off-the-shelf VLMs are already capable of detecting a certain range of visual bugs in QA gameplay videos, but further progress likely requires hybrid approaches that better separate textual and visual anomaly detection.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
