视觉大语言模型在监控视频中的评估
Evaluation of Vision-LLMs in Surveillance Video
AI总结：
本文研究视觉语言模型在监控视频中的空间推理能力，提出将异常动作识别作为零样本语言接地任务，评估四个模型在UCF-Crime和RWF-2000上的表现，发现其局限并指出改进方向。
摘要：
摄像头在我们社会中的广泛使用产生了海量的视频数据，远远超出了人类监控的能力。这对公共安全和安保构成了严峻挑战，因为及时检测异常或犯罪事件对于有效响应和预防至关重要。具身智能体识别意外事件的能力从根本上与其空间推理能力相关。本文通过将异常动作识别构建为零样本、语言接地任务，研究视觉语言模型（VLMs）的空间推理，解决从稀疏2D视频解释动态3D场景的具身感知挑战。具体而言，我们研究小型预训练视觉-大语言模型是否能够通过将视频转换为文本描述并通过文本蕴含对标签进行评分，充当空间接地、零样本的异常检测器。我们在UCF-Crime和RWF-2000数据集上评估了四个开放模型，在提示和隐私保护条件下进行测试。少量示例（few-shot exemplars）可以提高某些模型的准确性，但可能增加误报，而隐私过滤器——尤其是全身GAN变换——引入不一致性，从而降低准确性。这些结果描绘了当前视觉-大语言模型在哪些方面成功（简单、空间显著的事件）以及哪些方面失败（嘈杂的空间线索、身份混淆）。展望未来，我们概述了无需任务特定训练即可增强空间接地的具体路径：结构感知提示、跨剪辑的轻量级空间记忆、描述过程中的场景图或3D姿态先验，以及保留动作相关几何的隐私方法。这将零样本、语言接地的流程定位为具身、真实世界视频理解的自适应构建模块。我们用于评估VLM的实现公开可用：https://github.com/pascalbenschopTU/VLLM_AnomalyRecognition
英文摘要：
The widespread use of cameras in our society has created an overwhelming amount of video data, far exceeding the capacity for human monitoring. This presents a critical challenge for public safety and security, as the timely detection of anomalous or criminal events is crucial for effective response and prevention. The ability for an embodied agent to recognize unexpected events is fundamentally tied to its capacity for spatial reasoning. This paper investigates the spatial reasoning of vision-language models (VLMs) by framing anomalous action recognition as a zero-shot, language-grounded task, addressing the embodied perception challenge of interpreting dynamic 3D scenes from sparse 2D video. Specifically, we investigate whether small, pre-trained vision--LLMs can act as spatially-grounded, zero-shot anomaly detectors by converting video into text descriptions and scoring labels via textual entailment. We evaluate four open models on UCF-Crime and RWF-2000 under prompting and privacy-preserving conditions. Few-shot exemplars can improve accuracy for some models, but may increase false positives, and privacy filters -- especially full-body GAN transforms -- introduce inconsistencies that degrade accuracy. These results chart where current vision--LLMs succeed (simple, spatially salient events) and where they falter (noisy spatial cues, identity obfuscation). Looking forward, we outline concrete paths to strengthen spatial grounding without task-specific training: structure-aware prompts, lightweight spatial memory across clips, scene-graph or 3D-pose priors during description, and privacy methods that preserve action-relevant geometry. This positions zero-shot, language-grounded pipelines as adaptable building blocks for embodied, real-world video understanding. Our implementation for evaluating VLMs is publicly available at: https://github.com/pascalbenschopTU/VLLM_AnomalyRecognition
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
