VERA：通过视觉语言模型的言语化学习实现可解释的视频异常检测
VERA: Explainable Video Anomaly Detection via Verbalized Learning of Vision-Language Models
AI总结：
提出言语化学习框架VERA，通过将复杂推理分解为可学习的指导性问题并优化模型提示，在不修改参数的情况下显著提升了视觉语言模型在视频异常检测中的性能与可解释性。
摘要：
视觉语言模型（VLMs）的快速发展确立了视频异常检测（VAD）的新范式：利用VLMs同时检测异常并为决策提供可理解的解释。该方向的现有工作通常假设VAD所需的复杂推理超出了预训练VLMs的能力。因此，这些方法要么在推理过程中引入专门的推理模块，要么依赖指令微调数据集通过额外训练来使VLMs适应VAD。然而，此类策略往往带来巨大的计算成本或数据标注开销。为了解决可解释VAD中的这些挑战，我们引入了名为VERA的言语化学习框架，使VLMs能够在不修改模型参数的情况下执行VAD。具体而言，VERA将VAD所需的复杂推理自动分解为对更简单、更聚焦的指导性问题的反思，这些问题捕捉了不同的异常模式。它将这些反思性问题视为可学习参数，利用粗略标注的训练数据，通过学习者VLMs和优化器VLMs之间数据驱动的言语交互来优化它们。在推理阶段，VERA将学习到的问题嵌入到模型提示中，引导VLMs生成片段级异常分数，随后通过融合场景上下文和时间上下文将其细化为帧级分数。在具有挑战性的基准测试上的实验结果表明，VERA学习到的问题具有高度适应性，显著提升了VLMs在VAD中的检测性能和可解释性。
英文摘要：
The rapid advancement of vision-language models (VLMs) has established a new paradigm in video anomaly detection (VAD): leveraging VLMs to simultaneously detect anomalies and provide comprehendible explanations for the decisions. Existing work in this direction often assumes the complex reasoning required for VAD exceeds the capabilities of pretrained VLMs. Consequently, these approaches either incorporate specialized reasoning modules during inference or rely on instruction tuning datasets through additional training to adapt VLMs for VAD. However, such strategies often incur substantial computational costs or data annotation overhead. To address these challenges in explainable VAD, we introduce a verbalized learning framework named VERA that enables VLMs to perform VAD without model parameter modifications. Specifically, VERA automatically decomposes the complex reasoning required for VAD into reflections on simpler, more focused guiding questions capturing distinct abnormal patterns. It treats these reflective questions as learnable parameters and optimizes them through data-driven verbal interactions between learner and optimizer VLMs, using coarsely labeled training data. During inference, VERA embeds the learned questions into model prompts to guide VLMs in generating segment-level anomaly scores, which are then refined into frame-level scores via the fusion of scene and temporal contexts. Experimental results on challenging benchmarks demonstrate that the learned questions of VERA are highly adaptable, significantly improving both detection performance and explainability of VLMs for VAD.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
