CoLoRSMamba：基于条件LoRA的Mamba用于监督多模态暴力检测
AViS-Mamba: Adaptive Visual Steering of Audio State-Space Dynamics for Violence Detection
AI总结：
本文提出CoLoRSMamba，通过CLS引导的条件LoRA将VideoMamba与AudioMamba耦合，实现场景感知的音频动态，提升多模态暴力检测性能。
摘要：
暴力检测受益于音频，但现实中的声音环境可能嘈杂或与可见场景弱相关。我们提出了CoLoRSMamba，一种方向性的视频到音频多模态架构，通过CLS引导的条件LoRA将VideoMamba和AudioMamba耦合。在每一层，VideoMamba的CLS标记生成通道调节向量和稳定门，适应AudioMamba负责选择性状态空间参数（Delta，B，C）的投影，包括步长路径，从而产生场景感知的音频动态，而无需在token层面进行交叉注意力。训练结合二元分类与对称的AV-InfoNCE目标，对片段级音频和视频嵌入进行对齐。为了支持公平的多模态评估，我们从时间注释中整理了NTU-CCTV和DVD数据集的音频过滤片段子集，仅保留有可用音频的片段。在这些子集上，CoLoRSMamba优于代表性的纯音频、纯视频和多模态基线，分别在NTU-CCTV上达到88.63%的准确率/86.24%的F1-V，在DVD上达到75.77%的准确率/72.94%的F1-V。此外，它还提供了有利的准确率-效率权衡，超越了参数和FLOPs较少的几个较大模型。
英文摘要：
Automatic violence detection from video is challenging because violent interactions may be distant, occluded, or only partially visible. Audio can provide complementary evidence for violent events that are difficult to recognize from visual information alone. However, audio itself may be absent, dubbed, or dominated by environmental noise, making the central challenge not whether to incorporate audio but how to adapt reliance on it according to the visual scene. We introduce \emph{AViS-Mamba}, an audiovisual Mamba-based architecture in which the visual stream directly governs the behavior of the audio stream. At each layer of the audio encoder, a compact visual representation produces a modulation vector that conditions the encoder's internal temporal operators together with a routing gate that regulates the strength of this visual intervention. Rather than fusing or reweighting features after they have been extracted, visual context directly shapes the temporal dynamics of the audio encoder. We further propose Adaptive AV-InfoNCE, a contrastive objective that learns to balance the audio-to-video and video-to-audio alignment directions rather than weighting them uniformly. On the audio-valid NTU-CCTV and DVD benchmarks, AViS-Mamba establishes state-of-the-art results, attaining 88.59% and 75.74% accuracy. We demonstrate that adaptive visual conditioning consistently outperforms fixed routing and improves performance under degraded and missing-audio conditions. Layer-wise analysis further reveals that the model adapts the audio stream selectively across network depth rather than applying a single global routing policy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
