arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.05375cs.MM
收藏

DAT:双重视觉与带宽感知的自适应传输,用于边缘-云计算系统中高效多模态大语言模型推理

DAT: Dual-Aware Adaptive Transmission with Small-Large Model Collaboration for Edge-Cloud Anomaly Detection

Institute of Computing Technology, Chinese Academy of Science(中国科学院计算技术研究所)

Qi Guo, Zheming Yang, Yunqing Hu, Chang Zhao, Wen Ji

AI总结:

本文提出DAT方法,通过轻量级边缘模型过滤非目标帧并触发MLLM推理,结合高效微调策略和多流自适应传输优化,实现高效多模态大语言模型推理,提升语义生成质量与低延迟警报性能。

摘要:

多模态大语言模型(MLLMs)在语义理解和视觉推理方面表现出色,但其在带宽受限的边缘-云计算系统中处理连续视频流时,会带来显著的计算和通信开销,阻碍低延迟警报和有效视觉证据交付。为解决这一挑战,我们提出了DAT,以实现高质量的语义生成、低延迟事件警报和有效的视觉证据补充。为减少不必要的深度推理成本,我们提出了一种协作的小-大模型级联。一个轻量级的边缘侧小模型作为门控模块,过滤非目标事件帧并执行目标检测,仅对可疑帧触发MLLM推理。在此基础上,我们引入了一种高效的微调策略,结合视觉引导和语义提示,提高了结构化事件理解、目标检测和输出一致性。为确保在带宽限制下实现低延迟语义警报和有效的视觉证据补充,我们进一步设计了一种语义和带宽感知的多流自适应传输优化方法。实验结果表明,DAT实现了98.83%的识别准确率和100%的输出一致性。在严重拥堵情况下,它将加权语义警报延迟减少了高达77.5%,并在0.5秒内交付98.33%的视觉证据,证明了联合优化级联推理和弹性传输的有效性。

英文摘要:

Continuous video anomaly detection is critical for timely safety monitoring in edge--cloud visual systems, but it remains challenging because abnormal events are sparse, transient, and require both low-latency anomaly detection and verifiable visual evidence. Multimodal large language models (MLLMs) offer strong semantic understanding and visual reasoning capabilities for such event-centric analysis, yet applying them to continuous video streams incurs prohibitive computation and communication overhead under bandwidth-constrained edge--cloud deployments. To address this challenge, we propose DAT, a dual-aware adaptive transmission framework that supports high-quality semantic generation, low-latency anomaly detection, and progressive visual evidence supplementation. DAT first uses a collaborative small--large model cascade, where a lightweight edge-side model filters non-target-event frames, performs object detection, and triggers MLLM inference only for suspicious frames. Building on this cascade, DAT introduces visual guidance and semantic prompting to improve structured event understanding, localization, and output consistency. It further schedules semantic anomaly alarms and visual evidence through a semantic- and bandwidth-aware multi-stream transmission optimization method. Experimental results show that DAT achieves 97.77\% F1 in traffic-accident detection while substantially reducing inference and communication overhead. On continuous video workloads, DAT lowers the MLLM call rate to 13.71\% and reduces upload volume by at least 95.72\% compared with representative edge--cloud baselines. Under severe congestion, DAT reduces weighted alarm delay by up to 87.9\%, demonstrating efficient MLLM-based anomaly detection under constrained edge--cloud networks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑