StreamBridge：将离线视频大语言模型转变为主动式流式助手
StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistant
AI总结：
StreamBridge 通过记忆缓冲区与轮次衰减压缩、轻量级激活模型及 Stream-IT 数据集，将离线 Video-LLM 转换为具备多轮实时理解和主动响应能力的流式助手，性能超越 GPT-4o 等专有模型。
摘要：
我们提出 StreamBridge，一个简单而有效的框架，可无缝地将离线 Video-LLM 转换为具备流式处理能力的模型。它解决了将现有模型适配到在线场景时的两个根本挑战：（1）多轮实时理解能力有限，以及（2）缺乏主动响应机制。具体来说，StreamBridge 包含（1）一个结合轮次衰减压缩策略的记忆缓冲区，支持长上下文多轮交互，以及（2）一个解耦的轻量级激活模型，可轻松集成到现有 Video-LLM 中，实现持续的主动响应。为进一步支持 StreamBridge，我们构建了 Stream-IT，一个专为流式视频理解定制的大规模数据集，包含交错的视频-文本序列和多样化的指令格式。大量实验表明，StreamBridge 在各种任务上显著提升了离线 Video-LLM 的流式理解能力，甚至优于 GPT-4o 和 Gemini 1.5 Pro 等专有模型。同时，它在标准视频理解基准上取得了有竞争力或更优的性能。
英文摘要：
We present StreamBridge, a simple yet effective framework that seamlessly transforms offline Video-LLMs into streaming-capable models. It addresses two fundamental challenges in adapting existing models into online scenarios: (1) limited capability for multi-turn real-time understanding, and (2) lack of proactive response mechanisms. Specifically, StreamBridge incorporates (1) a memory buffer combined with a round-decayed compression strategy, supporting long-context multi-turn interactions, and (2) a decoupled, lightweight activation model that can be effortlessly integrated into existing Video-LLMs, enabling continuous proactive responses. To further support StreamBridge, we construct Stream-IT, a large-scale dataset tailored for streaming video understanding, featuring interleaved video-text sequences and diverse instruction formats. Extensive experiments show that StreamBridge significantly improves the streaming understanding capabilities of offline Video-LLMs across various tasks, outperforming even proprietary models such as GPT-4o and Gemini 1.5 Pro. Simultaneously, it achieves competitive or superior performance on standard video understanding benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
