Vista：面向事后查询的场景感知流视频问答优化
Vista: Scene-Aware Optimization for Streaming Video Question Answering under Post-Hoc Queries
AI总结：
Vista通过场景感知分割、压缩和召回技术，实现了高效且可扩展的流视频问答，提升了长上下文推理能力，同时保持低延迟和内存效率。
摘要：
流式视频问答（流式视频QA）对多模态大语言模型（MLLMs）提出了独特的挑战，因为视频帧是按顺序到达的，而用户查询可以在任意时间点发出。现有的解决方案依赖于固定大小的内存或简单的压缩，往往会导致上下文丢失或内存溢出，限制了它们在长形式、实时场景中的有效性。我们提出了Vista，一种新的面向场景的流式视频QA框架，能够高效且可扩展地处理连续视频流。Vista的创新可以总结为三个方面：（1）场景感知分割，其中Vista动态地将到来的帧划分为时间上和视觉上连贯的场景单元；（2）场景感知压缩，其中每个场景被压缩成紧凑的token表示并存储在GPU内存中，以便高效地基于索引检索，同时高分辨率帧被卸载到CPU内存；（3）场景感知召回，其中相关的场景在接收到查询时被选择性地召回并重新整合到模型输入中，实现了效率和完整性的平衡。Vista是模型无关的，并且能够无缝集成到各种视觉-语言后端中，实现了长上下文推理，而不会牺牲延迟或内存效率。在StreamingBench上的广泛实验表明，Vista实现了最先进的性能，为现实世界的流视频理解建立了强有力的基准。
英文摘要：
Streaming video question answering (Streaming Video QA) poses distinct challenges for multimodal large language models (MLLMs), as video frames arrive sequentially and user queries can be issued at arbitrary time points. Existing solutions relying on fixed-size memory or naive compression often suffer from context loss or memory overflow, limiting their effectiveness in long-form, real-time scenarios. We present Vista, a novel framework for scene-aware streaming video QA that enables efficient and scalable reasoning over continuous video streams. The innovation of Vista can be summarized in three aspects: (1) scene-aware segmentation, where Vista dynamically clusters incoming frames into temporally and visually coherent scene units; (2) scene-aware compression, where each scene is compressed into a compact token representation and stored in GPU memory for efficient index-based retrieval, while full-resolution frames are offloaded to CPU memory; and (3) scene-aware recall, where relevant scenes are selectively recalled and reintegrated into the model input upon receiving a query, enabling both efficiency and completeness. Vista is model-agnostic and integrates seamlessly with a variety of vision-language backbones, enabling long-context reasoning without compromising latency or memory efficiency. Extensive experiments on StreamingBench demonstrate that Vista achieves state-of-the-art performance, establishing a strong baseline for real-world streaming video understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
