arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.08620cs.CV
收藏

StreamReady: 在长视频流中学习何时回答

StreamReady: Learning What to Answer and When in Long Streaming Videos

Center for Research in Computer Vision, University of Central Florida(计算机视觉研究中心,中央佛罗里达大学);Microsoft Research(微软研究院)

Shehreen Azad, Vibhav Vineet, Yogesh Singh Rawat

AI总结:

StreamReady通过引入答案准备度评分,统一时间推理与及时回答,实现对长视频流中何时回答的高效学习与准确判断。

摘要:

视频流理解通常涉及时间敏感的场景,其中模型需要在支持性视觉证据出现时恰好回答:在证据出现前回答是猜测,在证据过去后回答会降低实时效用。为了捕捉这种行为,我们引入了一种具有答案准备度评分(ARS)的视频流理解准备意识公式,这是一种具有时间意识的目标,具有不对称的早期和晚期惩罚。当与正确性结合时,ARS定义了一个有效的准确性,衡量的不仅仅是模型是否正确,而是是否在适当的时间回答。基于这一公式,我们引入了StreamReady框架,通过一个轻量级的准备机制来统一时间推理与及时回答,该机制在回应前决定是否已观察到足够的证据。为了评估这种能力,我们进一步引入了ProReady-QA基准,该基准具有标注的答案证据窗口和跨本地和全局上下文的主动多轮问题。StreamReady在ProReady-QA上实现了优越的性能,并在八个额外的视频流和离线长视频基准上一致优于先前方法,证明了其鲁棒且广泛通用的视频理解能力。

英文摘要:

Streaming video understanding often involves time-sensitive scenarios where models need to answer exactly when the supporting visual evidence appears: answering before the evidence reflects speculation, answering after it has passed reduces real-time utility. To capture this behavior, we introduce a readiness-aware formulation of streaming video understanding with the Answer Readiness Score (ARS), a timing-aware objective with asymmetric early and late penalties. When combined with correctness, ARS defines an effective accuracy that measures not just whether a model is right, but whether it answers at the appropriate moment. Building on this formulation, we introduce StreamReady, a framework to unify temporal reasoning with on-time answering through a lightweight readiness mechanism that decides if sufficient evidence has been observed before responding. To evaluate this capability, we further introduce ProReady-QA, a benchmark with annotated answer evidence windows and proactive multi-turn questions across local and global contexts. StreamReady achieves superior performance on ProReady-QA, and consistently outperforms prior methods across eight additional streaming and offline long-video benchmarks, demonstrating robust and broadly generalizable video understanding capability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑