MedStreamBench: 面向流式与主动式医疗视频理解的时间感知基准
MedStreamBench: A Time-Aware Benchmark for Streaming and Proactive Medical Video Understanding
AI总结：
提出MedStreamBench基准，通过时间受限证据窗口和主动监控设置，评估模型在流式与主动式医疗视频理解中的回答时机与正确性，发现离线识别与时间感知决策间存在显著差距。
摘要：
现有的医疗视频基准主要评估模型是否给出正确答案，但很少评估其是否在正确的时间回答。在真实临床环境中，AI系统不仅需要决定预测什么，还要决定何时回答、推迟判断或主动发出警报。这造成了基准评估与部署需求之间的关键差距。我们提出了MedStreamBench，一个用于时间感知医疗视频理解的基准。MedStreamBench整合了22个医疗数据集和5,419个问答实例，涵盖四种时间设置：回顾性、当前、未来和主动式。与假设完整视频访问的传统基准不同，MedStreamBench将模型限制在时间受限的证据窗口内，并支持单轮和流式评估。我们进一步引入了一种主动监控设置，要求模型确定是否以及何时应触发临床相关警报。除了答案正确性外，MedStreamBench还通过响应性和事后稳定性来评估时间行为。在主流通用和医疗视觉语言模型上的实验表明，离线识别与时间感知决策之间存在显著差距，在流式和主动设置中性能显著下降。我们的基准可在该 https URL 获取。
英文摘要：
Existing medical video benchmarks primarily evaluate whether a model produces the correct answer, but rarely assess whether it answers at the right time. In real clinical settings, AI systems must decide not only what to predict, but also when to answer, defer judgment, or proactively raise alerts. This creates a critical gap between benchmark evaluation and deployment requirements. We present MedStreamBench, a benchmark for time-aware medical video understanding. MedStreamBench integrates 22 medical datasets and 5,419 QA instances across four temporal settings: retrospective, present, future, and proactive. Unlike conventional benchmarks that assume full-video access, MedStreamBench restricts models to temporally bounded evidence windows and supports both single-turn and streaming evaluation. We further introduce a proactive monitoring setting that requires models to determine whether and when clinically relevant alerts should be triggered. Beyond answer correctness, MedStreamBench evaluates temporal behavior through responsiveness and post-evidence stability. Experiments on leading general-purpose and medical vision-language models reveal a substantial gap between offline recognition and temporally grounded decision-making, with performance dropping markedly in streaming and proactive settings. Our benchmark is available at https://huggingface.co/datasets/Venn2024/MedStreamBench.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
