跟随显著性：用于检索增强的密集视频描述的监督显著性
Follow the Saliency: Supervised Saliency for Retrieval-augmented Dense Video Captioning
AI总结：
本文提出STaRC框架，通过显著性检测模块监督帧级显著性，以提升密集视频描述中时间分割的准确性，实现更精确的检索和上下文相关的描述生成。
摘要：
现有的检索增强密集视频描述（DVC）方法往往无法准确地进行时间分割，因为它们依赖于启发式策略，忽略了真实事件边界。所提出的框架STaRC通过显著性检测模块监督帧级显著性来克服这一限制。注意，显著性检测模块是通过直接从DVC地面真实标注中获得二元标签进行训练的，而无需额外标注。我们还提出利用显著性分数作为统一的时间信号，通过显著性引导的分割进行检索，并通过显式注入解码器的显著性提示来指导描述生成。通过强制显著性约束的分割，我们的方法生成了时间上连贯的片段，这些片段与实际事件转换紧密对齐，从而实现了更准确的检索和上下文相关的描述生成。我们在YouCook2和ViTT基准上进行了全面评估，其中STaRC在大多数指标上均达到最先进的性能。我们的代码可在https://github.com/ermitaju1/STaRC获取。
英文摘要：
Existing retrieval-augmented approaches for Dense Video Captioning (DVC) often fail to achieve accurate temporal segmentation aligned with true event boundaries, as they rely on heuristic strategies that overlook ground truth event boundaries. The proposed framework, \textbf{STaRC}, overcomes this limitation by supervising frame-level saliency through a highlight detection module. Note that the highlight detection module is trained on binary labels derived directly from DVC ground truth annotations without the need for additional annotation. We also propose to utilize the saliency scores as a unified temporal signal that drives retrieval via saliency-guided segmentation and informs caption generation through explicit Saliency Prompts injected into the decoder. By enforcing saliency-constrained segmentation, our method produces temporally coherent segments that align closely with actual event transitions, leading to more accurate retrieval and contextually grounded caption generation. We conduct comprehensive evaluations on the YouCook2 and ViTT benchmarks, where STaRC achieves state-of-the-art performance across most of the metrics. Our code is available at https://github.com/ermitaju1/STaRC
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
