DynaStride: 基于MMCoT的动态步长窗多场景描述生成
DynaStride: Dynamic Stride Windowing with MMCoT for Instructional Multi-Scene Captioning
AI总结：
DynaStride通过多模态链式思考和动态步长窗算法，实现无需手动分割的多场景教学视频连贯描述生成，提升描述的连贯性和信息量。
摘要：
在教学视频中进行场景级描述可以增强学习，因为它需要理解视觉线索和时间结构。通过将视觉线索与文本指导对齐，这种理解支持程序学习和多模态推理，为技能获取提供更丰富的上下文。然而，未能捕捉到这种结构的描述可能缺乏连贯性和质量，从而造成混淆并削弱视频的教育意图。为了解决这一缺口，我们引入了DynaStride，一种生成连贯、场景级描述的管道，而无需手动场景分割。利用YouCookII数据集的场景注释，DynaStride执行自适应帧采样和多模态窗框以捕捉每个场景中的关键转换。然后，它采用多模态链式思考过程生成多个动作-对象对，这些对通过动态步长窗选择算法进行细化和融合，该算法能够自适应地平衡时间上下文和冗余。最终的场景级描述在单个教学描述中整合了视觉语义和时间推理。与VLLaMA3和GPT-4o等强大基线的实证评估表明，在基于n-gram的度量（BLEU、METEOR）和语义相似性度量（BERTScore、CLIPScore）上都表现出一致的提升。定性分析进一步表明，DynaStride生成的描述在时间上更加连贯和信息丰富，表明了在改进AI驱动的教学内容生成方面的有前途的方向。
英文摘要：
Scene-level captioning in instructional videos can enhance learning by requiring an understanding of both visual cues and temporal structure. By aligning visual cues with textual guidance, this understanding supports procedural learning and multimodal reasoning, providing a richer context for skill acquisition. However, captions that fail to capture this structure may lack coherence and quality, which can create confusion and undermine the video's educational intent. To address this gap, we introduce DynaStride, a pipeline to generate coherent, scene-level captions without requiring manual scene segmentation. Using the YouCookII dataset's scene annotations, DynaStride performs adaptive frame sampling and multimodal windowing to capture key transitions within each scene. It then employs a multimodal chain-of-thought process to produce multiple action-object pairs, which are refined and fused using a dynamic stride window selection algorithm that adaptively balances temporal context and redundancy. The final scene-level caption integrates visual semantics and temporal reasoning in a single instructional caption. Empirical evaluations against strong baselines, including VLLaMA3 and GPT-4o, demonstrate consistent gains on both N-gram-based metrics (BLEU, METEOR) and semantic similarity measures (BERTScore, CLIPScore). Qualitative analyses further show that DynaStride produces captions that are more temporally coherent and informative, suggesting a promising direction for improving AI-powered instructional content generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
