用于全模态密集视频字幕的并行自回归解码
Parallelized Autoregressive Decoding for Omni-Modal Dense Video Captioning
AI总结：
研究密集视频字幕生成，提出并行自回归框架，利用事件间弱局部依赖重组因果依赖图，引入全局规划和事件分解并行解码机制，提升效率与字幕性能。
摘要：
密集视频字幕旨在生成视频事件的时间定位描述。自回归视频大语言模型因强大生成和跨模态能力成为主流范式，但逐令牌范式限制推理效率。本文提出并行自回归框架，利用弱局部依赖重组因果图实现无损并行生成，介绍关键组件并通过实验验证优势。
英文摘要：
Dense video captioning aims to generate temporally grounded descriptions of video events, benefiting both event-level video understanding and generation. In this domain, autoregressive video large language models have emerged as a prevalent paradigm due to their strong generative and cross-modal modeling capacity. However, generating dense captions under the token-by-token paradigm severely limits inference efficiency and hinders scalability as video length and event density increase. In this work, we propose a parallelized autoregressive framework that not only improves generation efficiency but also enhances temporally grounded captioning performance. Our key insight is to exploit the weak local dependencies across temporally distinct events to restructure the causal dependency graph, thereby enabling lossless parallel generation. Specifically, tokens with weak cross-event dependencies can be decoded in parallel, while tightly coupled tokens within each event retain sequential decoding to preserve local semantic coherence. To realize this insight, we introduce two key components for lossless parallel decoding: (1) a latent global planning mechanism that automatically learns the event-level structure and produces compact tokens encoding global inter-event causality while adaptively aggregating event-level audio-visual semantics, guiding subsequent dependency restructuring and parallel decoding; and (2) an event-factorized parallel decoding mechanism that effectively balances local focus with global inter-event awareness. Experiments on various benchmarks demonstrate the clear advantage of our approach in both efficiency and performance in omni-modal event grounding and captioning. Project website: https://github.com/showlab/PadCaptioner.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
