HiCM$^2$：用于密集视频描述的层次化紧凑记忆建模
HiCM$^2$: Hierarchical Compact Memory Modeling for Dense Video Captioning
AI总结：
针对密集视频描述任务，提出受人类认知启发的层次化紧凑记忆模型，通过记忆事件聚类与大语言模型总结构建高效层次化记忆及读取模块，在YouCook2和ViTT数据集上达到最先进性能。
摘要：
随着现实世界视频挑战解决方案的需求不断增长，对密集视频描述（DVC）的兴趣日益上升。DVC涉及对未剪辑视频的自动描述和定位。多项研究强调了DVC的挑战，并引入了利用先验知识（如预训练和外部记忆）的改进方法。在本研究中，我们提出一个模型，利用受人类记忆层次和认知启发的面向人类的层次化紧凑记忆的先验知识。为模仿人类记忆回忆，我们构建了层次化记忆和层次化记忆读取模块。我们通过对记忆事件进行聚类并使用大语言模型进行总结，构建了高效的层次化紧凑记忆。对比实验表明，该层次化记忆回忆过程通过在YouCook2和ViTT数据集上取得最先进性能，提升了DVC的性能。
英文摘要：
With the growing demand for solutions to real-world video challenges, interest in dense video captioning (DVC) has been on the rise. DVC involves the automatic captioning and localization of untrimmed videos. Several studies highlight the challenges of DVC and introduce improved methods utilizing prior knowledge, such as pre-training and external memory. In this research, we propose a model that leverages the prior knowledge of human-oriented hierarchical compact memory inspired by human memory hierarchy and cognition. To mimic human-like memory recall, we construct a hierarchical memory and a hierarchical memory reading module. We build an efficient hierarchical compact memory by employing clustering of memory events and summarization using large language models. Comparative experiments demonstrate that this hierarchical memory recall process improves the performance of DVC by achieving state-of-the-art performance on YouCook2 and ViTT datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
