PCA：用于快速视频大语言模型的持久性感知压缩与聚合
PCA: Persistence-Aware Compression and Aggregation for Fast Video Large Language Models
- Institute for Artificial Intelligence, Great Bay University（大湾区大学人工智能研究院）
- Sun Yat-sen University（中山大学）
- Youtu Lab, Tencent（腾讯优图实验室）
- Shenzhen University（深圳大学）
- Dongguan Key Laboratory for Intelligence and Information Technology（东莞市智能信息技术重点实验室）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
研究视频大语言模型长时帧冗余问题，提出PCA方法，含动态下采样和持久性感知运动增强模块，能在编码前保留视觉信息，提升效率与准确性，优于现有方法，速度有显著提升。
AI中文摘要：
尽管视频大语言模型（VLLMs）在视频理解方面取得了进展，但其长时帧冗余阻碍了高效推理。本文介绍了一种无需训练的持久性感知压缩与聚合（PCA）方法，旨在编码前保留高保真原始视觉信息。它由动态下采样（DD）模块和持久性感知运动增强（PAME）模块组成。实验表明，PCA在效率和准确性上均优于现有方法，速度提升1.8至2.5倍。
英文摘要：
Despite advances in Video Large Language Models (VLLMs) that have displayed promising outcomes in video understanding, the redundancy in the long-duration frames remains a hindrance to efficient reasoning. This paper introduces a training-free $\mathbf{P}$ersistence-Aware $\mathbf{C}$ompression and $\mathbf{A}$ggregation (PCA) method designed to preserve high-fidelity raw visual information before the encoding stage. PCA can be built on arbitrary VLLMs and consists of two modules: 1) A Dynamic Downsampling (DD) module that adaptively removes redundant frames by analyzing frame-wise similarity. 2) A Persistence-Aware Motion Enhancement (PAME) module that enriches each selected keyframe by aggregating the temporal context of its neighbors, ensuring that essential information is preserved even after aggressive frame reduction. Our approach substantially reduces the computation of long-context modeling, while enhancing the performance of the baseline model. Extensive experiments demonstrate that PCA consistently outperforms existing state-of-the-art approaches in both efficiency and accuracy, achieving a speedup of 1.8$\times$ to 2.5$\times$ compared to the baseline VLLM. The code is open-sourced at https://github.com/Heisenberg10110/PCA.
