TriPSS：一种利用感知、结构与语义表征的三模态关键帧提取框架
TriPSS: A Tri-Modal Keyframe Extraction Framework Using Perceptual, Structural, and Semantic Representations
AI总结：
本研究提出三模态关键帧提取框架TriPSS，融合感知、结构与语义表征，经聚类和优化生成关键帧，在两个基准上达到SOTA，可用于视频摘要等多媒体任务。
摘要：
高效的关键帧提取对视频摘要与检索至关重要，但完整捕捉视频内容的语义与视觉丰富性仍具挑战。我们提出TriPSS，这一三模态框架整合了CIELAB颜色空间的感知特征、ResNet-50的结构嵌入，以及由LLaMA-3.2-11B-Vision-Instruct生成的帧级字幕的语义上下文。这些模态通过主成分分析融合，形成紧凑的多模态嵌入，进而通过HDBSCAN聚类实现自适应视频分割。包含质量评估与重复过滤的优化阶段，确保最终关键帧集合既简洁又具备语义多样性。在TVSum20和SumMe基准上的评估显示，TriPSS达到了当前最优性能，显著优于单模态方法与此前的多模态方法。这些结果凸显了TriPSS捕捉互补视觉与语义线索的能力，使其成为视频摘要、检索及大规模多媒体理解的有效解决方案。
英文摘要：
Efficient keyframe extraction is critical for video summarization and retrieval, yet capturing the full semantic and visual richness of video content remains challenging. We introduce TriPSS, a tri-modal framework that integrates perceptual features from the CIELAB color space, structural embeddings from ResNet-50, and semantic context from frame-level captions generated by LLaMA-3.2-11B-Vision-Instruct. These modalities are fused using principal component analysis to form compact multi-modal embeddings, enabling adaptive video segmentation via HDBSCAN clustering. A refinement stage incorporating quality assessment and duplicate filtering ensures the final keyframe set is both concise and semantically diverse. Evaluations on the TVSum20 and SumMe benchmarks show that TriPSS achieves state-of-the-art performance, significantly outperforming both unimodal and prior multimodal approaches. These results highlight TriPSS' ability to capture complementary visual and semantic cues, establishing it as an effective solution for video summarization, retrieval, and large-scale multimedia understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
