arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.07720cs.IR
收藏

查询并非孤立:聚类文本嵌入用于视频搜索

Queries Are Not Alone: Clustering Text Embeddings for Video Search

National Engineering Research Center for Software Engineering, Peking University(软件工程国家工程研究中心,北京大学);Peking University(北京大学);City University of Hong Kong(香港城市大学)

Peyang Liu, Xi Wang, Ziqiang Cui, Wei Ye

AI总结:

本文提出视频-文本聚类(VTC)框架,通过聚类文本查询并引入Sweeper去噪和VTC-Att注意力机制,在五个公共数据集上超越现有最先进视频检索模型。

摘要:

各类平台上视频内容的迅速激增,凸显了开发先进视频检索系统的迫切需求。传统方法主要依赖于将文本查询与视频元数据进行直接匹配,往往难以弥合文本描述与视频内容多面性之间的语义鸿沟。本文提出了一种新颖的框架——视频-文本聚类(VTC),通过聚类文本查询来捕获更广泛的语义范围,从而增强视频检索。我们提出了一种独特的聚类机制,将相关查询分组,使系统能够考虑每个查询的多种解释和细微差别。这种聚类通过我们创新的Sweeper模块进一步优化,该模块识别并减轻这些聚类中的噪声。此外,我们引入了视频-文本聚类注意力(VTC-Att)机制,该机制根据视频内容动态调整聚类内的焦点,确保检索过程强调最相关的文本特征。进一步的实验表明,我们提出的模型在五个公共数据集上超越了现有的最先进模型。

英文摘要:

The rapid proliferation of video content across various platforms has highlighted the urgent need for advanced video retrieval systems. Traditional methods, which primarily depend on directly matching textual queries with video metadata, often fail to bridge the semantic gap between text descriptions and the multifaceted nature of video content. This paper introduces a novel framework, the Video-Text Cluster (VTC), which enhances video retrieval by clustering text queries to capture a broader semantic scope. We propose a unique clustering mechanism that groups related queries, enabling our system to consider multiple interpretations and nuances of each query. This clustering is further refined by our innovative Sweeper module, which identifies and mitigates noise within these clusters. Additionally, we introduce the Video-Text Cluster-Attention (VTC-Att) mechanism, which dynamically adjusts focus within the clusters based on the video content, ensuring that the retrieval process emphasizes the most relevant textual features. Further experiments have demonstrated that our proposed model surpasses existing state-of-the-art models on five public datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑