StructAlign：结构化跨模态对齐用于连续文本到视频检索
StructAlign: Structured Cross-Modal Alignment for Continual Text-to-Video Retrieval
AI总结：
本文提出StructAlign，通过结构化跨模态对齐方法解决连续文本到视频检索中的模态错位和特征漂移问题，提升模型持续学习能力。
摘要：
连续文本到视频检索（CTVR）是一种具有挑战性的多模态持续学习场景，其中模型必须逐步学习新语义类别，同时保持对之前学习类别的准确文本-视频对齐，这使其特别容易出现灾难性遗忘。CTVR的关键挑战是特征漂移，表现为两种形式：由每个模态内持续学习引起的模态内特征漂移，以及跨模态非合作性特征漂移导致的模态错位。为缓解这些问题，我们提出StructAlign，一种用于CTVR的结构化跨模态对齐方法。首先，StructAlign引入了一个简单等角紧框架（ETF）几何作为统一的几何先验，以缓解模态错位。在此几何先验基础上，我们设计了一种跨模态ETF对齐损失，将文本和视频特征与类别级ETF原型对齐，促使学习的表示形成近似的ETF几何结构。此外，为抑制模态内特征漂移，我们设计了一种跨模态关系保持损失，利用互补模态来保持跨模态相似性关系，为特征更新提供稳定的相对监督。通过共同解决跨模态非合作性特征漂移和模态内特征漂移，StructAlign有效缓解了CTVR中的灾难性遗忘。在基准数据集上的广泛实验表明，我们的方法在持续检索方面始终优于最先进的持续检索方法。
英文摘要：
Continual Text-to-Video Retrieval (CTVR) is a challenging multimodal continual learning setting, where models must incrementally learn new semantic categories while maintaining accurate text-video alignment for previously learned ones, thus making it particularly prone to catastrophic forgetting. A key challenge in CTVR is feature drift, which manifests in two forms: intra-modal feature drift caused by continual learning within each modality, and non-cooperative feature drift across modalities that leads to modality misalignment. To mitigate these issues, we propose StructAlign, a structured cross-modal alignment method for CTVR. First, StructAlign introduces a simplex Equiangular Tight Frame (ETF) geometry as a unified geometric prior to mitigate modality misalignment. Building upon this geometric prior, we design a cross-modal ETF alignment loss that aligns text and video features with category-level ETF prototypes, encouraging the learned representations to form an approximate simplex ETF geometry. In addition, to suppress intra-modal feature drift, we design a Cross-modal Relation Preserving loss, which leverages complementary modalities to preserve cross-modal similarity relations, providing stable relational supervision for feature updates. By jointly addressing non-cooperative feature drift across modalities and intra-modal feature drift, StructAlign effectively alleviates catastrophic forgetting in CTVR. Extensive experiments on benchmark datasets demonstrate that our method consistently outperforms state-of-the-art continual retrieval approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
