面向向后兼容多模态表示的球面插值
Spherical Interpolation for Backward-Compatible Multimodal Representations
浏览论文内容
中文总结 AI 辅助
本文提出在正交对齐后沿球面测地线对新旧模型查询表示进行插值，以在不重新索引图库的情况下改善跨模态检索的向后兼容性，并通过理论刻画和实验验证其有效性。
中文摘要 AI 辅助
对比视觉-语言模型将视觉和文本表示映射到一个共享的归一化嵌入空间中，使得余弦相似度成为跨模态检索的自然度量。在模型升级过程中会出现一个实际挑战：独立训练的模型通常会产生不兼容的表示空间，因此替换已部署的模型通常需要重新计算整个图库的嵌入，这在规模上代价高昂。正交事后对齐可以通过将新模型查询映射到旧模型图库空间来部分缓解此问题。然而，由于独立训练的模型在细粒度表示结构上可能存在差异，正交对齐仍然是近似的，在旧模型查询和对齐后的新模型查询之间会留下残余的角度差异。我们研究了在这两个归一化查询表示之间的球面测地线上进行插值是否可以在不重新索引图库的情况下改善检索。我们刻画了该路径何时包含一个比两个端点更接近理想检索最优方向的内部查询方向，并通过基于局部边际的认证结果将此刻画与Recall@$K$联系起来。跨多个基准和模型家族的实验表明，对齐后的球面插值优于仅使用正交对齐，在大多数评估设置中恢复了向后兼容性。与我们的几何刻画一致，逐查询的oracle分析表明，有利于检索的内部点在实践中频繁出现。代码可在以下网址获取：此 https URL。
英文摘要
Contrastive vision-language models map visual and textual representations into a shared normalized embedding space, making cosine similarity the natural metric for cross-modal retrieval. A practical challenge arises during model upgrades: independently trained models generally produce incompatible representation spaces, so replacing a deployed model typically requires recomputing embeddings for the entire gallery, which is prohibitively expensive at scale. Orthogonal post-hoc alignment can partially mitigate this problem by mapping new-model queries into the old-model gallery space. However, because independently trained models can differ in fine-grained representation structure, the orthogonal alignment remains approximate, leaving a residual angular discrepancy between the old-model query and the aligned new-model query. We study whether interpolation along the spherical geodesic between these two normalized query representations can improve retrieval without re-indexing the gallery. We characterize when this path contains an interior query direction closer to an idealized retrieval-optimal direction than either endpoint, and connect this characterization to Recall@$K$ through a local margin-based certification result. Experiments across multiple benchmarks and model families show that post-alignment spherical interpolation improves over orthogonal alignment alone, recovering backward-compatibility in most evaluated settings. Consistent with our geometric characterization, per-query oracle analysis shows that retrieval-favorable interior points occur frequently in practice. Code is available at https://github.com/miccunifi/SLERP_backward_compatibility .
发表机构
- University of Florence（佛罗伦萨大学）
- University of Trento（特伦托大学）
机构由 AI 辅助整理，请以论文原文为准。
