共享视觉-语言嵌入空间的拓扑对齐
Topological Alignment of Shared Vision-Language Embedding Space
AI总结：
ToMCLIP通过拓扑对齐方法提升多语言视觉-语言模型的结构一致性和零样本性能
摘要：
对比性视觉-语言模型（VLMs）已展示了强大的零样本能力。然而，由于多语言多模态数据有限，其跨模态对齐仍然偏向英语。最近的多语言扩展已经缓解了这一差距，但强制实例级对齐而忽略了共享嵌入空间的全局几何结构。我们通过引入ToMCLIP（用于多语言CLIP的拓扑对齐），一种拓扑感知的框架，对嵌入空间进行拓扑保持约束的对齐。所提出的方法利用持久同调来定义拓扑对齐损失，并使用图稀疏化策略近似持久图，并通过理论误差界来近似持久图。本工作验证了所提出的方法，展示了多语言表示的增强结构一致性，CIFAR-100上的更高零样本准确性，以及在xFlickr&CO上的更强多语言检索性能。除了VLMs之外，所提出的方法还提供了一种通用方法，用于将拓扑对齐纳入表示学习。代码可在https://github.com/junwon0/ToMCLIP.git上获得。
英文摘要：
Contrastive Vision-Language Models (VLMs) have demonstrated strong zero-shot capabilities. However, their cross-modal alignment remains biased toward English due to limited multilingual multimodal data. Recent multilingual extensions have alleviated this gap but enforce instance-level alignment while neglecting the global geometry of the shared embedding space. We address this problem by introducing ToMCLIP (Topological Alignment for Multilingual CLIP), a topology-aware framework aligning embedding spaces with topology-preserving constraints. The proposed method applies persistent homology to define a topological alignment loss and approximates persistence diagram with theoretical error bounds using graph sparsification strategy. This work validates the proposed approach, showing enhanced structural coherence of multilingual representations, higher zero-shot accuracy on the CIFAR-100, and stronger multilingual retrieval performance on the xFlickr&CO. Beyond VLMs, the proposed approach provides a general method for incorporating topological alignment into representation learning. Code is available at https://github.com/junwon0/ToMCLIP.git.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
