DisCoCLIP：一种用于视觉-语言理解的分布式组合张量网络编码器
DisCoCLIP: A Distributional Compositional Tensor Network Encoder for Vision-Language Understanding
AI总结：
DisCoCLIP通过张量网络文本编码器显式编码句法结构，结合冻结的CLIP视觉变换器，以参数高效方式显著提升视觉-语言组合推理，如动词准确率提升至82.4%，SVO-Swap达93.7%。
摘要：
最近的视觉-语言模型在大规模图像-文本对齐方面表现出色，但往往忽略了语言的组合结构，导致在依赖词序和谓词-论元结构的任务上失败。我们引入了DisCoCLIP，一种多模态编码器，它将冻结的CLIP视觉变换器与一种新颖的张量网络文本编码器相结合，该编码器显式编码句法结构。句子通过组合范畴语法解析器进行解析，以产生分布式词张量，其收缩镜像了句子的语法推导。为了保持模型高效，高阶张量通过张量分解进行因子化，将参数数量从数千万减少到不足一百万。使用自监督对比损失进行端到端训练，DisCoCLIP显著提高了对动词语义和词序的敏感性：它将CLIP的SVO-Probes动词准确率从77.6%提高到82.4%，将ARO属性得分和关系得分分别提高了超过9%和4%，并在新引入的SVO-Swap基准上达到了93.7%。这些结果表明，通过张量网络嵌入显式语言结构能够产生可解释、参数高效的表示，从而显著提高视觉-语言任务中的组合推理能力。
英文摘要：
Recent vision-language models excel at large-scale image-text alignment but often neglect the compositional structure of language, leading to failures on tasks that hinge on word order and predicate-argument structure. We introduce DisCoCLIP, a multimodal encoder that combines a frozen CLIP vision transformer with a novel tensor network text encoder that explicitly encodes syntactic structure. Sentences are parsed with a Combinatory Categorial Grammar parser to yield distributional word tensors whose contractions mirror the sentence's grammatical derivation. To keep the model efficient, high-order tensors are factorized with tensor decompositions, reducing parameter count from tens of millions to under one million. Trained end-to-end with a self-supervised contrastive loss, DisCoCLIP markedly improves sensitivity to verb semantics and word order: it raises CLIP's SVO-Probes verb accuracy from 77.6% to 82.4%, boosts ARO attribution and relation scores by over 9% and 4%, and achieves 93.7% on a newly introduced SVO-Swap benchmark. These results demonstrate that embedding explicit linguistic structure via tensor networks yields interpretable, parameter-efficient representations that substantially improve compositional reasoning in vision-language tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
