基于部分-整体语义代表性与不确定性引导的组合对齐超几何视觉语言模型
Uncertainty-guided Compositional Alignment with Part-to-Whole Semantic Representativeness in Hyperbolic Vision-Language Models
AI总结：
本文提出UNCHA方法，通过超几何不确定性建模部分-整体语义代表性，提升超几何VLM对多物体场景的组合结构理解能力，实现零样本分类等任务的最先进性能。
摘要：
尽管视觉语言模型（VLMs）已取得显著成效，但其欧几里得嵌入在捕捉层次关系如部分-整体或父-子结构时仍有限，且在多物体组合场景中面临挑战。超几何VLMs通过蕴含关系更好地保留层次结构和部分-整体关系。然而，现有方法未建模每个部分对整体的不同语义代表性。我们提出UNcertainty-guided Compositional Hyperbolic Alignment（UNCHA）以增强超几何VLMs。UNCHA通过超几何不确定性建模部分-整体语义代表性，将更具有代表性的部分分配较低的不确定性，而较不具有代表性的部分分配较高的不确定性。此代表性随后被纳入对比目标中，通过不确定性引导的权重。最后，不确定性进一步通过基于熵的项正则化的蕴含损失进行校准。通过所提出的损失，UNCHA学习出更准确的部分-整体顺序的超几何嵌入，捕捉图像中的潜在组合结构，并提升对复杂多物体场景的理解。UNCHA在零样本分类、检索和多标签分类基准上实现了最先进的性能。我们的代码和模型可在：https://github.com/jeeit17/UNCHA.git 获取。
英文摘要：
While Vision-Language Models (VLMs) have achieved remarkable performance, their Euclidean embeddings remain limited in capturing hierarchical relationships such as part-to-whole or parent-child structures, and often face challenges in multi-object compositional scenarios. Hyperbolic VLMs mitigate this issue by better preserving hierarchical structures and modeling part-whole relations (i.e., whole scene and its part images) through entailment. However, existing approaches do not model that each part has a different level of semantic representativeness to the whole. We propose UNcertainty-guided Compositional Hyperbolic Alignment (UNCHA) for enhancing hyperbolic VLMs. UNCHA models part-to-whole semantic representativeness with hyperbolic uncertainty, by assigning lower uncertainty to more representative parts and higher uncertainty to less representative ones for the whole scene. This representativeness is then incorporated into the contrastive objective with uncertainty-guided weights. Finally, the uncertainty is further calibrated with an entailment loss regularized by entropy-based term. With the proposed losses, UNCHA learns hyperbolic embeddings with more accurate part-whole ordering, capturing the underlying compositional structure in an image and improving its understanding of complex multi-object scenes. UNCHA achieves state-of-the-art performance on zero-shot classification, retrieval, and multi-label classification benchmarks. Our code and models are available at: https://github.com/jeeit17/UNCHA.git.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
