arXiv 2508.169742025-08-23cs.CV

分层上下文接地LVLM：通过鲁棒接地增强细粒度视觉语言理解

Hierarchical Contextual Grounding LVLM: Enhancing Fine-Grained Visual-Language Understanding with Robust Grounding

Zhongkai University of Agriculture and Engineering（仲恺农业工程大学）；EDP University of Puerto Rico: San Sebastian（波多黎各圣塞巴斯蒂安EDP大学）

Leilei Guo, Antonio Carlos Rivera, Peiyu Tang, Haoxuan Ren, Zheyu Song

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对现有LVLM在复杂场景中鲁棒性不足、易幻觉及推理错误的问题，提出模仿人类粗到细认知的HCG-LVLM分层架构，通过全局-局部双层接地增强细粒度视觉语言理解，实验证明其优于现有模型。

摘要：

大型语言模型（LLMs）和视觉语言大型模型（LVLMs）在自然语言处理和多模态理解领域取得了显著进展。尽管泛化能力出色，但当前LVLMs在复杂真实场景中常表现出鲁棒性不足、易产生幻觉及推理错误的问题，尤其在需要精确图像区域定位和细粒度视觉推理时。为解决这些局限，我们提出分层上下文接地LVLM（HCG-LVLM），一种模仿人类从粗到细认知过程的新型架构。HCG-LVLM采用双层方法：全局上下文感知层用于初步宽泛理解，细粒度局部接地层则包含局部细节增强模块（提取高分辨率特征）和语义一致性验证器（确保准确、无幻觉的视觉语言对齐）。通过自适应融合机制，两层信息被整合以生成鲁棒且精确的输出。在GQA、A-OKVQA（细粒度视觉问答）、RefCOCO/+/g（指代表达理解）等挑战性数据集上的大量实验表明，HCG-LVLM持续优于Flamingo、BLIP-2、MiniGPT-4等现有模型，准确率更高且显著减少幻觉，验证了其分层设计在增强细粒度视觉语言理解和精确接地能力方面的有效性。

英文摘要：

Large Language Models (LLMs) and Vision-Language Large Models (LVLMs) have achieved remarkable progress in natural language processing and multimodal understanding. Despite their impressive generalization capabilities, current LVLMs often exhibit insufficient robustness, proneness to hallucination, and reasoning errors in complex real-world scenarios, particularly when precise image region localization and fine-grained visual reasoning are required. To address these limitations, we propose the Hierarchical Contextual Grounding LVLM (HCG-LVLM), a novel architecture that mimics human coarse-to-fine cognitive processing. HCG-LVLM employs a two-layered approach: a Global Contextual Perception layer for initial broad understanding and a Fine-grained Local Grounding layer. The latter incorporates a Local Detail Enhancement Module to extract high-resolution features and a Semantic Consistency Validator to ensure accurate, hallucination-free visual-language alignment. Through an adaptive fusion mechanism, information from both layers is integrated for robust and precise outputs. Extensive experiments on challenging datasets, including GQA, A-OKVQA for fine-grained VQA, and RefCOCO/+/g for Referring Expression Comprehension, demonstrate that HCG-LVLM consistently outperforms state-of-the-art models such as Flamingo, BLIP-2, and MiniGPT-4. Our model achieves superior accuracy and significantly reduces hallucination, validating the effectiveness of its hierarchical design in enhancing fine-grained visual-language understanding and precise grounding capabilities.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。