Granulon: 通过自适应多粒度语义唤醒像素级视觉编码器以实现MLLM
Granulon: Awakening Pixel-Level Visual Encoders with Adaptive Multi-Granularity Semantics for MLLM
AI总结：
Granulon通过自适应多粒度语义增强，提升多模态大语言模型的像素级视觉理解和多粒度推理能力。
摘要：
最近多模态大语言模型的进步主要依赖于基于CLIP的视觉编码器，这些编码器强调全局语义对齐，但在细粒度视觉理解上存在困难。相比之下，DINOv3提供了强大的像素级感知能力，但缺乏粗粒度语义抽象，导致多粒度推理有限。为了解决这一差距，我们提出了Granulon，一种基于DINOv3的新型MLLM，具有自适应粒度增强功能。Granulon引入了一个文本条件的粒度控制器，根据文本输入的语义范围动态调整视觉抽象级别，并引入了一个自适应令牌聚合模块，通过粒度引导的池化和关系感知聚类生成紧凑且语义丰富的视觉令牌。这种设计使模型能够在单次前向传递中实现统一的
英文摘要：
Recent advances in multimodal large language models largely rely on CLIP-based visual encoders, which emphasize global semantic alignment but struggle with fine-grained visual understanding. In contrast, DINOv3 provides strong pixel-level perception yet lacks coarse-grained semantic abstraction, leading to limited multi-granularity reasoning. To address this gap, we propose Granulon, a novel DINOv3-based MLLM with adaptive granularity augmentation. Granulon introduces a text-conditioned granularity Controller that dynamically adjusts the visual abstraction level according to the semantic scope of the textual input, and an Adaptive Token Aggregation module that performs granularity-guided pooling and relation-aware clustering to produce compact, semantically rich visual tokens. This design enables unified "pixel-to-fine-to-coarse" reasoning within a single forward pass. Extensive and interpretable experiments demonstrate that Granulon improves accuracy by ~30% and reduces hallucination by ~20%, outperforming all visual encoders under identical settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
