EGM: 高效视觉语义语言模型
EGM: Efficient Visual Grounding Language Models
AI总结：
本文提出EGM，通过生成大量中等质量token来提升小模型的视觉语义能力，实验证明其在效率和性能上优于大模型。
摘要：
视觉语义是视觉语言模型理解现实物理世界的关键能力。现有最先进的视觉语义模型通常具有大模型规模，导致部署困难和推理缓慢。然而，我们发现小规模和大规模VLM的视觉编码器大小几乎相同，主要差异在于语言模型的规模。小VLM在视觉语义任务中落后于大VLM并非因为视觉信息处理能力，而是语言理解能力的差距。为缩小这一差距，我们引入了'高效视觉语义语言模型'（EGM）：通过生成大量中等质量的token来匹配大模型的性能，使用少量高质量但昂贵的token。这种方法部署友好，且能显著提升端到端延迟：在RefCOCO基准测试中，我们的EGM-Qwen3-VL-8B在91.4 IoU的情况下，平均延迟仅为737ms（5.9倍更快），而Qwen3-VL-235B则需要4,320ms才能达到90.5 IoU。为了验证我们方法的通用性，我们进一步设定了新的amodal视觉语义设置，要求模型预测物体的可见和遮挡部分。实验表明，我们的方法在提升小模型的视觉语义能力和amodal视觉语义能力方面表现优异，能够匹配或超越大模型，从而提高视觉语义的效率。
英文摘要：
Visual grounding is an essential capability of Visual Language Models (VLMs) to understand the real physical world. Previous state-of-the-art grounding visual language models usually have large model sizes, making them heavy for deployment and slow for inference. However, we notice that the sizes of visual encoders are nearly the same for small and large VLMs and the major difference is the sizes of the language models. Small VLMs fall behind larger VLMs in grounding because of the difference in language understanding capability rather than visual information handling. To mitigate the gap, we introduce 'Efficient visual Grounding language Models' (EGM): generate many mid-quality tokens (from small models) to match the performance of large VLMs with few high-quality but expensive tokens. This method is deployment-friendly, and yields better end-to-end latency: On the RefCOCO benchmark, our EGM-Qwen3-VL-8B demonstrates 91.4 IoU with an average of 737ms (5.9x faster) latency while Qwen3-VL-235B demands 4,320ms to reach 90.5 IoU. To validate our approach's generality, we further set up a new amodal grounding setting that requires the model to predict both the visible and occluded parts of the objects. Experiments show our method consistently improves both vanilla and amodal grounding capabilities of small models to match or outperform larger models, thereby improving efficiency for visual grounding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
