CropVLM: 为细粒度视觉语言感知学习动态聚焦
CropVLM: Learning to Zoom for Fine-Grained Vision-Language Perception
AI总结：
CropVLM通过强化学习提升VLM在细粒度图像理解任务中的性能，无需人工标注或合成数据，有效增强模型对细节的捕捉能力。
摘要：
视觉语言模型（VLMs）在需要细粒度图像理解的任务（如场景文本识别或文档分析）中常因感知限制和视觉碎片化而表现不佳。为解决这些问题，我们引入CropVLM作为外部低成本方法，使VLM能够动态'聚焦'相关图像区域，提升其捕捉细节的能力。CropVLM通过强化学习训练，不依赖人工标注的边界框作为监督信号，也不依赖昂贵的合成评估。该模型训练一次即可与开源或专有VLM配对，提升其性能。我们的方法在需要高分辨率图像理解的任务上取得显著改进，特别是在目标VLM超出领域基准上表现突出，无需修改或微调VLM，从而避免灾难性遗忘。
英文摘要：
Vision-Language Models (VLMs) often struggle with tasks that require fine-grained image understanding, such as scene-text recognition or document analysis, due to perception limitations and visual fragmentation. To address these challenges, we introduce CropVLM as an external low-cost method for boosting performance, enabling VLMs to dynamically ''zoom in'' on relevant image regions, enhancing their ability to capture fine details. CropVLM is trained using reinforcement learning, without using human-labeled bounding boxes as a supervision signal, and without expensive synthetic evaluations. The model is trained once and can be paired with both open-source and proprietary VLMs to improve their performance. Our approach delivers significant improvements on tasks that require high-resolution image understanding, notably for benchmarks that are out-of-domain for the target VLM, without modifying or fine-tuning the VLM, thus avoiding catastrophic forgetting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
