阅读≠视觉：诊断和缩小视觉语言模型中的字形差距
Reading $\neq$ Seeing: Diagnosing and Closing the Typography Gap in Vision-Language Models
AI总结：
本文研究了视觉语言模型在字形识别上的不足，发现字体风格检测能力差，通过微调提升模型性能，揭示训练数据缺失问题，并提出需架构创新以改进关系性视觉推理。
摘要：
视觉语言模型在图像中阅读文本时能够达到接近完美的准确性，但它们在字形方面却显得盲目：能够识别文本内容，但无法识别其外观。我们通过评估26种字体、四种脚本和三个难度级别，系统地调查这一差距。对15种最先进的VLMs进行评估，揭示出一个显著的感知层次：颜色识别几乎完美，但字体风格检测却普遍很差。我们进一步发现，模型规模无法预测性能，且准确性在不同难度级别上是均匀的，这表明可能是训练数据的缺失而非容量限制。在少量合成样本上进行LoRA微调显著提高了开源模型的性能，缩小了与最佳闭源系统的差距，并在字体大小识别上超越了它。字体风格本身仍难以通过微调来改进，这表明关系性视觉推理可能需要超出当前基于块的编码器的架构创新。我们发布我们的评估框架、数据和微调配方，以支持缩小视觉语言理解中的字形差距。
英文摘要：
Vision-Language Models achieve near-perfect accuracy at reading text in images, yet prove largely typography-blind: capable of recognizing what text says, but not how it looks. We systematically investigate this gap by evaluating font family, size, style, and color recognition across 26 fonts, four scripts, and three difficulty levels. Our evaluation of 15 state-of-the-art VLMs reveals a striking perception hierarchy: color recognition is near-perfect, yet font style detection remains universally poor. We further find that model scale fails to predict performance and that accuracy is uniform across difficulty levels, together pointing to a training-data omission rather than a capacity ceiling. LoRA fine-tuning on a small set of synthetic samples substantially improves an open-source model, narrowing the gap to the best closed-source system and surpassing it on font size recognition. Font style alone remains resistant to fine-tuning, suggesting that relational visual reasoning may require architectural innovation beyond current patch-based encoders. We release our evaluation framework, data, and fine-tuning recipe to support progress in closing the typographic gap in vision-language understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
