ThaiOCRBench: 一种任务多样化的泰语视觉-语言理解基准
ThaiOCRBench: A Task-Diverse Benchmark for Vision-Language Understanding in Thai
AI总结：
ThaiOCRBench是一个针对泰语视觉-语言理解任务的多样化基准，通过人工标注的数据集评估了多种VLMs，揭示了专有模型在性能上的优势及开源模型在细粒度文本识别中的挑战。
摘要：
我们提出了ThaiOCRBench，这是首个全面评估视觉-语言模型（VLMs）在泰语文本丰富的视觉理解任务上的基准。尽管多模态建模取得了进展，现有基准主要集中在高资源语言上，使泰语在需要文档结构理解的任务中代表性不足。ThaiOCRBench通过提供包含2,808个样本的多样化、人工标注的数据集，涵盖13个任务类别，填补了这一空白。我们评估了广泛的状态-of-the-art VLMs，在零样本设置下覆盖了专有和开源系统。结果表明，专有模型（如Gemini 2.5 Pro）在性能上显著优于开源模型。值得注意的是，细粒度文本识别和手写内容提取在开源模型中表现出最陡的性能下降。通过详细的错误分析，我们识别出关键挑战，如语言偏差、结构不匹配和幻觉内容。ThaiOCRBench为在低资源、脚本复杂环境下评估VLMs提供了标准化框架，并为改进泰语文档理解提供了可行的见解。
英文摘要：
We present ThaiOCRBench, the first comprehensive benchmark for evaluating vision-language models (VLMs) on Thai text-rich visual understanding tasks. Despite recent progress in multimodal modeling, existing benchmarks predominantly focus on high-resource languages, leaving Thai underrepresented, especially in tasks requiring document structure understanding. ThaiOCRBench addresses this gap by offering a diverse, human-annotated dataset comprising 2,808 samples across 13 task categories. We evaluate a wide range of state-of-the-art VLMs in a zero-shot setting, spanning both proprietary and open-source systems. Results show a significant performance gap, with proprietary models (e.g., Gemini 2.5 Pro) outperforming open-source counterparts. Notably, fine-grained text recognition and handwritten content extraction exhibit the steepest performance drops among open-source models. Through detailed error analysis, we identify key challenges such as language bias, structural mismatch, and hallucinated content. ThaiOCRBench provides a standardized framework for assessing VLMs in low-resource, script-complex settings, and provides actionable insights for improving Thai-language document understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
