arXiv 2603.238962026-03-25cs.CV

MMTIT-Bench: 一个具有认知-感知-推理引导的多语言多场景文本-图像机器翻译基准

MMTIT-Bench: A Multilingual and Multi-Scenario Benchmark with Cognition-Perception-Reasoning Guided Text-Image Machine Translation

Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；Tencent（腾讯）；Nankai University（南开大学）；University of Chinese Academy of Sciences（中国科学院大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Nanjing University of Science and Technology（南京理工大学）

Gengluo Li, Chengquan Zhang, Yupu Liang, Huawen Shen, Yaping Zhang, Pengyuan Lyu, Weinong Wang, Xingyu Wan, Gangyan Zeng, Han Hu, Can Ma, Yu Zhou

备注：: Accepted to CVPR 2026

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

MMTIT-Bench通过1400张覆盖14种非英语和非中文语言的图像，评估端到端文本-图像机器翻译的鲁棒性，并提出CPR-Trans数据范式提升翻译推理能力。

摘要：

端到端文本-图像机器翻译（TIMT）通过直接跨语言翻译图像中的文本内容，对现实世界多语言场景理解至关重要。尽管视觉语言大模型（VLLMs）取得进展，但因评估资源有限，跨多样视觉场景和低资源语言的鲁棒性仍待探索。我们提出MMTIT-Bench，一个经人类验证的多语言多场景基准，包含1400张图像，覆盖14种非英语和非中文语言及文档、场景、网络图像等多样化场景，从而严格评估端到端TIMT。除了基准测试，我们研究了以推理为导向的数据设计如何提升翻译。尽管最近的VLLMs开始整合长链思考（CoT）推理，但有效的TIMT思考范式仍不成熟：现有设计要么按顺序解析和翻译，要么专注于语言推理，忽略了VLLMs核心的视觉认知。我们提出翻译中的认知-感知-推理（CPR-Trans），一种整合场景认知、文本感知和翻译推理的数据范式，通过VLLM驱动的数据生成管道，提供结构化、可解释的监督，使感知与推理对齐。在3B和7B模型上的实验显示，准确性和可解释性均有所提升。我们将在接受后发布MMTIT-Bench，以促进多语言多场景TIMT研究。

英文摘要：

End-to-end text-image machine translation (TIMT), which directly translates textual content in images across languages, is crucial for real-world multilingual scene understanding. Despite advances in vision-language large models (VLLMs), robustness across diverse visual scenes and low-resource languages remains underexplored due to limited evaluation resources. We present MMTIT-Bench, a human-verified multilingual and multi-scenario benchmark with 1,400 images spanning fourteen non-English and non-Chinese languages and diverse settings such as documents, scenes, and web images, enabling rigorous assessment of end-to-end TIMT. Beyond benchmarking, we study how reasoning-oriented data design improves translation. Although recent VLLMs have begun to incorporate long Chain-of-Thought (CoT) reasoning, effective thinking paradigms for TIMT are still immature: existing designs either cascade parsing and translation in a sequential manner or focus on language-only reasoning, overlooking the visual cognition central to VLLMs. We propose Cognition-Perception-Reasoning for Translation (CPR-Trans), a data paradigm that integrates scene cognition, text perception, and translation reasoning within a unified reasoning process. Using a VLLM-driven data generation pipeline, CPR-Trans provides structured, interpretable supervision that aligns perception with reasoning. Experiments on 3B and 7B models show consistent gains in accuracy and interpretability. We will release MMTIT-Bench to promote the multilingual and multi-scenario TIMT research upon acceptance.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。