Global-Local Dual Perception for MLLMs in High-Resolution Text-Rich Image Translation
School of Computer Science and Technology, East China Normal University(东华大学计算机科学与技术学院);Labs, Huawei Technologies Co., LTD(华为技术有限公司2012实验室);School of Automation and Intelligent Sensing, Shanghai Jiao Tong University(上海交通大学自动化与智能感知学院)
Text Image Machine Translation (TIMT) aims to translate text embedded in images in the source-language into target-language, requiring synergistic integration of visual perception and linguistic understanding. Existing TIMT methods, whether cascaded pipelines or end-to-end multimodal large language models (MLLMs),struggle with high-resolution text-rich images due to cluttered layouts, diverse fonts, and non-textual distractions, resulting in text omission, semantic drift, and contextual inconsistency. To address these challenges, we propose GLoTran, a global-local dual visual perception framework for MLLM-based TIMT. GLoTran integrates a low-resolution global image with multi-scale region-level text image slices under an instruction-guided alignment strategy, conditioning MLLMs to maintain scene-level contextual consistency while faithfully capturing fine-grained textual details. Moreover, to realize this dual-perception paradigm, we construct GLoD, a large-scale text-rich TIMT dataset comprising 510K high-resolution global-local image-text pairs covering diverse real-world scenarios. Extensive experiments demonstrate that GLoTran substantially improves translation completeness and accuracy over state-of-the-art MLLMs, offering a new paradigm for fine-grained TIMT under high-resolution and text-rich conditions.
FaceBench: A Multi-View Multi-Level Facial Attribute VQA Dataset for Benchmarking Face Perception MLLMs
Shenzhen University(深圳大学);Tsinghua University(清华大学);The Hong Kong Polytechnic University(香港理工大学);City University of Hong Kong(香港城市大学);Sun Yat-sen University(中山大学)
CodePercept: Code-Grounded Visual STEM Perception for MLLMs
MoE Key Lab of Artificial Intelligence, AI Institute, School of Computer Science, Shanghai Jiao Tong University(人工智能大模型重点实验室,人工智能学院,计算机科学学院,上海交通大学);Qwen Team(通义实验室);Beijing Institute of Technology(北京理工大学);Tsinghua University(清华大学);Zhejiang University(浙江大学)
Reasoning-Aligned Perception Decoupling for Scalable Multi-modal Reasoning
Southern University of Science and Technology(南方科技大学);The Hong Kong University of Science and Technology(香港科技大学);Huawei Noah’s Ark Lab(华为诺亚实验室);Huawei Cloud Project(华为云项目)
FaceInsight: A Multimodal Large Language Model for Face Perception
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院);School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University(中山大学深圳校区计算机科学与技术学院);Key Laboratory of Education Informatization for Nationalities (Yunnan Normal University), Ministry of Education(民族教育信息化重点实验室(云南师范大学))
Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward
University of Science and Technology of China(中国科学技术大学);State Key Laboratory of Cognitive Intelligence(认知智能国家重点实验室);iFLYTEK AI Research(iFLYTEK人工智能研究院);Hong Kong University of Science and Technology(香港科学与技术大学)
ActiveScope: Actively Seeking and Correcting Perception for MLLMs
University of Chinese Academy of Sciences(中国科学院大学);State Key Lab of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China(中国科学院人工智能安全国家重点实验室,计算技术研究所,北京,中国);Harbin Institute of Technology (Weihai)(哈尔滨工业大学(威海))
Free-MoRef: Instantly Multiplexing Context Perception Capabilities of Video-MLLMs within Single Inference
Sun Yat-sen University(中山大学);Peng Cheng Laboratory(鹏城实验室);OPPO AI Center(OPPO人工智能中心);Research Institute(研究 institute);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Harbin Institute of Technology(哈尔滨工业大学);Shenzhen Key Laboratory of Digital Living Network and Content Service(深圳数字生活网络与内容服务重点实验室);Guangdong Key Laboratory of Big Data Analysis and Processing(广东省大数据分析与处理重点实验室)
Reinforcing Egocentric Spatial Perception in Multimodal Large Language Models via Ego Scene Augmentation
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Guangxi Zhuang Autonomous Region Information Center(广西壮族自治区信息中心);The Hong Kong University of Science and Technology(香港科技大学)
Enhancing Perception Capabilities of Multimodal LLMs with Training-Free Fusion
ZIP Lab, Zhejiang University(浙江大学 ZIP 实验室);South China University of Technology(华南理工大学);Pazhou Lab(琶洲实验室);Key Laboratory of Big Data and Intelligent Robot, Ministry of Education(大数据与智能机器人教育部重点实验室)
Unveiling and Bridging the Functional Perception Gap in MLLMs: Atomic Visual Alignment and Hierarchical Evaluation via PET-Bench
School of Biomedical Engineering, Southern Medical University(生物医学工程学院,南方医科大学);School of Biomedical Engineering, Shanghai Jiaotong University(生物医学工程学院,上海交通大学);Department of Electronic Engineering, Chinese University of Hong Kong(电子工程系,中国香港大学);Faculty of Dentistry, The University of Hong Kong(牙科学院,香港大学);Department of Nuclear Medicine, The Second Affiliated Hospital of Guangzhou University of Chinese Medicine(核医学科,广州中医药大学第二附属医院);PET Center, Department of Nuclear Medicine, Guangdong Provincial People’s Hospital, Southern Medical University(PET中心,核医学科,广东省人民医院,南方医科大学);Department of Nuclear Medicine, Nanfang Hospital, Southern Medical University(核医学科,南芳医院,南方医科大学);Division of Nuclear Medicine and Molecular Imaging, Geneva University Hospitals(核医学与分子影像学部,日内瓦大学医院);Departments of Radiology, Physics, and Biomedical Engineering, The University of British Columbia(放射学、物理和生物医学工程系,不列颠哥伦比亚大学);Medical Artificial Intelligence Laboratory, Westlake University(医学人工智能实验室,西湖大学)