The Large Vision-Language Models (LVLMs) have demonstrated great abilities in image perception and language understanding. However, existing multimodal benchmarks focus on primary perception abilities and commonsense knowledge which are insufficient to reflect the comprehensive capabilities of LVLMs. We propose GAOKAO-MM, a multimodal benchmark based on the Chinese College Entrance Examination (GAOKAO), comprising of 8 subjects and 12 types of images, such as diagrams, function graphs, maps and photos. GAOKAO-MM derives from native Chinese context and sets human-level requirements for the model's abilities, including perception, understanding, knowledge and reasoning. We evaluate 10 LVLMs and find that the accuracies of all of them are lower than 50%, with GPT-4-Vison (48.1%), Qwen-VL-Plus (41.2%) and Gemini-Pro-Vision (35.1%) ranking in the top three positions. The results of our multi-dimension analysis indicate that LVLMs have moderate distance towards Artificial General Intelligence (AGI) and provide insights facilitating the development of multilingual LVLMs.
CMMMU: A Chinese Massive Multi-discipline Multimodal Understanding Benchmark
Multimodal Art Projection Research Community（多模态艺术投影研究共同体）；Hong Kong University of Science and Technology（香港科技大学）；University of Waterloo（滑铁卢大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Peking University（北京大学）；The Hong Kong Polytechnic University（香港理工大学）；Waseda University（早稻田大学）；University of Manchester（曼彻斯特大学）
Creating a Lens of Chinese Culture: A Multimodal Dataset for Chinese Pun Rebus Art Understanding
University of Southern California（南加州大学）；Shanghai International Studies University（上海外国语大学）；The Butler Museum（巴特勒博物馆）；Jingdezhen Imperial Kiln Institute（景德镇御窑研究院）；The Ohio State University（俄亥俄州立大学）
本文构建深植中国传统文化的多模态 Pun Rebus Art Dataset，通过元素识别、象征匹配和信息解释任务评估 VLM，揭示其文化理解短板并推动更具包容性的模型发展。
ShizhenGPT: Towards Multimodal LLMs for Traditional Chinese Medicine
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Freedom AI；National Health Data Institute, Shenzhen（深圳国家健康数据研究院）；Shenzhen Research Institute of Big Data（深圳大数据研究院）；Guangdong University of Technology（广东工业大学）
Enhancing Multimodal Large Language Models for Ancient Chinese Character Evolution Analysis via Glyph-Driven Fine-Tuning
College of Computer Science and Technology, Jilin University, China（吉林大学计算机科学与技术学院）；Key Laboratory of Ancient Chinese Script, Culture Relics and Artificial Intelligence, Jilin University, China（吉林大学古文字、文物与人工智能重点实验室）；School of Artificial Intelligence, Jilin University, China（吉林大学人工智能学院）；School of Archaeology, Jilin University, China（吉林大学考古学院）
CNSL-bench: Benchmarking the Sign Language Understanding Capabilities of MLLMs on Chinese National Sign Language
School of Informatics, Xiamen University, China（厦门大学信息学院）；Key Lab of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian-Taiwan (XMU), Ministry of Culture and Tourism, China（福建省-台湾非物质文化遗产数字化保护与智能处理重点实验室（XMU），文化和旅游部，中国）；National Language Resources Monitoring and Research Center for Education and Teaching Media, Xiamen University, China（教育与教学媒体语言资源监测与研究中心，厦门大学，中国）
LLaVA-Ultra: Large Chinese Language and Vision Assistant for Ultrasound
ZJU-UIUC Institute, Zhejiang University（浙江大学ZJU-UIUC学院）；University of Washington（华盛顿大学）；Zhejiang University School of Medicine Sir Run Run Shaw Hospital（浙江大学医学院附属邵逸夫医院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
LingLan: An Advancing Traditional Chinese Medicine Diagnosis LLM with Multimodal Data
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Guangdong Provincial Laboratory of Traditional Chinese Medicine Hengqin（广东省中医药横琴实验室）
Hanfu-Bench: A Multimodal Benchmark on Cross-Temporal Cultural Understanding and Transcreation
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Research Institute of Big Data（深圳大数据研究院）；Chengdu Technological University（成都理工大学）；University of Copenhagen（哥本哈根大学）
FoodieQA: A Multimodal Dataset for Fine-Grained Understanding of Chinese Food Culture
University of Copenhagen（哥本哈根大学）；University of Waterloo（滑铁卢大学）；Comcast AI Technologies（康卡斯特人工智能技术公司）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Research Institute of Big Data（深圳大数据研究院）；University of Amsterdam（阿姆斯特丹大学）
Towards Comprehensive Detection of Chinese Harmful Memes
School of Computer Science and Technology（计算机科学与技术学院）；School of Foreign Languages（外国语学院）；Key Laboratory of Social Computing and Cognitive Intelligence（社会计算与认知智能重点实验室）；Dalian University of Technology（大连理工大学）
BenCao: An Instruction-Tuned Large Language Model for Traditional Chinese Medicine
Community Health Service Center Shanghai Pudong New Area（上海浦东新区社区卫生服务中心）；School of Acupuncture-Moxibustion and Tuina, Shanghai University of Traditional Chinese Medicine（上海中医药大学针灸推拿学院）
本文提出基于 ChatGPT 的中医药多模态助手 BenCao，通过自然语言指令微调、知识库、思维链模拟、专家反馈及外部 API 整合，提升诊断、药材识别与体质分类性能并实现部署。
Evidence-Based Intelligent Diagnostic and Therapeutic Visualization System with Large Language Models: Multi-Turn Interaction and Multimodal Treatment Plan Generation
Harbin Institute of Technology, Weihai（哈尔滨工业大学（威海））；Harbin Institute of Technology (Weihai) Qingdao Research Institute（哈尔滨工业大学（威海）青岛研究院）；Shandong Key Laboratory of Digital Service Computing Technology and Systems（山东省数字服务计算技术与系统重点实验室）；Weihai Municipal Hospital（威海市人民医院）；Shanghai Taizhu Technology Co., Ltd（上海泰山技术有限公司）；Tianjin Zhifu Qihuang Medical Technology Co., Ltd（天津中孚启黄医疗技术有限公司）
Can MLLMs Understand the Deep Implication Behind Chinese Images?
Huazhong University of Science and Technology（华中科技大学）；Shenzhen Institute of Advanced Technology, CAS（中国科学院深圳先进技术研究院）；University of Science and Technology of China（中国科学技术大学）；CDUT（成都理工大学）；University of Memphis（孟菲斯大学）；University of California, Santa Barbara（加利福尼亚大学圣巴巴拉分校）；SICAU（四川农业大学）；University of Manchester（曼彻斯特大学）；SWU（西南大学）；UCAS（中国科学院大学）；UMich（密歇根大学）
Can MLLMs Read Students' Minds? Unpacking Multimodal Error Analysis in Handwritten Math
Lehigh University（里海大学）；Squirrel Ai Learning（松鼠AI学习）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Michigan State University（密歇根州立大学）
ScImage: How Good Are Multimodal Large Language Models at Scientific Text-to-Image Generation?
University of Twente（特文特大学）；University of Technology Nuremberg (UTN)（纽伦堡工业大学（UTN））；University of Sheffield（谢菲尔德大学）；Harbin Institute of Technology（哈尔滨工业大学）；University of Mannheim（曼海姆大学）；Technische Universität Dresden（德累斯顿工业大学）
PALO: A Polyglot Large Multimodal Model for 5B People
Mohamed bin Zayed University of AI（穆罕默德·本·扎耶德人工智能大学）；Australian National University（澳大利亚国立大学）；Aalto University（阿尔托大学）；The University of Melbourne（墨尔本大学）；Linköping University（林雪平大学）
How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites
SenseTime Research（商汤科技研究院）；Shanghai AI Laboratory（上海人工智能实验室）；Tsinghua University（清华大学）；Nanjing University（南京大学）；Fudan University（复旦大学）；The Chinese University of Hong Kong（香港中文大学）
Human-Aligned Bench: Fine-Grained Assessment of Reasoning Ability in MLLMs vs. Humans
Wuhan University（武汉大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Shanghai Innovation Institute（上海创新研究院）；Harbin Institute of Technology（哈尔滨工业大学）
