Are Arabic Benchmarks Reliable? QIMMA's Quality-First Approach to LLM Evaluation
Technology Innovation Institute（技术创新研究所）
Leen AlQadi, Ahmed Alzubaidi, Mohammed Alyafeai, Hamza Alobeidli, Maitha Alhammadi, Shaikha Alsuwaidi, Omar Alkaabi, Basma El Amel Boussaha, Hakim Hacid
We present QIMMA, a quality-assured Arabic LLM leaderboard that places systematic benchmark validation at its core. Rather than aggregating existing resources as-is, QIMMA applies a multi-model assessment pipeline combining automated LLM judgment with human review to surface and resolve systematic quality issues in well-established Arabic benchmarks before evaluation. The result is a curated, multi-domain, multi-task evaluation suite of over 52k samples, grounded predominantly in native Arabic content; code evaluation tasks are the sole exception, as they are inherently language-agnostic. Transparent implementation via LightEval, EvalPlus and public release of per-sample inference outputs make QIMMA a reproducible and community-extensible foundation for Arabic NLP evaluation.
AraHalluEval: A Fine-grained Hallucination Evaluation Framework for Arabic LLMs
Information and Computer Science Department, King Fahd University of Petroleum and Minerals（信息与计算机科学系，国王法赫德石油和矿物大学）；SDAIA-KFUPM Joint Research Center for Artificial Intelligence（SDAIA-KFUPM人工智能联合研究中心）
AraTrust: An Evaluation of Trustworthiness for LLMs in Arabic
Center of Research Excellence in AI and Data Science（人工智能与数据科学卓越研究中心）；King Abdulaziz University（阿卜杜勒阿齐兹国王大学）；ASAS AI（ASAS人工智能公司）；University College London（伦敦大学学院）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Alexandria University（亚历山大大学）
The Landscape of Arabic Large Language Models (ALLMs): A New Era for Arabic Language Technology
is a Researcher at iWAN Research Group, King Saud University（iWAN研究组，国王沙特大学）；is a Senior Scientist at Qatar Computing Research Institute（卡塔尔计算研究所）；is a Professor at King Saud University（国王沙特大学）；Head of iWAN Research Group（iWAN研究组）
Alignment at Pre-training! Towards Native Alignment for Arabic LLMs
Shenzhen Research Institue of Big Data（深圳大数据研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；Shenzhen International Center for Industrial and Applied Mathematics（深圳国际工业与应用数学中心）
Second Language (Arabic) Acquisition of LLMs via Progressive Vocabulary Expansion
King Abdullah University of Science and Technology（国王阿卜杜勒阿齐兹大学科学与技术大学）；Shenzhen International Center for Industrial and Applied Mathematics, Shenzhen Research Institute of Big Data（深圳国际工业与应用数学中心，深圳大数据研究院）；Shenzhen Research Institute of Big Data（深圳大数据研究院）；The Chinese University of Hong Kong（香港中文大学）；King Abdulaziz University（国王阿卜杜勒阿齐兹大学）；Prince Sattam bin Abdulaziz University（普林斯萨塔姆·本·阿卜杜勒阿齐兹大学）；University of Bisha（比沙大学）
!MSA at AraHealthQA 2025 Shared Task: Enhancing LLM Performance for Arabic Clinical Question Answering through Prompt Engineering and Ensemble Learning
Faculty of Computer Science, MSA University（计算机科学学院，MSA大学）
Arabic Morphosyntactic Tagging and Dependency Parsing with Large Language Models
Computational Approaches to Modeling Language Lab（语言建模方法计算实验室）；New York University Abu Dhabi（纽约大学阿布扎克分校）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
AceGPT, Localizing Large Language Models in Arabic
Shenzhen International Center for Industrial and Applied Mathematics（深圳国际工业与应用数学中心）；Shenzhen Research Institute of Big Data（深圳大数据研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）
Can Dialects Be Steered Like Languages? Sparse Neurons and Distributed Directions in Arabic LLMs
Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Qatar Computing Research Institute, Hamad Bin Khalifa University（哈马德·本·哈利法大学卡塔尔计算研究所）；Tohoku University（东北大学）；RIKEN（理化学研究所）
Morphemes Without Borders: Evaluating Root-Pattern Morphology in Arabic Tokenizers and LLMs
Saudi Data & AI Authority (SDAIA)（沙特数据与人工智能局）；Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)（穆罕默德·本·扎耶德人工智能大学）；Princess Nourah Bint Abdulrahman University (PNU)（普罗瑟·努拉·本·阿卜杜勒拉赫曼大学）
HalluScore: Large Language Model Hallucination Question Answering Benchmark
Department of Information and Computer Science, King Fahd University of Petroleum and Minerals（国王法赫德石油与矿物大学信息与计算机科学系）；SDAIA-KFUPM Joint Research Center for Artificial Intelligence（SDAIA-KFUPM人工智能联合研究中心）
AraReasoner: Evaluating Reasoning-Based LLMs for Arabic NLP
King Fahd University of Petroleum and Minerals（国王法赫德石油和矿物大学）；SDAIA–KFUPM Joint Research Center for Artificial Intelligence（SDAIA-KFUPM人工智能联合研究中心）；Mohammed VI Polytechnic University（穆莱·易卜拉欣(polytechnic)大学）
