维基百科文章（内容页面）是自然语言处理（NLP）研究中常用的语料库，尤其是在英语以外的低资源语言中。然而，少数研究已经研究了三个阿拉伯语维基百科版本——阿拉伯语维基百科（AR）、埃及阿拉伯语维基百科（ARZ）和摩洛哥阿拉伯语维基百科（ARY），并记录了埃及阿拉伯语维基百科版本中存在的问题，即其文章通过基于模板的翻译从英语到阿拉伯语进行大规模自动创建，且没有人工参与，从而使埃及阿拉伯语维基百科充斥着不仅内容质量低下、而且不代表埃及人民、其文化和其方言的文章。在本文中，我们旨在通过探索性分析识别这些模板翻译的文章及其特征，并构建自动检测系统，从而缓解埃及阿拉伯语维基百科中出现的模板翻译问题。我们首先从密度、质量和人工贡献方面探索三个阿拉伯语维基百科版本的内容，并利用所得见解构建多元机器学习分类器，利用文章元数据自动检测模板翻译的文章。然后，我们公开部署并托管性能最佳的分类器XGBoost，作为一个名为EGYPTIAN WIKIPEDIA SCANNER的在线应用，并将提取、过滤和标注的数据集发布给研究社区，以便从我们的数据集和基于网络的在线检测系统中受益。
英文摘要：
Wikipedia articles (content pages) are commonly used corpora in Natural Language Processing (NLP) research, especially in low-resource languages other than English. Yet, a few research studies have studied the three Arabic Wikipedia editions, Arabic Wikipedia (AR), Egyptian Arabic Wikipedia (ARZ), and Moroccan Arabic Wikipedia (ARY), and documented issues in the Egyptian Arabic Wikipedia edition regarding the massive automatic creation of its articles using template-based translation from English to Arabic without human involvement, overwhelming the Egyptian Arabic Wikipedia with articles that do not only have low-quality content but also with articles that do not represent the Egyptian people, their culture, and their dialect. In this paper, we aim to mitigate the problem of template translation that occurred in the Egyptian Arabic Wikipedia by identifying these template-translated articles and their characteristics through exploratory analysis and building automatic detection systems. We first explore the content of the three Arabic Wikipedia editions in terms of density, quality, and human contributions and utilize the resulting insights to build multivariate machine learning classifiers leveraging articles' metadata to detect the template-translated articles automatically. We then publicly deploy and host the best-performing classifier, XGBoost, as an online application called EGYPTIAN WIKIPEDIA SCANNER and release the extracted, filtered, and labeled datasets to the research community to benefit from our datasets and the online, web-based detection system.
Fayoum University（法尤姆大学）；Nahdat Misr AI（埃及复兴人工智能公司）；American University in Cairo（开罗美国大学）
The largest dataset of Arabic speech mispronunciation detections in Egyptian dialogues is introduced. The dataset is composed of annotated audio files representing the top 100 words that are most frequently used in the Arabic language, pronounced by 100 Egyptian children (aged between 2 and 8 years old). The dataset is collected and annotated on segmental pronunciation error detections by expert listeners.
New York University Abu Dhabi（纽约大学阿布扎比分校）；University of Bahrain（巴林大学）
Most Arabic natural language processing tools and resources are developed to serve Modern Standard Arabic (MSA), which is the official written language in the Arab World. Some Dialectal Arabic varieties, notably Egyptian Arabic, have received some attention lately and have a growing collection of resources that include annotated corpora and morphological analyzers and taggers. Gulf Arabic, however, lags behind in that respect. In this paper, we present the Gumar Corpus, a large-scale corpus of Gulf Arabic consisting of 110 million words from 1,200 forum novels. We annotate the corpus for sub-d…
New York University Abu Dhabi（纽约大学阿布扎比分校）；University of Stuttgart（斯图加特大学）；The German University in Cairo（开罗德国大学）
We present our work on collecting ArzEn-ST, a code-switched Egyptian Arabic - English Speech Translation Corpus. This corpus is an extension of the ArzEn speech corpus, which was collected through informal interviews with bilingual speakers. In this work, we collect translations in both directions, monolingual Egyptian Arabic and monolingual English, forming a three-way speech translation corpus. We make the translation guidelines and corpus publicly available. We also report results for baseline systems for machine translation and speech translation tasks. We believe this is a valuable resour…
Qatar Computing Research Institute（卡塔尔计算研究所）；Columbia University（哥伦比亚大学）；New York University Abu Dhabi（纽约大学阿布扎比分校）
The paper describes the Egyptian Arabic-to-English statistical machine translation (SMT) system that the QCRI-Columbia-NYUAD (QCN) group submitted to the NIST OpenMT'2015 competition. The competition focused on informal dialectal Arabic, as used in SMS, chat, and speech. Thus, our efforts focused on processing and standardizing Arabic, e.g., using tools such as 3arrib and MADAMIRA. We further trained a phrase-based SMT system using state-of-the-art features and components such as operation sequence model, class-based language model, sparse features, neural network joint model, genre-based hier…
Cairo University（开罗大学）；Institute of Statistical Studies and Research（统计研究与研究所）
Labelling of user's utterances to understanding his attends which called Dialogue Act (DA) classification, it is considered the key player for dialogue language understanding layer in automatic dialogue systems. In this paper, we proposed a novel approach to user's utterances labeling for Egyptian spontaneous dialogues and Instant Messages using Machine Learning (ML) approach without relying on any special lexicons, cues, or rules. Due to the lack of Egyptian dialect dialogue corpus, the system evaluated by multi-genre corpus includes 4725 utterances for three domains, which are collected and …
Northwest University（西北大学）；University of Essex（埃塞克斯大学）
Arabic sentiment analysis has become an important research field in recent years. Initially, work focused on Modern Standard Arabic (MSA), which is the most widely-used form. Since then, work has been carried out on several different dialects, including Egyptian, Levantine and Moroccan. Moreover, a number of datasets have been created to support such work. However, up until now, less work has been carried out on Sudanese Arabic, a dialect which has 32 million speakers. In this paper, two new publicly available datasets are introduced, the 2-Class Sudanese Sentiment Dataset (SudSenti2) and the …
The German University in Cairo（开罗德国大学）；University of Stuttgart（斯图加特大学）；Raisa Energy LLC（赖萨能源有限责任公司）
Code-switching (CS) is a widespread phenomenon among bilingual and multilingual societies. The lack of CS resources hinders the performance of many NLP tasks. In this work, we explore the potential use of bilingual word embeddings for code-switching (CS) language modeling (LM) in the low resource Egyptian Arabic-English language. We evaluate different state-of-the-art bilingual word embeddings approaches that require cross-lingual resources at different levels and propose an innovative but simple approach that jointly learns bilingual word representations without the use of any parallel data, …
Institute of Statistical Studies and Research (ISSR), Cairo University（开罗大学统计研究与研究所）；Faculty of Computers and Information, Cairo University（开罗大学计算机与信息学院）
Text segmentation task is an essential processing task for many of Natural Language Processing (NLP) such as text summarization, text translation, dialogue language understanding, among others. Turns segmentation considered the key player in dialogue understanding task for building automatic Human-Computer systems. In this paper, we introduce a novel approach to turn segmentation into utterances for Egyptian spontaneous dialogues and Instance Messages (IM) using Machine Learning (ML) approach as a part of automatic understanding Egyptian spontaneous dialogues and IM task. Due to the lack of Eg…
SalamahBench: Dialect and Category Level Safety Evaluation of Arabic Language Models
Compumacy for Artificial Intelligence Solutions（Compumacy人工智能解决方案公司）；Taibah University（泰巴大学）；George Mason University（乔治梅森大学）；Queen’s University Belfast（贝尔法斯特女王大学）
This paper proposes a methodology to prepare corpora in Arabic language from online social network (OSN) and review site for Sentiment Analysis (SA) task. The paper also proposes a methodology for generating a stopword list from the prepared corpora. The aim of the paper is to investigate the effect of removing stopwords on the SA task. The problem is that the stopwords lists generated before were on Modern Standard Arabic (MSA) which is not the common language used in OSN. We have generated a stopword list of Egyptian dialect and a corpus-based list to be used with the OSN corpora. We compare…
ArAIEval Shared Task: Propagandistic Techniques Detection in Unimodal and Multimodal Arabic Content
Qatar Computing Research Institute（卡塔尔计算研究所）；HBKU（哈迈德·本·哈利法大学）；University of New Brunswick（新不伦瑞克大学）；Hamad Bin Khalifa University（哈迈德·本·哈利法大学）；Northwestern University in Qatar（西北大学卡塔尔分校）
Recent progress in neural machine translation (NMT) has made it possible to translate successfully between monolingual language pairs where large parallel data exist, with pre-trained models improving performance even further. Although there exists work on translating in code-mixed settings (where one of the pairs includes text from two or more languages), it is still unclear what recent success in NMT and language modeling exactly means for translating code-mixed text. We investigate one such context, namely MT from code-mixed Modern Standard Arabic and Egyptian Arabic (MSAEA) into English. W…
MIT（麻省理工学院）；Computer Science & Artificial Intelligence Laboratory（计算机科学与人工智能实验室）；HBKU（卡塔尔哈马德·本·哈利法大学）；Qatar Computing Research Institute（卡塔尔计算研究所）
We present the speech to text transcription system, called DARTS, for low resource Egyptian Arabic dialect. We analyze the following; transfer learning from high resource broadcast domain to low-resource dialectal domain and semi-supervised learning where we use in-domain unlabeled audio data collected from YouTube. Key features of our system are: A deep neural network acoustic model that consists of a front end Convolutional Neural Network (CNN) followed by several layers of Time Delayed Neural Network (TDNN) and Long-Short Term Memory Recurrent Neural Network (LSTM); sequence discriminative …
University of Stuttgart（斯图加特大学）；Ulm University（乌尔姆大学）；German International University（德国国际大学）
Code-switching (CS) is a common linguistic phenomenon exhibited by multilingual individuals, where they tend to alternate between languages within one single conversation. CS is a complex phenomenon that not only encompasses linguistic challenges, but also contains a great deal of complexity in terms of its dynamic behaviour across speakers. Given that the factors giving rise to CS vary from one country to the other, as well as from one person to the other, CS is found to be a speaker-dependant behaviour, where the frequency by which the foreign language is embedded differs across speakers. Wh…
NADI 2026: The Second Multidialectal Arabic Speech Processing Shared Task
The University of British Columbia（不列颠哥伦比亚大学）；King Fahd University of Petroleum & Minerals（法赫德国王石油与矿业大学）；Avignon Université（阿维尼翁大学）；ELYADATA（ELYADATA公司）；King Abdullah University of Science & Technology（阿卜杜拉国王科技大学）；NYU Abu Dhabi（纽约大学阿布扎比分校）
Telecom and Embedded Systems Team, SIME Lab ENSIAS（电信与嵌入式系统团队，SIME实验室ENSIAS）；University of Mohammed V（穆罕默德五世大学）；University of Mohammed V Souissi（穆罕默德五世苏伊西大学）
In this paper, we present a new approach dedicated to correcting the spelling errors of the Arabic language. This approach corrects typographical errors like inserting, deleting, and permutation. Our method is inspired from the Levenshtein algorithm, and allows a finer and better scheduling than Levenshtein. The results obtained are very satisfactory and encouraging, which shows the interest of our new approach.
ZAEBUC-Spoken: A Multilingual Multidialectal Arabic-English Speech Corpus
New York University Abu Dhabi（纽约大学阿布扎比分校）；University of Stuttgart（斯图加特大学）；University of Tübingen（蒂宾根大学）；United Arab Emirates University（阿联酋大学）；Zayed University（扎耶德大学）
The George Washington University（乔治华盛顿大学）；AWS, Amazon AI（亚马逊云科技亚马逊人工智能）
Linguistic Code Switching (CS) is a phenomenon that occurs when multilingual speakers alternate between two or more languages/dialects within a single conversation. Processing CS data is especially challenging in intra-sentential data given state-of-the-art monolingual NLP technologies since such technologies are geared toward the processing of one language at a time. In this paper, we address the problem of Part-of-Speech tagging (POS) in the context of linguistic code switching (CS). We explore leveraging multiple neural network architectures to measure the impact of different pre-trained em…
Computer Science and Engineering Department, Qatar University（卡塔尔大学计算机科学与工程系）；Qatar Computing Research Institute, HBKU（哈马德·本·哈利法大学卡塔尔计算研究所）；DIT, Università di Bologna（博洛尼亚大学DIT（此处DIT按原文保留，若需全称需更多上下文））；Department of Computer Science, University of Copenhagen（哥本哈根大学计算机科学系）
We present an overview of the second edition of the CheckThat! Lab at CLEF 2019. The lab featured two tasks in two different languages: English and Arabic. Task 1 (English) challenged the participating systems to predict which claims in a political debate or speech should be prioritized for fact-checking. Task 2 (Arabic) asked to (A) rank a given set of Web pages with respect to a check-worthy claim based on their usefulness for fact-checking that claim, (B) classify these same Web pages according to their degree of usefulness for fact-checking the target claim, (C) identify useful passages fr…
Mind the Gap: Exposing LLM Translation Blind Spots Using the AlphaMWE Multilingual Parallel Corpus
Leiden University（莱顿大学）；Leiden University Medical Centre（莱顿大学医学中心）；Institute of Computer Science PAS, Warsaw（波兰科学院计算机科学研究所）；University of Ljubljana（卢布尔雅那大学）；University of Granada（格拉纳达大学）；Technical University of Munich（慕尼黑工业大学）；Heinrich Heine University Düsseldorf（杜塞尔多夫海因里希·海涅大学）；Google DeepMind（谷歌DeepMind）
