The use of Project Gutenberg (PG) as a text corpus has been extremely popular in statistical analysis of language for more than 25 years. However, in contrast to other major linguistic datasets of similar importance, no consensual full version of PG exists to date. In fact, most PG studies so far either consider only a small number of manually selected books, leading to potential biased subsets, or employ vastly different pre-processing strategies (often specified in insufficient details), raising concerns regarding the reproducibility of published results. In order to address these shortcomings, here we present the Standardized Project Gutenberg Corpus (SPGC), an open science approach to a curated version of the complete PG data containing more than 50,000 books and more than $3 \times 10^9$ word-tokens. Using different sources of annotated metadata, we not only provide a broad characterization of the content of PG, but also show different examples highlighting the potential of SPGC for investigating language variability across time, subjects, and authors. We publish our methodology in detail, the code to download and process the data, as well as the obtained corpus itself on 3 different levels of granularity (raw text, timeseries of word tokens, and counts of words). In this way, we provide a reproducible, pre-processed, full-size version of Project Gutenberg as a new scientific resource for corpus linguistics, natural language processing, and information retrieval.
University of Hyderabad（海得拉巴大学）；Inria Paris Center（法国国家信息与自动化研究所巴黎中心）
The Digital Corpus of Sanskrit records around 650,000 sentences along with their morphological and lexical tagging. But inconsistencies in morphological analysis, and in providing crucial information like the segmented word, urges the need for standardization and validation of this corpus. Automating the validation process requires efficient analyzers which also provide the missing information. The Sanskrit Heritage Engine's Reader produces all possible segmentations with morphological and lexical analyses. Aligning these systems would help us in recording the linguistic differences, which can…
Freie Universität Berlin（柏林自由大学）；Center for Cognitive Neuroscience (CCNB), Freie Universität Berlin（柏林自由大学认知神经科学中心）；Learning Psychology, Freie Universität Berlin（柏林自由大学学习心理学系）
The Gutenberg Literary English Corpus (GLEC) provides a rich source of textual data for research in digital humanities, computational linguistics or neurocognitive poetics. However, so far only a small subcorpus, the Gutenberg English Poetry Corpus, has been submitted to quantitative text analyses providing predictions for scientific studies of literature. Here we show that in the entire GLEC quasi error-free text classification and authorship recognition is possible with a method using the same set of five style and five content features, computed via style and sentiment analysis, in both tas…
The Gutenberg Literary English Corpus (GLEC, Jacobs, 2018a) provides a rich source of textual data for research in digital humanities, computational linguistics or neurocognitive poetics. In this study we address differences among the different literature categories in GLEC, as well as differences between authors. We report the results of three studies providing i) topic and sentiment analyses for six text categories of GLEC (i.e., children and youth, essays, novels, plays, poems, stories) and its >100 authors, ii) novel measures of semantic complexity as indices of the literariness, creativit…
UniversalCEFR: Enabling Open Multilingual Research on Language Proficiency Assessment
University of Bath（巴斯大学）；Cardiff University（卡迪夫大学）；National University Philippines（菲律宾国家大学）；Bielefeld University（比勒菲尔德大学）；University of Exeter（埃克塞特大学）；University of Gothenburg（哥德堡大学）；UCLouvain（列日大学）；MBZUAI（马克斯·普朗克智能系统研究所）；Brigham Young University（Brigham Young 大学）；INESC-ID Lisboa（里斯本 INESC-ID）；Instituto Universitário de Lisboa (ISCTE-IUL)（里斯本大学学院（ISCTE-IUL））；Universitat Pompeu Fabra（庞培法华大学）；National Research Council, Canada（加拿大国家研究委员会）；King Abdulaziz University（国王阿卜杜勒-阿齐兹大学）
We present the results of a study of definite descriptions use in written texts aimed at assessing the feasibility of annotating corpora with information about definite description interpretation. We ran two experiments, in which subjects were asked to classify the uses of definite descriptions in a corpus of 33 newspaper articles, containing a total of 1412 definite descriptions. We measured the agreement among annotators about the classes assigned to definite descriptions, as well as the agreement about the antecedent assigned to those definites that the annotators classified as being relate…
SalamahBench: Dialect and Category Level Safety Evaluation of Arabic Language Models
Compumacy for Artificial Intelligence Solutions（Compumacy人工智能解决方案公司）；Taibah University（泰巴大学）；George Mason University（乔治梅森大学）；Queen’s University Belfast（贝尔法斯特女王大学）
Modern Models, Medieval Texts: A POS Tagging Study of Old Occitan
Bavarian Academy of Sciences（巴伐利亚科学院）；LMU Munich（慕尼黑大学）；Albert-Ludwigs-Universität Freiburg（弗莱堡大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）
TACOMORE: Exploring a replicable prompting protocol for LLM-assisted corpus analysis
Department of Linguistics and Communication, University of Birmingham（伯明翰大学语言学与传播系）；Department of Information Engineering and Computer Science, University of Trento（特伦托大学信息工程与计算机科学系）；Institute of Foreign Languages and Cultures, University of Tartu（塔尔图大学外国语言与文化研究所）
Stop Taking Tokenizers for Granted: They Are Core Design Decisions in Large Language Models
Princess Nourah Bint Abdulrahman University（普里丝琴·努拉·本·阿卜杜勒拉赫曼大学）；Saudi Data & AI Authority (SDAIA)（沙特数据与人工智能管理局）；University of Alberta（阿尔伯塔大学）；Dialpad（Dialpad公司）；York University（约克大学）；Qatar Computing Research Institute（卡塔尔计算研究所）；Amazon AGI（亚马逊人工智能研究所）
SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages
AI Singapore（新加坡人工智能中心）；IndoNLP（印尼自然语言处理研究组）；Universitas Indonesia（印度尼西亚大学）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）；RevComm, Inc.（RevComm公司）；Tohoku University（东北大学）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）；University of Bath（巴斯大学）；National University Philippines（菲律宾国立大学）；MOH Office for Healthcare Transformation (MOHT)（卫生部医疗转型办公室）；NAIST（奈良科学技术研究所）；Works Applications Lab（Works Applications实验室）；Universitas Gadjah Mada（加查马达大学）；Institut Teknologi Bandung（万隆理工学院）；Beijing Academy of Artificial Intelligence (BAAI)（北京人工智能研究院）；Institute for Infocomm Research, A*STAR（新加坡资讯通信研究院（新加坡科技研究局下属））；Samsung Research Philippines（三星菲律宾研究院）；University of Cambridge（剑桥大学）；Queensland University of Technology（昆士兰科技大学）
Technische Universität Darmstadt（达姆施塔特工业大学）；University of Edinburgh（爱丁堡大学）；New York University（纽约大学）
Despite an ever growing number of word representation models introduced for a large number of languages, there is a lack of a standardized technique to provide insights into what is captured by these models. Such insights would help the community to get an estimate of the downstream task performance, as well as to design more informed neural architectures, while avoiding extensive experimentation which requires substantial computational resources not all researchers have access to. A recent development in NLP is to use simple classification tasks, also called probing tasks, that test for a sin…
IndicMMLU-Pro: Benchmarking Indic Large Language Models on Multi-Task Language Understanding
Artificial Intelligence Institute, University of South Carolina（南卡罗来纳大学人工智能学院）；Rochester Institute of Technology（罗切斯特理工学院）；Meta AI；Amazon Gen AI（亚马逊生成式人工智能部门）
From Isolates to Families: Using Neural Networks for Automated Language Affiliation
Department of Linguistic and Cultural Evolution, Max-Planck Institute for Evolutionary Anthropology（语言与文化进化部门，马克斯·普朗克进化人类学研究所）；University of Passau, Chair of Multilingual Computational Linguistics（帕绍大学，多语计算语言学教授团）；University of Passau, Chair of AI Engineering（帕绍大学，人工智能工程教授团）
