Large language models (LLMs) are used globally, and because much of their training data is in English, they typically perform best on English inputs. As a result, many non-native English speakers interact with them in English as a second language (ESL), and these inputs often contain typographical errors. Prior work has largely studied the effects of ESL variation and typographical errors separately, even though they often co-occur in real-world use. In this study, we use the Trans-EnV framework to transform standard English inputs into eight ESL variants and apply MulTypo to inject typos at three levels: low, moderate, and severe. We find that combining ESL variation and typos generally leads to larger performance drops than either factor alone, though the combined effect is not simply additive. This pattern is clearest on closed-ended tasks, where performance degradation can be characterized more consistently across ESL variants and typo levels, while results on open-ended tasks are more mixed. Overall, these findings suggest that evaluations on clean standard English may overestimate real-world model performance, and that evaluating ESL variation and typographical errors in isolation does not fully capture model behavior in realistic settings.
Conditioned in Training, Dispensable at Inference: Typological Conditioning for Multilingual Speech Translation
School of Information Science and Electrical Engineering, Kyushu University(九州大学信息科学与电子工程学院);Recho Inc.(Recho公司);National Institute of Informatics(国家信息研究所);Interdisciplinary Research Centre on Security, Reliability and Trust (SnT), University of Luxembourg(卢森堡大学安全、可靠性与信任跨学科研究中心);Donghua University(东华大学);Department of Computer Science, The University of Tokyo(东京大学计算机科学系);Department of Electrical and Computer Engineering, University of Alberta(阿尔伯塔大学电子与计算机工程系)
How Can We Effectively Expand the Vocabulary of LLMs with 0.01GB of Target Language Text?
University of Sheffield, United Kingdom(英国谢菲尔德大学);University of Exeter, United Kingdom(英国埃克塞特大学);The Alan Turing Institute, United Kingdom(英国艾伦·图灵研究所);Federal University of Rio Grande do Norte, Brazil(巴西里奥格兰德杜纳伊联邦大学)
Multilingual Large Language Models do not comprehend all natural languages to equal degrees
Universitat Autònoma de Barcelona(巴塞罗那自治大学);Institució Catalana de Recerca i Estudis Avançats (ICREA)(加泰罗尼亚研究与高等教育机构);Radboud University Nijmegen(拉德堡德大学奈梅亨分校);Humboldt-Universität zu Berlin(柏林洪堡大学)
MasakhaPOS: Part-of-Speech Tagging for Typologically Diverse African Languages
Saarland University(萨尔大学);University of Pretoria(比勒陀利亚大学);Bayero University Kano(贝罗大学卡诺分校);Technical University of Munich(慕尼黑工业大学);Lanfrica;Lelapa AI;SADiLaR(南非语言资源中心);Malawi University of Business and Applied Science(马拉维商业与应用科学大学);Nnamdi Azikiwe University(纳姆迪·阿齐基韦大学);University of Buea(布埃亚大学);University of Ibadan(伊巴丹大学);Ewegbe Akademi(Ewegbe学院);AMALAN;University of Porto(波尔图大学)
Large Language Models for Mental Health: A Multilingual Evaluation
George Mason University(乔治·马歇尔大学);Interdisciplinary School of Doctoral Studies, University of Bucharest(布加勒斯特大学跨学科博士研究学院);PRHLT Research Center, Universitat Politècnica de València(瓦伦西亚理工大学PRHLT研究中心);University of Minnesota(明尼苏达大学)
IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models
Mila, McGill University(米拉实验室、麦吉尔大学);Lelapa AI(拉拉帕人工智能公司);Saarland University(萨尔大学);University of Toronto(多伦多大学);University College London(伦敦大学学院);Microsoft Research Africa(微软研究院非洲分部);Cohere For AI(科here人工智能);SADiLaR(南非数字语言资源中心);Ontario Tech University(安大略理工大学);Lancaster University(兰卡斯特大学);Princeton University(普林斯顿大学);Makerere University(马凯雷雷大学);Leibniz Universität Hannover(莱布尼茨汉诺威大学);Le CNAM(法国国立工艺学院);Imperial College London(伦敦帝国学院);Universidad de Deusto(德乌斯托大学);DAUST(达累斯萨拉姆科技大学);Haramaya University(哈拉马亚大学)
BIASEDTALES-ML: A Multilingual Dataset for Analyzing Narrative Attribute Distributions in LLM-Generated Stories
School of Computer Science and Engineering, Northeastern University, Shenyang, China(东北大学计算机科学与工程学院,中国沈阳);NiuTrans Research, Shenyang, China(NiuTrans研究院,中国沈阳)
Counting the Bugs in ChatGPT's Wugs: A Multilingual Investigation into the Morphological Capabilities of a Large Language Model
Carnegie Mellon University(卡内基梅隆大学);LMU Munich(慕尼黑大学);University of Oxford(牛津大学);Munich Center for Machine Learning(慕尼黑机器学习中心);Allen Institute for AI(艾伦人工智能研究所);IIT Delhi(德里印度理工学院)
No Text Needed: Forecasting MT Quality and Inequity from Fertility and Metadata
Institute for Disease Modeling(疾病模型研究所);Gates Foundation(盖茨基金会);University of San Francisco(旧金山大学);Mila, McGill University & Canada CIFAR AI Chair(Mila,麦吉尔大学及加拿大CIFAR人工智能主席)
Annotating Errors in English Learners' Written Language Production: Advancing Automated Written Feedback Systems
Tohoku University, Sendai, Japan(东京东京大学);RIKEN, Wako, Japan(日本理化学研究所);ALTA Institute, Computer Laboratory, University of Cambridge, U.K(剑桥大学计算机实验室ALTA研究所);INSA Lyon, Villeurbanne, France(里昂国立应用科学学院);CNRS, LIS, Aix-Marseille University, Marseille, France(国家科学研究中心(CNRS));MBZUAI, Abu Dhabi, United Arab Emirates(阿布扎赫德人工智能研究所)
语言模型(LM)已经被证明在通过完成完形填空题(如“Punta Cana is located in _.“然而,虽然知识是书面和查询在许多语言,研究的事实表征能力的LM几乎总是在英语进行。为了评估不同语言的LM中的事实知识检索,我们为23种类型学上不同的语言创建了一个多语言的完形填空式探针基准。为了正确处理语言的变化,我们扩展探测方法从单个到多个字的实体,并开发几个解码算法来生成多令牌的预测。广泛的实验结果提供了关于当前最先进的LM在具有或多或少可用资源的语言中执行此任务的情况如何的见解。我们进一步提出了一种基于代码转换的方法来提高多语言语言学习者获取知识的能力,并在几种基准语言上验证了其有效性。基准数据和代码已在https://x-factr.github.io上发布。
Exploring Adversarial Robustness and Safety Alignment in Multilingual Multi-Modal Large Language Models
Mohamed Bin Zayed University of AI, UAE(穆罕默德·本·扎耶德人工智能大学,阿联酋);Khalifa University, UAE(卡比拉大学,阿联酋);Australian National University, Australia(澳大利亚国立大学,澳大利亚)