Large Language Models (LLMs) are increasingly deployed as agents that invoke external tools through structured function calls. While recent work reports strong tool-calling performance under standard English-centric evaluations, the robustness of tool calling under multilingual user interactions remains underexplored. In this work, we introduce MLCL, a diagnostic benchmark, and conduct a systematic evaluation of multilingual tool calling across Chinese, Hindi, and the low-resource language Igbo. Through fine-grained error analysis, we show that many failures occur despite correct intent understanding and tool selection. We identify parameter value language mismatch as a dominant failure mode, where models generate semantically appropriate parameter values in the user's language, violating language-invariant execution conventions. We further evaluate several inference-time system strategies and find that while these strategies substantially reduce language-induced execution errors, none of them can fully recover English-level performance.
Lost in Reordering: Structural Sensitivity of Multilingual LLMs under Semantics-Preserving Perturbations
Indian Institute of Technology Bombay（印度理工学院孟买分校）；Indian Institute of Technology Mandi（印度理工学院曼迪分校）；Indian Institute of Technology Hyderabad（印度理工学院海得拉巴分校）
The Reasoning Lingua Franca: A Double-Edged Sword for Multilingual AI
Nilekani Centre at AI4Bharat（AI4Bharat人工智能中心）；Indian Institute of Technology Madras（印度理工学院马德拉斯分校）；Google（谷歌）；Microsoft（微软）；IT University of Copenhagen（丹麦技术大学）
Lost in Multilinguality: Dissecting Cross-lingual Factual Inconsistency in Transformer Language Models
Bosch Center for Artificial Intelligence（博世人工智能中心）；LMU Munich（慕尼黑大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Hochschule der Medien（斯图加特媒体大学）；Karlsruhe University of Applied Sciences（卡尔斯鲁厄应用技术大学）
The appearance of complex attention-based language models such as BERT, Roberta or GPT-3 has allowed to address highly complex tasks in a plethora of scenarios. However, when applied to specific domains, these models encounter considerable difficulties. This is the case of Social Networks such as Twitter, an ever-changing stream of information written with informal and complex language, where each message requires careful evaluation to be understood even by humans given the important role that context plays. Addressing tasks in this domain through Natural Language Processing involves severe ch…
Lost in translation: using global fact-checks to measure multilingual misinformation prevalence, spread, and evolution
Department of Mathematical Modeling and Machine Learning, University of Zurich, Switzerland（苏黎世大学数学建模与机器学习系）；Digital Society Initiative, University of Zurich, Switzerland（苏黎世大学数字社会倡议）；Oxford Internet Institute, University of Oxford, United Kingdom（牛津大学网络研究所）；Meedan, San Francisco, United States（梅丹公司，旧金山，美国）
Technical University of Darmstadt（达姆施塔特工业大学）；New York University（纽约大学）；University of Cambridge（剑桥大学）
Recent advances in multimodal vision and language modeling have predominantly focused on the English language, mostly due to the lack of multilingual multimodal datasets to steer modeling efforts. In this work, we address this gap and provide xGQA, a new multilingual evaluation benchmark for the visual question answering task. We extend the established English GQA dataset to 7 typologically diverse languages, enabling us to detect and explore crucial challenges in cross-lingual visual question answering. We further propose new adapter-based approaches to adapt multimodal transformer-based mode…
The University of Manchester（曼彻斯特大学）；Logrus Global
Pre-trained language models (PLMs) often take advantage of the monolingual and multilingual dataset that is freely available online to acquire general or mixed domain knowledge before deployment into specific tasks. Extra-large PLMs (xLPLMs) are proposed very recently to claim supreme performances over smaller-sized PLMs such as in machine translation (MT) tasks. These xLPLMs include Meta-AI's wmt21-dense-24-wide-en-X (2021) and NLLB (2022). In this work, we examine if xLPLMs are absolutely superior to smaller-sized PLMs in fine-tuning toward domain-specific MTs. We use two different in-domain…
Fair Fact-Checking: Closing the Cross-Lingual Gap in LLM Factual Judgement with RoSh
Shifa Tameer-e-Millat University（希法塔米尔米拉特大学）；BRAINS, Brandenburg Research Center for Applied Intelligent Systems（勃兰登堡应用智能系统研究中心）；Max Planck Institute for Security and Privacy（马克斯·普朗克安全与隐私研究所）；University of Cambridge（剑桥大学）；The Pennsylvania State University（宾夕法尼亚州立大学）；GISMA University of Applied Sciences（GISMA应用科学大学）
Socially Responsible Data for Large Multilingual Language Models
Google Research（谷歌研究院）；Namibia University of Science & Technology（纳米比亚科技大学）；New York University（纽约大学）；Navanti（纳万蒂公司）；University of California, Berkeley（加州大学伯克利分校）；University of Florida（佛罗里达大学）
