As large language models (LLMs) have advanced rapidly, concerns regarding their safety have become prominent. In this paper, we discover that code-switching in red-teaming queries can effectively elicit undesirable behaviors of LLMs, which are common practices in natural language. We introduce a simple yet effective framework, CSRT, to synthesize codeswitching red-teaming queries and investigate the safety and multilingual understanding of LLMs comprehensively. Through extensive experiments with ten state-of-the-art LLMs and code-switching queries combining up to 10 languages, we demonstrate that the CSRT significantly outperforms existing multilingual red-teaming techniques, achieving 46.7% more attacks than standard attacks in English and being effective in conventional safety domains. We also examine the multilingual ability of those LLMs to generate and understand codeswitching texts. Additionally, we validate the extensibility of the CSRT by generating codeswitching attack prompts with monolingual data. We finally conduct detailed ablation studies exploring code-switching and propound unintended correlation between resource availability of languages and safety alignment in existing multilingual LLMs.
The Role of Mixed-Language Documents for Multilingual Large Language Model Pretraining
University College London（伦敦大学学院）；Nanyang Technological University（南洋理工大学）；University of Waterloo（滑铁卢大学）；NVIDIA（英伟达）；National Institute of Informatics（信息研究院）
AKCIT-FN at CheckThat! 2025: Switching Fine-Tuned SLMs and LLM Prompting for Multilingual Claim Normalization
Institute of Informatics, Federal University of Goiás, Brazil（信息学院，戈亚斯联邦大学，巴西）；Advanced Knowledge Center in Immersive Technology (AKCIT), Federal University of Goiás, Brazil（沉浸式技术高级知识中心（AKCIT），戈亚斯联邦大学，巴西）
Can Large Language Models Understand, Reason About, and Generate Code-Switched Text?
Capital One（第一资本）；Stanford University（斯坦福大学）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）；KAIST（韩国科学技术院）；Brown University（布朗大学）；NAIST（奈良先端科学技术大学院大学）；UC Berkeley（加州大学伯克利分校）；SAS Posos；Oracle（甲骨文公司）；ITB（万隆理工学院）；The Hong Kong Polytechnic University（香港理工大学）
Language models (LMs) have proven surprisingly successful at capturing factual knowledge by completing cloze-style fill-in-the-blank questions such as "Punta Cana is located in _." However, while knowledge is both written and queried in many languages, studies on LMs' factual representation ability have almost invariably been performed on English. To assess factual knowledge retrieval in LMs in different languages, we create a multilingual benchmark of cloze-style probes for 23 typologically diverse languages. To properly handle language variations, we expand probing methods from single- to mu…
Aurora-M: Open Source Continual Pre-training for Multilingual Language and Code
Babelscape（巴别景观公司）；LAION（LAION组织）；TU Darmstadt（达姆施塔特工业大学）；UNC Chapel-Hill（北卡罗来纳大学教堂山分校）；Detomo Inc.（Detomo公司）；CSIRO’s Data61（联邦科学与工业研究组织Data61）；Monash University（莫纳什大学）；Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；Carnegie Mellon University（卡内基梅隆大学）；Allen Institute for AI（艾伦人工智能研究所）；DeepTensor AB（DeepTensor AB公司）；WASP Media & Language（WASP媒体与语言机构）；Umeå University（于默奥大学）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；TU Wien（维也纳技术大学）；Institute of Science Tokyo（东京科学大学）；Hudson Labs（哈德逊实验室）；Luleå University of Technology（吕勒奥理工大学）；ETH Zurich（苏黎世联邦理工学院）；RunPod（RunPod公司）；OpenChat（OpenChat机构）；RIKEN CCS（理化学研究所计算科学中心）；Chattermill AI（Chattermill人工智能公司）；ASC27（ASC27机构）；University of Massachuset（马萨诸塞大学）
Tailored untruths: How personalisation challenges LLM safeguards
University of Sheffield（谢菲尔德大学）；University of Copenhagen（哥本哈根大学）；Big Data for Smart Society Institute (GATE)（大数据智能社会研究所（GATE））；University of São Paulo（圣保罗大学）
Pretrained language models are now of widespread use in Natural Language Processing. Despite their success, applying them to Low Resource languages is still a huge challenge. Although Multilingual models hold great promise, applying them to specific low-resource languages e.g. Roman Urdu can be excessive. In this paper, we show how the code-switching property of languages may be used to perform cross-lingual transfer learning from a corresponding high resource language. We also show how this transfer learning technique termed Bilingual Language Modeling can be used to produce better performing…
Information Sciences Institute（信息科学研究所）；University of Southern California（南加州大学）
Code-switching is a common phenomenon among multilingual speakers, where alternation between two or more languages occurs within the context of a single conversation. While multilingual humans can seamlessly switch back and forth between languages, multilingual neural machine translation (NMT) models are not robust to such sudden changes in input. This work explores multilingual NMT models' ability to handle code-switched text. First, we propose checks to measure switching capability. Second, we investigate simple and effective data augmentation methods that can enhance an NMT model's ability …
LTCI, Telecom Paris, Institut Polytechnique de Paris（巴黎综合理工学院电信巴黎分校LTCI）；IBM GBS France（IBM GBS法国）
Spoken dialog systems need to be able to handle both multiple languages and multilinguality inside a conversation (\textit{e.g} in case of code-switching). In this work, we introduce new pretraining losses tailored to learn multilingual spoken dialog representations. The goal of these losses is to expose the model to code-switched language. To scale up training, we automatically build a pretraining corpus composed of multilingual conversations in five different languages (French, Italian, English, German and Spanish) from \texttt{OpenSubtitles}, a huge multilingual corpus composed of 24.3G tok…
Multilingual neural machine translation models are trained to maximize the likelihood of a mix of examples drawn from multiple language pairs. The dominant inductive bias applied to these models is a shared vocabulary and a shared set of parameters across languages; the inputs and labels corresponding to examples drawn from different language pairs might still reside in distinct sub-spaces. In this paper, we introduce multilingual crossover encoder-decoder (mXEncDec) to fuse language pairs at an instance level. Our approach interpolates instances from different language pairs into joint `cross…
In this paper, we show that Multilingual BERT (M-BERT), released by Devlin et al. (2018) as a single language model pre-trained from monolingual corpora in 104 languages, is surprisingly good at zero-shot cross-lingual model transfer, in which task-specific annotations in one language are used to fine-tune the model for evaluation in another language. To understand why, we present a large number of probing experiments, showing that transfer is possible even to languages in different scripts, that transfer works best between typologically similar languages, that monolingual corpora can train mo…
Improving Code-Switching Speech Recognition with TTS Data Augmentation
Institute for Infocomm Research (I 2 R), A*STAR, Singapore（信息通信研究所（I 2 R），A*STAR，新加坡）；College of Computing and Data Science, Nanyang Technological University, Singapore（计算与数据科学学院，南洋理工大学，新加坡）
With increasing globalization and immigration, various studies have estimated that about half of the world population is bilingual. Consequently, individuals concurrently use two or more languages or dialects in casual conversational settings. However, most research is natural language processing is focused on monolingual text. To further the work in code-switched sentiment analysis, we propose a multi-step natural language processing algorithm utilizing points of code-switching in mixed text and conduct sentiment analysis around those identified points. The proposed sentiment analysis algorit…
Subword tokenization is a commonly used input pre-processing step in most recent NLP models. However, it limits the models' ability to leverage end-to-end task learning. Its frequency-based vocabulary creation compromises tokenization in low-resource languages, leading models to produce suboptimal representations. Additionally, the dependency on a fixed vocabulary limits the subword models' adaptability across languages and domains. In this work, we propose a vocabulary-free neural tokenizer by distilling segmentation information from heuristic-based subword tokenization. We pre-train our char…
Despite the great success of spoken language understanding (SLU) in high-resource languages, it remains challenging in low-resource languages mainly due to the lack of labeled training data. The recent multilingual code-switching approach achieves better alignments of model representations across languages by constructing a mixed-language context in zero-shot cross-lingual SLU. However, current code-switching methods are limited to implicit alignment and disregard the inherent semantic structure in SLU, i.e., the hierarchical inclusion of utterances, slots, and words. In this paper, we propose…
LLM Safety Alignment in Low-Resource Languages: A Systematic Literature Review
African Institute for Mathematical Sciences (AIMS)（非洲数学科学研究所）；Bayero University Kano（卡诺贝罗大学）；Brown University（布朗大学）；Centrum Wiskunde & Informatica（数学与计算机科学中心）；University of Pretoria（比勒陀利亚大学）；University of Hamburg（汉堡大学）；Imperial College London（伦敦帝国学院）
