We study the effect of one type of imbalance often present in real-life multilingual classification datasets: an uneven distribution of labels across languages. We show evidence that fine-tuning a transformer-based Large Language Model (LLM) on a dataset with this imbalance leads to worse performance, a more pronounced separation of languages in the latent space, and the promotion of uninformative features. We modify the traditional class weighing approach to imbalance by calculating class weights separately for each language and show that this helps mitigate those detrimental effects. These results create awareness of the negative effects of language-specific class imbalance in multilingual fine-tuning and the way in which the model learns to rely on the separation of languages to perform the task.
Language Imbalance Driven Rewarding for Multilingual Self-improving
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Wuhan AI Research(武汉人工智能研究院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
Multilingual Pretraining and Instruction Tuning Improve Cross-Lingual Knowledge Alignment, But Only Shallowly
National Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室);Department of Linguistics and Translation, City University of Hong Kong(香港城市大学语言学及翻译学系)
One Jailbreak, Many Tongues: Learning Language-Insensitive Intention Representations for Multilingual Jailbreak Detection
School of Cyber Science and Engineering, Sichuan University(四川大学网络空间安全学院);School of Computer Science and Engineering, Nanyang Technological University(南洋理工大学计算机科学与工程学院);School of Computer Science and Engineering, University of Electronic Science and Technology of China(电子科技大学计算机科学与工程学院)
UltraLink: An Open-Source Knowledge-Enhanced Multilingual Supervised Fine-tuning Dataset
Tsinghua University(清华大学);Beijing Language and Culture University(北京语言大学);Northeastern University(东北大学);Beijing University of Posts and Telecommunications(北京邮电大学)
MultiLingPoT: Enhancing Mathematical Reasoning with Multilingual Program Fine-tuning
Fudan University(复旦大学);MYbank, Ant Group(蚂蚁集团MYbank);School of Computer Science, Fudan University(复旦大学计算机学院);School of Data Science, Fudan University(复旦大学数据科学学院)
Multi-Agent Collaboration for Multilingual Code Instruction Tuning
Alibaba Group(阿里巴巴集团);Shanghai Jiao Tong University(上海交通大学);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University of Chinese Academy of Sciences(中国科学院大学)
NCL-BU at SemEval-2026 Task 3: Fine-tuning XLM-RoBERTa for Multilingual Dimensional Sentiment Regression
Independent Researcher(独立研究者);Centre for Applied Creative Technologies (CFACT+), Bournemouth University, UK(应用创意技术中心(CFACT+),伯恩茅斯大学,英国);School of Computing, Newcastle University, Newcastle upon Tyne, UK(计算学院,新castle大学,新castle upon Tyne,英国)
No Optimal Language Set Exists for Multilingual Instruction Tuning: Insights from a Linguistically-Informed Study
Wageningen University and Research(瓦赫宁根大学与研究中心);Friedrich-Alexander-Universität Erlangen-Nürnberg(埃尔朗根-纽伦堡弗里德里希-亚历山大大学);Koç University(科克大学);KUIS AI Lab(KUIS人工智能实验室)
Is Translation All You Need? A Study on Solving Multilingual Tasks with Large Language Models
Nanyang Technological University(南洋理工大学);DAMO Academy, Alibaba Group(阿里达摩院);Hupan Lab(虎斑实验室);National University of Singapore(新加坡国立大学);Shanda AI Research Institute(沙达AI研究院)
Towards Better Understanding of Program-of-Thought Reasoning in Cross-Lingual and Multilingual Environments
School of Information Science and Technology, VISTEC(信息科学与技术学院,VISTEC);KAIST(韩国科学技术院);Cohere;SCB 10X;AI Singapore(AI新加坡);Department of Computer Engineering, Chulalongkorn University(朱拉隆梭大学计算机工程系)
Language on Demand, Knowledge at Core: Composing LLMs with Encoder-Decoder Translation Models for Extensible Multilinguality
Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences(智能信息处理重点实验室,计算技术研究所,中国科学院);State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences(人工智能安全国家重点实验室,计算技术研究所,中国科学院);University of Chinese Academy of Sciences, Beijing, China(中国科学院大学,北京,中国)
Towards Understanding the Fragility of Multilingual LLMs against Fine-Tuning Attacks
Meta;University of Pisa(比萨大学);University of Modena and Reggio Emilia(摩德纳与雷焦艾米利亚大学);Brown University(布朗大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
Investigating the Multilingual Calibration Effects of Language Model Instruction-Tuning
Mila - Quebec AI Institute(魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);The University of Tokyo(东京大学);Western University(西方大学);Vector Institute(向量研究所);Polytechnique Montréal(蒙特利尔理工学院);CIFAR AI Chair(CIFAR人工智能主席);AIST(日本产业技术综合研究所)
VFA: Empowering Multilingual MLLMs via Vision-Free Adaptation
Southern University of Science and Technology(南方科技大学);Microsoft Research Asia(微软亚洲研究院);Shanghai University of Finance and Economics(上海财经大学);Peking University(北京大学)
Exploring Adversarial Robustness and Safety Alignment in Multilingual Multi-Modal Large Language Models
Mohamed Bin Zayed University of AI, UAE(穆罕默德·本·扎耶德人工智能大学,阿联酋);Khalifa University, UAE(卡比拉大学,阿联酋);Australian National University, Australia(澳大利亚国立大学,澳大利亚)
From Unaligned to Aligned: Scaling Multilingual LLMs with Multi-Way Parallel Corpora
Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系);Technical University of Munich(慕尼黑技术大学);Munich Center for Machine Learning(慕尼黑机器学习中心);Minzu University of China(民族大学);Institute for AI, Tsinghua University(清华大学人工智能研究院)