大型语言模型（LLMs）在复杂推理任务中表现出色。然而，尽管它们在高资源语言（如英语和中文）中具有强大的推理能力，但在其他语言中仍存在显著的性能差距。为了研究韩语中的这一差距，我们引入了HRM8K，这是一个包含8,011道英韩平行双语数学问题的基准。通过对模型行为的系统分析，我们得出了一个关键发现：这些性能差异主要源于对非英语输入的理解困难，而非推理能力的限制。基于这些发现，我们提出了UST（Understand, Solve, and Translate，理解、求解与翻译）方法，该方法战略性地将英语作为推理和生成解决方案的锚点。通过在130k个合成生成的数据点上对模型进行微调，UST在HRM8K基准上实现了10.91%的提升，并将多语言性能差距从11.6%缩小至0.7%。此外，我们表明UST的改进能有效地泛化到不同的韩语领域，这证明从机器可验证内容中获取的能力可以泛化到其他领域。我们公开发布了该基准、训练数据集和模型。
英文摘要：
Large language models (LLMs) demonstrate exceptional performance on complex reasoning tasks. However, despite their strong reasoning capabilities in high-resource languages (e.g., English and Chinese), a significant performance gap persists in other languages. To investigate this gap in Korean, we introduce HRM8K, a benchmark comprising 8,011 English-Korean parallel bilingual math problems. Through systematic analysis of model behaviors, we identify a key finding: these performance disparities stem primarily from difficulties in comprehending non-English inputs, rather than limitations in reasoning capabilities. Based on these findings, we propose UST (Understand, Solve, and Translate), a method that strategically uses English as an anchor for reasoning and solution generation. By fine-tuning the model on 130k synthetically generated data points, UST achieves a 10.91% improvement on the HRM8K benchmark and reduces the multilingual performance gap from 11.6% to 0.7%. Additionally, we show that improvements from UST generalize effectively to different Korean domains, demonstrating that capabilities acquired from machine-verifiable content can be generalized to other areas. We publicly release the benchmark, training dataset, and models.
Knowledge Beyond Language: Bridging the Gap in Multilingual Machine Unlearning Evaluation
GSCST, Seoul National University（首尔国立大学GSCST）；AIIS, Seoul National University（首尔国立大学AIIS）；Department of Artificial Intelligence, Chung-Ang University（Chung-Ang大学人工智能系）；Korean Surgical Researcher Foundation, Republic of Korea（韩国外科研究员基金会）
Multilingual Language Model Pretraining using Machine-translated Data
Centre for Artificial Intelligence, University College London（伦敦大学学院人工智能中心）；Together AI；Mila, McGill University（米拉-麦吉尔大学）；Research and Development Center for Large Language Models, National Institute of Informatics（国立信息学研究所大语言模型研发中心）
Copy First, Translate Later: Interpreting Translation Dynamics in Multilingual Pretraining
MaiNLP, Center for Information and Language Processing, LMU Munich（MaiNLP，信息与语言处理中心，慕尼黑大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Department of Mathematics, LMU Munich（数学系，慕尼黑大学）；Department of Physics and Technology, University of Tromsø（物理与技术系，特罗姆瑟大学）；DLR-German Aerospace Center（德国航空航天中心）
Learn Globally, Speak Locally: Bridging the Gaps in Multilingual Reasoning
Massachusetts Institute of Technology（麻省理工学院）；Havard Univeristy（哈佛大学）；LG CNS；Université de Montréal（蒙特利尔大学）；Mila（米拉研究所）；Google（谷歌）；Stanford University（斯坦福大学）
Ready to Translate, Not to Represent? Bias and Performance Gaps in Multilingual LLMs Across Language Families and Domains
United International University（国际联合大学）；Qatar Computing Research Institute（卡塔尔计算研究所）；Amazon GenAI（亚马逊生成人工智能）；University of Virginia（弗吉尼亚大学）；BRAC University（布拉克大学）
Tower+: Bridging Generality and Translation Specialization in Multilingual LLMs
Unbabel；Instituto de Telecomunicações（电信研究所）；Instituto Superior Técnico & Universidade de Lisboa (Lisbon ELLIS Unit)（里斯本大学技术学院及里斯本大学（里斯本ELLIS单位））；MICS, CentraleSupélec, Université Paris-Saclay（MICS，CentraleSupélec，巴黎萨克雷大学）
KBioXLM: A Knowledge-anchored Biomedical Multilingual Pretrained Language Model
Institute of Artificial Intelligence, Soochow University（苏州大学人工智能研究院）；Peking University（北京大学）；The Hong Kong Polytechnic University（香港理工大学）；Pharmcube（药智魔方）；Changping Laboratory（昌平实验室）
Indian Institute of Technology Bombay（印度理工学院孟买分校）；Google Research India（谷歌印度研究院）
Despite cross-lingual generalization demonstrated by pre-trained multilingual models, the translate-train paradigm of transferring English datasets across multiple languages remains to be a key mechanism for training task-specific multilingual models. However, for many low-resource languages, the availability of a reliable translation service entails significant amounts of costly human-annotated translation pairs. Further, translation services may continue to be brittle due to domain mismatch between task-specific input text and general-purpose text used for training translation models. For mu…
Center for Information and Language Processing (CIS), LMU Munich（慕尼黑大学信息与语言处理中心（CIS））
Recently, it has been found that monolingual English language models can be used as knowledge bases. Instead of structural knowledge base queries, masked sentences such as "Paris is the capital of [MASK]" are used as probes. We translate the established benchmarks TREx and GoogleRE into 53 languages. Working with mBERT, we investigate three questions. (i) Can mBERT be used as a multilingual knowledge base? Most prior work only considers English. Extending research to multiple languages is important for diversity and accessibility. (ii) Is mBERT's performance as knowledge base language-independ…
XplaiNLP at CheckThat! 2025: Multilingual Subjectivity Detection with Finetuned Transformers and Prompt-Based Inference with Large Language Models
Quality and Usability Lab, Technische Universität Berlin, Berlin, Germany（柏林技术大学质量与可用性实验室）；German Research Center for Artificial Intelligence (DFKI), Berlin, Germany（德国人工智能研究中心（DFKI））
Zero-shot cross-lingual transfer learning has been shown to be highly challenging for tasks involving a lot of linguistic specificities or when a cultural gap is present between languages, such as in hate speech detection. In this paper, we highlight this limitation for hate speech detection in several domains and languages using strict experimental settings. Then, we propose to train on multilingual auxiliary tasks -- sentiment analysis, named entity recognition, and tasks relying on syntactic information -- to improve zero-shot transfer of hate speech detection models across languages. We sh…
Sparse language vectors from linguistic typology databases and learned embeddings from tasks like multilingual machine translation have been investigated in isolation, without analysing how they could benefit from each other's language characterisation. We propose to fuse both views using singular vector canonical correlation analysis and study what kind of information is induced from each source. By inferring typological features and language phylogenies, we observe that our representations embed typology and strengthen correlations with language relationships. We then take advantage of our m…
University of Trento（特伦托大学）；Fondazione Bruno Kessler（布鲁诺·凯斯勒基金会）；University of the Basque Country UPV/EHU（巴斯克地区大学）
The growing interest in employing counter narratives for hatred intervention brings with it a focus on dataset creation and automation strategies. In this scenario, learning to recognize counter narrative types from natural text is expected to be useful for applications such as hate speech countering, where operators from non-governmental organizations are supposed to answer to hate with several and diverse arguments that can be mined from online sources. This paper presents the first multilingual work on counter narrative type classification, evaluating SoTA pre-trained language models in mon…
We present a study on the design of multilingual Answer Sentence Selection (AS2) models, which are a core component of modern Question Answering (QA) systems. The main idea is to transfer data, created from one resource rich language, e.g., English, to other languages, less rich in terms of resources. The main findings of this paper are: (i) the training data for AS2 translated into a target language can be used to effectively fine-tune a Transformer-based model for that language; (ii) one multilingual Transformer model it is enough to rank answers in multiple languages; and (iii) mixed-langua…
China University of Petroleum (East China)（中国石油大学（华东））；JD Explore Academy（京东探索研究院）
For multilingual sequence-to-sequence pretrained language models (multilingual Seq2Seq PLMs), e.g. mBART, the self-supervised pretraining task is trained on a wide range of monolingual languages, e.g. 25 languages from CommonCrawl, while the downstream cross-lingual tasks generally progress on a bilingual language subset, e.g. English-German, making there exists the data discrepancy, namely domain discrepancy, and cross-lingual learning objective discrepancy, namely task discrepancy, between the pretraining and finetuning stages. To bridge the above cross-lingual domain and task gaps, we exten…
Microsoft Cognitive Services Research Group（微软认知服务研究组）
Commonsense reasoning (CSR) requires the model to be equipped with general world knowledge. While CSR is a language-agnostic process, most comprehensive knowledge sources are in few popular languages, especially English. Thus, it remains unclear how to effectively conduct multilingual commonsense reasoning (XCSR) for various languages. In this work, we propose to utilize English knowledge sources via a translate-retrieve-translate (TRT) strategy. For multilingual commonsense questions and choices, we collect related knowledge via translation and retrieval from the knowledge sources. The retrie…
Language Technologies Institute, Carnegie Mellon University（卡内基梅隆大学语言技术研究所）
To improve the performance of Neural Machine Translation~(NMT) for low-resource languages~(LRL), one effective strategy is to leverage parallel data from a related high-resource language~(HRL). However, multilingual data has been found more beneficial for NMT models that translate from the LRL to a target language than the ones that translate into the LRLs. In this paper, we aim to improve the effectiveness of multilingual transfer for NMT models that translate \emph{into} the LRL, by designing a better decoder word embedding. Extending upon a general-purpose multilingual encoding method Soft …
