arXiv 2504.018572025-04-02cs.CLcs.AI

跨语言一致性：一种提升大型语言模型推理能力的新型推理框架

Cross-Lingual Consistency: A Novel Inference Framework for Advancing Reasoning in Large Language Models

Shandong Inspur Artificial Intelligence Research Institute Co., Ltd.（山东浪潮人工智能研究院有限公司）；Shandong Yunhai Guochuang Cloud Computing Equipment Industry Innovation Center Co., Ltd.（山东云海国创云计算装备产业创新中心有限公司）

Zhiwei Yu, Tuo Li, Changhong Wang, Hui Chen, Lang Zhou

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对多语言偏差导致小模型推理不一致的问题，提出跨语言一致性（CLC）框架，通过多语言推理路径的多数投票提升LLM推理能力，在多个数据集上取得显著准确率提升。

摘要：

思维链（CoT）已成为增强大型语言模型（LLMs）推理能力的关键机制，其中自一致性在提升性能方面展现出显著潜力。然而，多语言训练语料库中固有的语言偏差经常导致语义漂移和逻辑不一致，尤其是在处理复杂推理任务的低于10B参数的LLMs中。为了克服这些限制，我们提出了跨语言一致性（CLC）框架，这是一种创新的推理范式，通过多数投票整合多语言推理路径，以提升LLMs的推理能力。在CMATH数据集上的实证评估表明，CLC优于传统的自一致性方法，分别为DeepSeek-Math-7B-Instruct、Qwen2.5-Math-7B-Instruct和Gemma2-9B-Instruct带来了9.5%、6.5%和6.0%的绝对准确率提升。将CLC的语言范围扩展至11种不同语言带来了两项协同效益：1）通过多语言集成投票中和多语言训练语料库中的语言偏差；2）通过探索更广泛的多语言解空间来规避单语言推理陷阱。这种双重效益在经验上促成了比单语言自一致性基线更具全局最优性的推理路径，这一点可通过Gemma2-9B-Instruct在MGSM数据集上取得4.1%-18.5%的准确率提升得到证明。

英文摘要：

Chain-of-thought (CoT) has emerged as a critical mechanism for enhancing reasoning capabilities in large language models (LLMs), with self-consistency demonstrating notable promise in boosting performance. However, inherent linguistic biases in multilingual training corpora frequently cause semantic drift and logical inconsistencies, especially in sub-10B parameter LLMs handling complex inference tasks. To overcome these constraints, we propose the Cross-Lingual Consistency (CLC) framework, an innovative inference paradigm that integrates multilingual reasoning paths through majority voting to elevate LLMs' reasoning capabilities. Empirical evaluations on the CMATH dataset reveal CLC's superiority over the conventional self-consistency method, delivering 9.5%, 6.5%, and 6.0% absolute accuracy gains for DeepSeek-Math-7B-Instruct, Qwen2.5-Math-7B-Instruct, and Gemma2-9B-Instruct respectively. Expanding CLC's linguistic scope to 11 diverse languages implies two synergistic benefits: 1) neutralizing linguistic biases in multilingual training corpora through multilingual ensemble voting, 2) escaping monolingual reasoning traps by exploring the broader multilingual solution space. This dual benefits empirically enables more globally optimal reasoning paths compared to monolingual self-consistency baselines, as evidenced by the 4.1%-18.5% accuracy gains using Gemma2-9B-Instruct on the MGSM dataset.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。