MALT: Improving Reasoning with Multi-Agent LLM Training
University of Oxford(牛津大学);Cooperative AI Foundation(合作人工智能基金会);MBZUAI(穆扎夫卡尔人工智能研究所);Stanford University(斯坦福大学)
Sumeet Ramesh Motwani, Chandler Smith, Rocktim Jyoti Das, Rafael Rafailov, Ivan Laptev, Philip H. S. Torr, Fabio Pizzati, Ronald Clark, Christian Schroeder de Witt
Large Language Models (LLMs) often produce answers with a single chain-of-thought, which restricts their ability to explore reasoning paths or self-correct flawed outputs in complex tasks. In this paper, we introduce MALT (Multi-Agent LLM Training), a novel post-training strategy that divides the reasoning process into generation, verification, and refinement steps using a sequential pipeline of heterogeneous agents. During data generation, each agent is repeatedly sampled to form a multi-agent search tree, where final outputs are graded against ground-truth data. We then apply value iteration to propagate reward signals back to each role-conditioned model, automatically producing multi-agent post-training data without human or teacher-model supervision. Our off-policy approach allows each agent to specialize by learning from correct and incorrect trajectories, ultimately improving the end-to-end reasoning chain. On MATH, GSM8K, and CSQA, MALT surpasses the same baseline LLM with a relative improvement of 15.66%, 7.42%, and 9.40% respectively, making it an important advance towards multi-agent cooperative training.
Neuro-Symbolic Artificial Intelligence: Towards Improving the Reasoning Abilities of Large Language Models
National Key Laboratory for Novel Software Technology, Nanjing University, China(国家新型软件技术重点实验室,南京大学,中国);School of Artificial Intelligence, Nanjing University, China(人工智能学院,南京大学,中国);School of Intelligence Science and Technology, Nanjing University, China(智能科学与技术学院,南京大学,中国)
Improving Rule-based Reasoning in LLMs using Neurosymbolic Representations
School of Computer Science, University of Waterloo, Waterloo, Canada(计算机科学学院,滑铁卢大学,滑铁卢,加拿大);Centre for Theoretical Neuroscience, University of Waterloo(理论神经科学中心,滑铁卢大学)
Structured, flexible, and robust: benchmarking and improving large language models towards more human-like behavior in out-of-distribution reasoning tasks
Scaling Reasoning Hop Exposes Weaknesses: Demystifying and Improving Hop Generalization in Large Language Models
University of Science and Technology of China(中国科学技术大学);City University of Hong Kong(香港城市大学);Zhejiang University(浙江大学);City University of Hong Kong, Shenzhen Research Institute(香港城市大学深圳研究院)
Grad2Reward: From Sparse Judgment to Dense Rewards for Improving Open-Ended LLM Reasoning
School of Information Science and Technology, ShanghaiTech University(信息科学与技术学院,上海科技大学);State Key Laboratory of General Artificial Intelligence, BIGAI(通用人工智能国家重点实验室,BIGAI)
Active-GRPO: Adaptive Imitation and Self-Improving Reasoning for Molecular Optimization
School of Medicine, Stanford University(斯坦福大学医学院);Data Science Institute, University of Chicago(芝加哥大学数据科学研究所);Pritzker School of Molecular Engineering, University of Chicago(芝加哥大学普利兹克分子工程学院);Department of Computer Science, University of Chicago(芝加哥大学计算机科学系);Argonne National Laboratory(阿贡国家实验室)
SyncLoop: A Multimodal Dual-Loop Framework for Self-Improving Mathematical Reasoning
Sun Yat-sen University(中山大学);The Hong Kong Polytechnic University(香港理工大学);MBZUAI;Huawei Noah’s Ark Lab(华为诺亚实验室);Yinwang Intelligent Technology Co. Ltd(依文智科有限公司)