Mulberry：通过集体蒙特卡洛树搜索赋予多模态大语言模型类o1推理与反思能力
Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Search
- Nanyang Technological University（南洋理工大学）
- Tsinghua University（清华大学）
- Baidu Inc.（百度公司）
- Sun Yat-sen University（中山大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文提出集体蒙特卡洛树搜索CoMCTS，通过多模型协作搜索推理路径，构建Mulberry-260k数据集并训练出具备类o1逐步推理与反思能力的多模态大语言模型Mulberry，在多个基准上取得优异表现。
AI中文摘要：
在这项工作中，我们旨在开发一种多模态大语言模型（MLLM），使其能够通过学习创建从推理过程到最终答案的每一个中间步骤来理解和解决问题。为此，我们提出了集体蒙特卡洛树搜索（CoMCTS），这是一种面向多模态大语言模型的新型“学习推理”方法，它将集体学习的概念引入“树搜索”中，以实现有效且高效的推理路径搜索与学习。CoMCTS的核心思想是利用来自多个模型的集体知识，通过四个迭代操作——扩展、模拟与错误定位、回传和选择——来协作地推测、搜索并识别通向正确答案的有效推理路径。利用CoMCTS，我们构建了Mulberry-260k，这是一个多模态数据集，其中每个问题都带有一棵由丰富、显式且定义明确的推理节点构成的树。基于Mulberry-260k，我们执行集体监督微调（collective SFT）来训练我们的模型Mulberry，这是一系列具备类o1逐步推理与反思能力的多模态大语言模型。大量实验证明了我们所提方法在多个基准上的优越性。代码将发布于 https://github.com/HJYao00/Mulberry。
英文摘要：
In this work, we aim to develop an MLLM that understands and solves questions by learning to create each intermediate step of the reasoning involved till the final answer. To this end, we propose Collective Monte Carlo Tree Search (CoMCTS), a new learning-to-reason method for MLLMs, which introduces the concept of collective learning into ``tree search'' for effective and efficient reasoning-path searching and learning. The core idea of CoMCTS is to leverage collective knowledge from multiple models to collaboratively conjecture, search and identify effective reasoning paths toward correct answers via four iterative operations including Expansion, Simulation and Error Positioning, Backpropagation, and Selection. Using CoMCTS, we construct Mulberry-260k, a multimodal dataset with a tree of rich, explicit and well-defined reasoning nodes for each question. With Mulberry-260k, we perform collective SFT to train our model, Mulberry, a series of MLLMs with o1-like step-by-step Reasoning and Reflection capabilities. Extensive experiments demonstrate the superiority of our proposed methods on various benchmarks. Code will be available at https://github.com/HJYao00/Mulberry
