Chameleon：大语言模型的即插即用组合推理
Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models
AI总结：
Chameleon 通过即插即用模块组合多种工具，以 LLM 规划器为核心解决复杂多模态推理任务，并在 ScienceQA 和 TabMWP 上显著提升准确率。
摘要：
大语言模型（LLMs）凭借涌现的推理能力，在解决各类自然语言处理任务方面取得了显著进展。然而，LLMs 存在固有局限：它们无法访问最新信息（存储于网络或任务特定知识库中）、无法使用外部工具，也难以进行精确的数学与逻辑推理。本文提出 Chameleon，一个通过为 LLMs 增加即插即用组合推理模块来缓解上述局限的 AI 系统。Chameleon 通过组合多种工具（如 LLMs、现成的视觉模型、网络搜索引擎、Python 函数以及基于启发式的模块）来合成程序，以完成复杂推理任务。Chameleon 的核心是一个基于 LLM 的规划器，它组装一系列待执行的工具以生成最终响应。我们在两个多模态知识密集型推理任务 ScienceQA 和 TabMWP 上展示了 Chameleon 的有效性。由 GPT-4 驱动的 Chameleon 在 ScienceQA 上取得了 86.54% 的整体准确率，将已发表的最佳少样本结果提升了 11.37%。在 TabMWP 上，GPT-4 驱动的 Chameleon 将准确率提高了 17.0%，把最优水平提升至 98.78%。我们的分析还表明，与 ChatGPT 驱动的规划器相比，GPT-4 驱动的规划器通过从指令中推断潜在约束，展现出更一致且更合理的工具选择。项目地址为 https://chameleon-llm.github.io。
英文摘要：
Large language models (LLMs) have achieved remarkable progress in solving various natural language processing tasks due to emergent reasoning abilities. However, LLMs have inherent limitations as they are incapable of accessing up-to-date information (stored on the Web or in task-specific knowledge bases), using external tools, and performing precise mathematical and logical reasoning. In this paper, we present Chameleon, an AI system that mitigates these limitations by augmenting LLMs with plug-and-play modules for compositional reasoning. Chameleon synthesizes programs by composing various tools (e.g., LLMs, off-the-shelf vision models, web search engines, Python functions, and heuristic-based modules) for accomplishing complex reasoning tasks. At the heart of Chameleon is an LLM-based planner that assembles a sequence of tools to execute to generate the final response. We showcase the effectiveness of Chameleon on two multi-modal knowledge-intensive reasoning tasks: ScienceQA and TabMWP. Chameleon, powered by GPT-4, achieves an 86.54% overall accuracy on ScienceQA, improving the best published few-shot result by 11.37%. On TabMWP, GPT-4-powered Chameleon improves the accuracy by 17.0%, lifting the state of the art to 98.78%. Our analysis also shows that the GPT-4-powered planner exhibits more consistent and rational tool selection via inferring potential constraints from instructions, compared to a ChatGPT-powered planner. The project is available at https://chameleon-llm.github.io.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
