arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.00071cs.AIcs.CLcs.ETcs.LG
收藏

组合推理:通过组合优化在生成式 AI 流水线中选择理由

Combinatorial Reasoning: Selecting Reasons in Generative AI Pipelines via Combinatorial Optimization

Icosa Computing Inc.(伊科萨计算公司);NASA ARC(美国国家航空航天局艾姆斯研究中心);USRA Research Institute for Advanced Computer Science (RIACS)(美国大学空间研究协会高级计算机科学研究所);Hewlett Packard Labs(惠普实验室);DataStax(DataStax公司)

Mert Esencan, Tarun Advaith Kumar, Ata Akbari Asanjan, P. Aaron Lott, Masoud Mohseni, Can Unlu, Davide Venturelli, Alan Ho

AI总结:

本文提出完全自动化的组合推理框架 CR,将 LLM 流水线采样的理由建模为 QUBO 问题并选择子集构建思维链提示,实验表明组合优化可增强生成式 AI 推理。

摘要:

近期的大语言模型(LLM)在需要人类智能的任务上展示了令人瞩目的能力,并向着类人人工智能(AI)迈出了重要一步。然而,LLM 在推理任务上的表现一直不够理想,且其推理能力仍存在重大争议。尽管已有研究表明,面向 LLM 的提示技术选择会改变其在包括推理在内的众多任务上的表现,但性能最佳的技术需要在了解当前任务的情况下由人工编写提示。我们提出了一个称为组合推理(Combinatorial Reasoning,CR)的框架,这是一种完全自动化的提示方法:从 LLM 流水线中采样理由,并将其映射为二次无约束二值优化(Quadratic Unconstrained Binary Optimization,QUBO)问题。该框架研究 QUBO 的解能否被有效用于选择一个有用的理由子集,以构建思维链风格的提示。我们探索了利用专用求解器加速 CR 的方法。我们还研究了更简单的零样本策略的性能,例如线性多数规则或随机选择理由。我们的初步研究表明,将组合求解器与生成式 AI 流水线相结合是 AI 推理的一条有吸引力的途径,并阐明了未来 CR 方法的设计原则。

英文摘要:

Recent Large Language Models (LLMs) have demonstrated impressive capabilities at tasks that require human intelligence and are a significant step towards human-like artificial intelligence (AI). Yet the performance of LLMs at reasoning tasks have been subpar and the reasoning capability of LLMs is a matter of significant debate. While it has been shown that the choice of the prompting technique to the LLM can alter its performance on a multitude of tasks, including reasoning, the best performing techniques require human-made prompts with the knowledge of the tasks at hand. We introduce a framework for what we call Combinatorial Reasoning (CR), a fully-automated prompting method, where reasons are sampled from an LLM pipeline and mapped into a Quadratic Unconstrained Binary Optimization (QUBO) problem. The framework investigates whether QUBO solutions can be profitably used to select a useful subset of the reasons to construct a Chain-of-Thought style prompt. We explore the acceleration of CR with specialized solvers. We also investigate the performance of simpler zero-shot strategies such as linear majority rule or random selection of reasons. Our preliminary study indicates that coupling a combinatorial solver to generative AI pipelines is an interesting avenue for AI reasoning and elucidates design principles for future CR methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑