市政厅辩论提示法：通过多角色交互增强大语言模型的逻辑推理能力
Town Hall Debate Prompting: Enhancing Logical Reasoning in LLMs through Multi-Persona Interaction
AI总结：
本研究提出市政厅式辩论提示法（THDP），通过让LLM拆分多角色辩论提升逻辑推理，经ZebraLogic基准测试，5个LLM自定性格的角色配置可显著提升不同模型的推理准确率。
摘要：
辩论是人类用于解决问题的常用沟通形式，因其高效性而被广泛使用。从本质上讲，辩论能让人们在解决问题时提出多种观点，而对于复杂问题，每种观点都能开辟一条新的解题路径。在本研究中，我们将这一理念应用于大语言模型（LLM）决策，提出了市政厅式辩论提示法（THDP），这是一种将语言模型拆分为多个角色、让角色间相互辩论以得出结论的提示方法。我们的实验流程同时调整角色数量和每个角色的性格类型，以找到在ZebraLogic基准测试上取得最优性能的最佳市政厅规模与性格配置；ZebraLogic是一项侧重推理的基准测试，包含选择题和填空题两种题型。实验结果表明，由5个角色组成、角色性格类型由LLM自行确定的市政厅配置在ZebraLogic上表现最优：在GPT-4o上，其每格准确率相比单样本思维链（CoT）基线提升13%；在Claude 3.5 Sonnet上，谜题准确率提升9%；高难度谜题的准确率提升了10-15%。
英文摘要：
Debate is a commonly used form of human communication catered towards problem-solving because of its efficiency. Debate fundamentally allows multiple viewpoints to be brought up in problem-solving, and for complex problems, each viewpoint opens a new path for problem-solving. In this work, we apply this concept to LLM decision-making by proposing town hall-style debate prompting (THDP), a prompting method that splices a language model into multiple personas that will debate one another to reach a conclusion. Our experimental pipeline varies both the number of personas and the personality types of each persona to find the optimum town hall size and personality for benchmark performance as measured by ZebraLogic bench, a reasoning-intensive benchmark characterized by both multiple-choice and fill-in-the-blank questions. Our experimental results demonstrate that a town hall size of 5 personas with LLM-determined personality types performs optimally on ZebraLogic, achieving a 13\% improvement over one-shot CoT baselines in per-cell accuracy in GPT-4o, 9% puzzle accuracy increase in Claude 3.5 Sonnet, and an improvement in hard puzzle accuracy from 10-15%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
