GitHub 问题能否用思维树解决？
Can Github issues be solved with Tree Of Thoughts?
AI总结：
本研究探索将思维树（ToT）推理框架应用于解决GitHub问题，实验表明仅靠ToT不足以超越现有方法，并提出了深化思维与引入智能体能力等改进方向。
摘要：
虽然已有大量关于大型语言模型（LLM）代码生成的研究，其中像HumanEval这样的基准已被以惊人的96.3%成功率超越，但这些基准主要评判模型在基本函数级代码生成上的性能，缺乏解决GitHub问题等真实世界场景所需的批判性思维和范围概念。本研究引入了思维树（ToT）语言模型推理框架的应用，以增强LLM在此复杂任务中的决策和问题解决能力。与传统的输入-输出（IO）提示和检索增强生成（RAG）技术相比，ToT旨在通过促进对多个推理轨迹的结构化探索并实现对潜在解决方案的自我评估来提高性能。我们实验性地将ToT部署于解决SWE-bench实例中包含的一个GitHub问题。然而，我们的结果显示，仅靠ToT框架不足以赋予LLM超越现有方法的批判性推理能力。在本文中，我们分析了这些不足的潜在原因，并确定了关键改进领域，如深化思维过程和引入智能体能力。本研究的见解旨在为未来完善ToT应用和更好地发挥LLM在真实世界问题解决场景中的潜力提供方向。
英文摘要：
While there have been extensive studies in code generation by large language models (LLM), where benchmarks like HumanEval have been surpassed with an impressive 96.3% success rate, these benchmarks predominantly judge a model's performance on basic function-level code generation and lack the critical thinking and concept of scope required of real-world scenarios such as solving GitHub issues. This research introduces the application of the Tree of Thoughts (ToT) language model reasoning framework for enhancing the decision-making and problem-solving abilities of LLMs for this complex task. Compared to traditional input-output (IO) prompting and Retrieval Augmented Generation (RAG) techniques, ToT is designed to improve performance by facilitating a structured exploration of multiple reasoning trajectories and enabling self-assessment of potential solutions. We experimentally deploy ToT in tackling a Github issue contained within an instance of the SWE-bench. However, our results reveal that the ToT framework alone is not enough to give LLMs the critical reasoning capabilities to outperform existing methods. In this paper we analyze the potential causes of these shortcomings and identify key areas for improvement such as deepening the thought process and introducing agentic capabilities. The insights of this research are aimed at informing future directions for refining the application of ToT and better harnessing the potential of LLMs in real-world problem-solving scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
