arXiv 2412.125442024-12-28cs.AIcs.SE

Seed-CTS：释放树搜索在竞争级编程任务中实现卓越性能的潜力

Seed-CTS: Unleashing the Power of Tree Search for Superior Performance in Competitive Coding Tasks

ByteDance Seed（字节跳动种子项目（或：字节跳动 Seed））

Hao Wang, Boyi Liu, Yufeng Zhang, Jie Chen

AI总结：

针对竞争级代码生成，本文提出一种token级树搜索方法Seed-CTS，在LiveCodeBench-Hard上以Qwen2.5-Coder-32B-Instruct取得0.305通过率，并结合思维链提升至0.351，证明树搜索能显著增强竞争级代码生成性能。

摘要：

竞争级代码生成任务对当前最先进的大语言模型（LLMs）构成了重大挑战。例如，在 LiveCodeBench-Hard 数据集上，O1-Mini 和 O1-Preview 等模型的 pass@1 率分别仅为 0.366 和 0.143。尽管树搜索技术在数学和通用编程等领域已被证明有效，但其在竞争级代码生成中的潜力仍未得到充分探索。在本工作中，我们提出了一种专为代码生成设计的全新 token 级树搜索方法。借助 Qwen2.5-Coder-32B-Instruct，我们的方法在 LiveCodeBench-Hard 上取得了 0.305 的通过率，超过了 GPT4o-0513 的 pass@100 性能（0.245）。此外，通过结合思维链（Chain-of-Thought，CoT）提示，我们将方法的性能提升至 0.351，接近 O1-Mini 的 pass@1 率。为确保可复现性，我们报告了树搜索方法在测试集上每个问题所需的平均生成次数。我们的发现凸显了树搜索在显著提升竞争级代码生成任务性能方面的潜力。这为大规模合成具有挑战性的代码问题监督微调（SFT）数据开辟了新的可能性，从而推动竞争级代码生成任务的发展。

英文摘要：

Competition-level code generation tasks pose significant challenges for current state-of-the-art large language models (LLMs). For example, on the LiveCodeBench-Hard dataset, models such as O1-Mini and O1-Preview achieve pass@1 rates of only 0.366 and 0.143, respectively. While tree search techniques have proven effective in domains like mathematics and general coding, their potential in competition-level code generation remains under-explored. In this work, we propose a novel token-level tree search method specifically designed for code generation. Leveraging Qwen2.5-Coder-32B-Instruct, our approach achieves a pass rate of 0.305 on LiveCodeBench-Hard, surpassing the pass@100 performance of GPT4o-0513 (0.245). Furthermore, by integrating Chain-of-Thought (CoT) prompting, we improve our method's performance to 0.351, approaching O1-Mini's pass@1 rate. To ensure reproducibility, we report the average number of generations required per problem by our tree search method on the test set. Our findings underscore the potential of tree search to significantly enhance performance on competition-level code generation tasks. This opens up new possibilities for large-scale synthesis of challenging code problems supervised fine-tuning (SFT) data, advancing competition-level code generation tasks.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。