Code generation aims to automatically generate source code from high-level task specifications, which can significantly increase productivity of software engineering. Recently, approaches based on large language models (LLMs) have shown remarkable code generation abilities on simple tasks. However, generate code for more complex tasks, such as competition-level problems, remains challenging. In this paper, we introduce Brainstorm framework for code generation. It leverages a brainstorming step that generates and selects diverse thoughts on the problem to facilitate algorithmic reasoning, where the thoughts are possible blueprint of solving the problem. We demonstrate that Brainstorm significantly enhances the ability of LLMs to solve competition-level programming problems, resulting in a more than 50% increase in the pass@$k$ metrics for ChatGPT on the CodeContests benchmark, achieving state-of-the-art performance. Furthermore, our experiments conducted on LeetCode contests show that our framework boosts the ability of ChatGPT to a level comparable to that of human programmers.
A Framework for Collaborating a Large Language Model Tool in Brainstorming for Triggering Creative Thoughts
R.B. Annis School of Engineering(R.B.安尼斯工程学院);University of Indianapolis(印第安纳波利斯大学);School of Journalism and Strategic Communication(新闻与战略传播学院);Ball State University(鲍尔州立大学)
Iterative Forward Tuning Boosts In-Context Learning in Language Models
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University of Chinese Academy of Sciences(中国科学院大学);Alibaba Group(阿里巴巴集团);MIT CSAIL(麻省理工学院计算机科学与人工智能实验室);Shanghai AI Laboratory(上海人工智能实验室)
Step Back to Leap Forward: Self-Backtracking for Boosting Reasoning of Language Models
Nanjing University(南京大学);National Key Laboratory for Novel Software Technology(现代软件工程国家重点实验室);School of Artificial Intelligence, Nanjing University(南京大学人工智能学院);School of Intelligence Science and Technology, Nanjing University(南京大学智能科学与技术学院)
Thinking Before Running! Efficient Code Generation with Thorough Exploration and Optimal Refinement
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学人工智能学院);Moonshot AI;Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);University of Electronic Science and Technology of China(电子科技大学);School of Artifcial Intelligence, Wuhan University(武汉大学人工智能学院);Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MoE(下一代智能搜索与推荐工程技术研究中心)
Unlocking Recursive Thinking of LLMs: Alignment via Refinement
Soochow University(苏州大学);Zhipu AI(智谱AI);Tsinghua University(清华大学);Key Laboratory of Data Intelligence and Advanced Computing, Soochow University(苏州大学数据智能与先进计算重点实验室)
DeepRAG: Thinking to Retrieve Step by Step for Large Language Models
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所信息处理实验室);University of Chinese Academy of Sciences(中国科学院大学);Pattern Recognition Center, WeChat AI, Tencent Inc, China(腾讯人工智能部模式识别中心)
Pratt School of Engineering(普拉特工程学院);Duke University(杜克大学);School of Computer Science(计算机科学学院);Northeast Electric Power University(东北电力大学);Department of Information and Communication Engineering(信息与通信工程系);Tongji University(同济大学);School of Computer Science and Technology(计算机科学与技术学院);School of International Education(国际教育学院);Beijing University of Chemical Technology(北京化工大学);East China Normal University(华东师范大学);Information Hub(信息枢纽);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Chinese Academy of Science(中国科学院)
DART: Difficulty-Adaptive Reasoning Truncation for Efficient Large Language Models
School of Artificial Intelligence, Shanghai Jiao Tong University(上海交通大学人工智能学院);CSE Department, The Chinese University of Hong Kong(香港中文大学计算机科学与工程系);Research and Development Department, SIMMIR Tech(Simmir科技研发部);School of Information Science and Technology, Xiamen University Tan Kah Kee College(厦门大学信息科学与技术学院);The University of Hong Kong(香港大学)
Reinforcing the Diffusion Chain of Lateral Thought with Diffusion Language Models
Zhejiang Univeristy(浙江大学);MAPLE Lab, Westlake University(西湖大学MAPLE实验室);Matterwave Intelligence;Institute of Advanced Technology, Westlake Institute for Advanced Study(西湖研究所以高级技术研究所)
Enhancing LLM Reliability via Explicit Knowledge Boundary Modeling
X-LANCE Lab, School of Computer Science MoE Key Lab of Artificial Intelligence, SJTU AI Institute Shanghai Jiao Tong University, Shanghai, China(X-LANCE实验室,计算机科学学院,人工智能关键实验室,上海交通大学人工智能研究院,上海,中国);Jiangsu Key Lab of Language Computing, Suzhou, China(语言计算重点实验室,苏州,中国);AISpeech Co., Ltd., Suzhou, China(AISpeech公司,苏州,中国);Center for Artificial Intelligence Research (CAiRE) Hong Kong University of Science and Technology, Hong Kong, China(人工智能研究中心(CAiRE)香港科技大学,香港,中国);Suzhou Laboratory, Suzhou, China(苏州实验室,苏州,中国)
VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Models
National University of Singapore(新加坡国立大学);Fudan University(复旦大学);Tsinghua University(清华大学);Zhejiang University(浙江大学);University of Science and Technology of China(中国科学技术大学);vivo