OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement
Multimodal Art Projection Research Community(多模态艺术投影研究共同体);University of Waterloo(滑铁卢大学);Allen Institute for Artificial Intelligence(艾伦人工智能研究所);HKUST(香港科技大学);IN.AI Research(IN.AI研究机构)
Tianyu Zheng, Ge Zhang, Tianhao Shen, Xueling Liu, Bill Yuchen Lin, Jie Fu, Wenhu Chen, Xiang Yue
The introduction of large language models has significantly advanced code generation. However, open-source models often lack the execution capabilities and iterative refinement of advanced systems like the GPT-4 Code Interpreter. To address this, we introduce OpenCodeInterpreter, a family of open-source code systems designed for generating, executing, and iteratively refining code. Supported by Code-Feedback, a dataset featuring 68K multi-turn interactions, OpenCodeInterpreter integrates execution and human feedback for dynamic code refinement. Our comprehensive evaluation of OpenCodeInterpreter across key benchmarks such as HumanEval, MBPP, and their enhanced versions from EvalPlus reveals its exceptional performance. Notably, OpenCodeInterpreter-33B achieves an accuracy of 83.2 (76.4) on the average (and plus versions) of HumanEval and MBPP, closely rivaling GPT-4's 84.2 (76.2) and further elevates to 91.6 (84.6) with synthesized human feedback from GPT-4. OpenCodeInterpreter brings the gap between open-source code generation models and proprietary systems like GPT-4 Code Interpreter.
Write, Execute, Refine: From Skill Follower to Skill Optimizer via Reinforcement Learning from Execution Feedback
Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));The Chinese University of Hong Kong(香港中文大学);Huawei Technologies Co., Ltd.(华为技术有限公司);Harbin Institute of Technology(哈尔滨工业大学)
ErrorLLM: Modeling SQL Errors for Text-to-SQL Refinement
The Hong Kong Polytechnic University(香港理工大学);City University of Macau(澳门城市大学);Jilin University(吉林大学);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳));Beihang University(北航大学)
A Stitch in Time Saves Nine: Proactive Self-Refinement for Language Models
Shanghai Institute of Artificial Intelligence for Education(上海人工智能教育研究院);East China Normal University(华东师范大学);School of Data Science, Fudan University(复旦大学数据科学学院);College of Computer Science and Artificial Intelligence, Fudan University(复旦大学计算机科学与人工智能学院);Antgroup(蚂蚁集团)
Large Language Model Agent for Modular Task Execution in Drug Discovery
Department of Chemical Engineering, Carnegie Mellon University(化学工程系,卡内基梅隆大学);Department of Material Science and Engineering, Carnegie Mellon University(材料科学与工程系,卡内基梅隆大学);Department of Mechanical Engineering, Carnegie Mellon University(机械工程系,卡内基梅隆大学)
LLM-Based SQL Generation: Prompting, Self-Refinement, and Adaptive Weighted Majority Voting
Institute of Information Management(信息管理研究所);National Yang Ming Chiao Tung University(阳明交通大学);R. B. Annis School of Engineering(R. B. Annis工程学院);University of Indianapolis(印第安纳大学)
InteractWeb-Bench: Can Multimodal Agent Escape Blind Execution in Interactive Website Generation?
Shenzhen Institute of Advanced Technology Chinese Academy of Sciences(深圳先进技术研究院中国科学院);University of Chinese Academy of Sciences(中国科学院大学);Dalian University of Technology(大连理工大学);UNSW Sydney(悉尼大学);Shenzhen University of Advanced Technology(深圳先进技术大学)
A Pair Programming Framework for Code Generation via Multi-Plan Exploration and Feedback-Driven Refinement
State Key Laboratory for Novel Software Technology(计算机软件新技术国家重点实验室);Nanjing University(南京大学);National Institute of Healthcare Data Science(国家卫生健康数据科学研究院)
LLM-Driven Self-Refinement for Embodied Drone Task Planning
School of Computer Science and Engineering, Central South University(中南大学计算机科学与工程学院);Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系)
Agent Planning Benchmark: A Diagnostic Framework for Planning Capabilities in LLM Agents
Tongji University(同济大学);Shanghai AI Laboratory(上海人工智能实验室);Harbin Institute of Technology(哈尔滨工业大学);Fudan University(复旦大学);Skywork AI;University of California, Santa Cruz(加州大学圣克鲁兹分校);Shanghai Jiao Tong University(上海交通大学);The Chinese University of Hong Kong(香港中文大学)
Knowing How to Edit: Reliable Evaluation Signals for Diagnosing and Optimizing Prompts at Query Level
School of Information Sciences, University of Illinois Urbana-Champaign(信息科学学院,伊利诺伊大学厄巴纳-香槟分校);College of Engineering, Carnegie Mellon University(工程学院,卡内基梅隆大学)
Multi-LLM Orchestration for High-Quality Code Generation: Exploiting Complementary Model Strengths
Chinese Academy of Science(中国科学院);University of Arizona(亚利桑那大学);Wuhan University(武汉大学);East Carolina University(东卡罗来纳大学);University of Georgia(佐治亚大学);University of Waterloo(滑铁卢大学)