In 2023, we are using the latest models of GPT-4 to advance program synthesis. The large language models have significantly improved the state-of-the-art for this purpose. To make these advancements more accessible, we have created a repository that connects these models to Huamn Eval. This dataset was initally developed to be used with a language model called CODEGEN on natural and programming language data. The utility of these trained models is showcased by demonstrating their competitive performance in zero-shot Python code generation on HumanEval tasks compared to previous state-of-the-art solutions. Additionally, this gives way to developing more multi-step paradigm synthesis. This benchmark features 160 diverse problem sets factorized into multistep prompts that our analysis shows significantly improves program synthesis over single-turn inputs. All code is open source at https://github.com/daniel442li/gpt-human-eval .
OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement
Multimodal Art Projection Research Community（多模态艺术投影研究共同体）；University of Waterloo（滑铁卢大学）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）；HKUST（香港科技大学）；IN.AI Research（IN.AI研究机构）
Is GPT-OSS Good? A Comprehensive Evaluation of OpenAI's Latest Open Source Models
AI Agent Lab, Vokram Group（AI代理实验室，Vokram集团）；Purdue University（普渡大学）；Georgia Institute of Technology（佐治亚理工学院）；University of Minnesota（明尼苏达大学）；Emory University（埃默里大学）；Imperial College London（伦敦帝国学院）
MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code Generation
University of Edinburgh（爱丁堡大学）；The University of Hong Kong（香港大学）；Harbin Institute of Technology（哈尔滨工业大学）；Singapore University of Technology and Design（新加坡科技设计大学）；South China University of Technology（华南理工大学）；City University of Hong Kong（香港城市大学）；University of Cambridge（剑桥大学）
An Empirical Study on Information Extraction using Large Language Models
College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University（吉林大学符号计算与知识工程教育部重点实验室）；Shenzhen Research Institute of Big Data, The Chinese University of Hong Kong, Shenzhen, China（香港中文大学（深圳）大数据研究院）；School of Data Science, The Chinese University of Hong Kong, Shenzhen, China（香港中文大学（深圳）数据科学学院）；College of Software, Jilin University（吉林大学软件学院）；School of Information Technology, Halmstad University, Sweden（哈尔姆斯塔德大学信息技术学院）
Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verification
The Chinese University of Hong Kong（香港中文大学）；Nanjing University（南京大学）；University of Science and Technology of China（中国科学技术大学）；Tsinghua University（清华大学）；City University of Hong Kong（香港城市大学）；Changsha University of Science and Technology（长沙理工大学）；Tufts University（塔夫茨大学）
An Early Investigation into the Utility of Multimodal Large Language Models in Medical Imaging
Faculty of Computing and Information Technology, Sohar University（苏哈尔大学计算与信息技术学院）；Foundation University School of Sciences and Technology（基础大学科学与技术学院）；College of Science and Engineering, Hamad Bin Khalifa University（哈马德·本·哈利法大学科学与工程学院）
An evaluation of GPT models for phenotype concept recognition
Rare Care Centre（罕见病护理中心）；Perth Children's Hospital（珀斯儿童医院）；Telethon Kids Institute（Telethon儿童研究所）；School of Electrical Engineering, Computing and Mathematical Sciences, Curtin University（科廷大学电气工程、计算与数学科学学院）；SingHealth Duke-NUS Institute of Precision Medicine（新加坡保健服务集团杜克-新加坡国立大学精准医学研究所）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）；Western Australian Register of Developmental Anomalies（西澳大利亚发育异常登记处）；King Edward Memorial Hospital（爱德华国王纪念医院）；Faculty of Health and Medical Sciences, University of Western Australia（西澳大利亚大学健康与医学科学学院）；University of Colorado Anschutz Medical Campus（科罗拉多大学安舒茨医学校区）；The Jackson Laboratory for Genomic Medicine（杰克逊基因组医学实验室）；Institute for Systems Genomics, University of Connecticut（康涅狄格大学系统基因组学研究所）
Towards Large Language Model Aided Program Refinement
National University of Singapore（新加坡国立大学）；Griffith University（格里菲斯大学）；Peking University（北京大学）；Singapore Institute of Technology（新加坡科技设计大学）；Shanghai Jiaotong University（上海交通大学）；Singapore Management University（新加坡管理大学）
