Pre-trained Large Language Models (LLMs) are beginning to dominate the discourse around automatic code generation with natural language specifications. In contrast, the best-performing synthesizers in the domain of formal synthesis with precise logical specifications are still based on enumerative algorithms. In this paper, we evaluate the abilities of LLMs to solve formal synthesis benchmarks by carefully crafting a library of prompts for the domain. When one-shot synthesis fails, we propose a novel enumerative synthesis algorithm, which integrates calls to an LLM into a weighted probabilistic search. This allows the synthesizer to provide the LLM with information about the progress of the enumerator, and the LLM to provide the enumerator with syntactic guidance in an iterative loop. We evaluate our techniques on benchmarks from the Syntax-Guided Synthesis (SyGuS) competition. We find that GPT-3.5 as a stand-alone tool for formal synthesis is easily outperformed by state-of-the-art formal synthesis algorithms, but our approach integrating the LLM into an enumerative synthesis algorithm shows significant performance gains over both the LLM and the enumerative synthesizer alone and the winning SyGuS competition tool.
ScaleWoB: Guiding GUI Agents with Coding Agents via Large-Scale Environmental Synthesis
Institute for AI Industry Research (AIR), Tsinghua University(人工智能产业研究院(AIR),清华大学);University of Electronic Science and Technology of China(电子科技大学);MiLM Plus, Xiaomi Inc.(小米公司MiLM Plus团队)
A GPT-4 Reticular Chemist for Guiding MOF Discovery
University of California, Berkeley(加州大学伯克利分校);Kavli Energy Nanoscience Institute(卡夫利能源纳米科学研究所);Bakar Institute of Digital Materials for the Planet(巴卡尔全球数字材料研究所);College of Computing, Data Science, and Society(计算、数据科学与社会学院);Department of Chemistry(化学系);Department of Electrical Engineering and Computer Sciences(电气工程与计算机科学系);Department of Statistics(统计学系);Department of Mathematics(数学系);School of Information(信息学院);King Abdulaziz City for Science and Technology(阿卜杜勒阿齐兹国王科学技术城)
FlowSteer: Guiding Few-Step Image Synthesis with Authentic Trajectories
Tsinghua University(清华大学);WeChat Vision, Tencent Inc.(腾讯公司微信视觉部门);The Hong Kong University of Science and Technology(香港科学与技术大学);The University of Hong Kong(香港大学);The Hong Kong Polytechnic University(香港理工大学)
An Information-Theoretic Criterion for Efficient Data Synthesis
CFCS, School of Computer Science, Peking University, Beijing, China(计算机科学系,北京大学,北京,中国);Department of Information Management, Peking University, Beijing, China(信息管理系,北京大学,北京,中国)
Huawei Noah’s Ark Lab, UK(华为诺亚实验室,英国);Imperial College London(伦敦帝国理工学院);UCL Centre for Artificial Intelligence(伦敦大学学院人工智能中心);University of Zurich(苏黎世大学);University of Sheffield(谢菲尔德大学);University of Cambridge(剑桥大学)
SARGes: Semantically Aligned Reliable Gesture Generation via Intent Chain
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Beijing Engineering Research Center of Mixed Reality and Advanced Display(混合现实与先进显示北京工程研究中心);Institute of Technology, Beijing(北京理工大学);Kuaishou Technology(快手科技)
Scaling Neural Program Synthesis with Distribution-based Search
CNRS, LaBRI and Université de Bordeaux(法国国家科学研究中心、波尔多计算机科学实验室与波尔多大学);The Alan Turing Institute of data science(艾伦·图灵数据科学研究所);Cornell University(康奈尔大学);University of Paris(巴黎大学);Indian Institute of Technology Bombay(印度理工学院孟买分校)
Reshaping MOFs text mining with a dynamic multi-agents framework of large language model
Center for Environment and Water Resources, College of Chemistry and Chemical Engineering, Central South University(环境与水资源中心,化学与化工学院,中南大学);Key Laboratory of Hunan Province for Water Environment and Agriculture Product Safety(湖南省水环境与农产品安全重点实验室);School of Resources and Environment, Hunan University of Technology and Business(资源与环境学院,湖南工业大学);School of Artificial Intelligence, University of Chinese Academy of Sciences(人工智能学院,中国科学院大学);Faculty of Data Science, City University of Macau(数据科学学院,澳门城市大学);State Key Laboratory of High Performance Ceramics and Superfine Microstructure, Shanghai Institute of Ceramics, Chinese Academy of Sciences(高性能陶瓷与超细微结构重点实验室,上海陶瓷研究所,中国科学院);Beijing Key Laboratory for Green Catalysis and Separation, Department of Chemical Engineering, College of Materials Science and Engineering, Beijing University of Technology(绿色催化与分离北京市重点实验室,化学工程系,材料科学与工程学院,北京理工大学);State Key Joint Laboratory of Environment Simulation and Pollution Control, School of Environment, Tsinghua University(环境模拟与污染控制国家重点联合实验室,环境学院,清华大学);School of Chemical Engineering and Materials Science, Yueyang University(化学工程与材料科学学院,岳阳大学);School of Computer Science and Engineering, Central South University(计算机科学与工程学院,中南大学);School of Software Engineering, Sun Yat-sen University(软件工程学院,中山大学)