A key challenge for reinforcement learning is solving long-horizon planning problems. Recent work has leveraged programs to guide reinforcement learning in these settings. However, these approaches impose a high manual burden on the user since they must provide a guiding program for every new task. Partially observed environments further complicate the programming task because the program must implement a strategy that correctly, and ideally optimally, handles every possible configuration of the hidden regions of the environment. We propose a new approach, model predictive program synthesis (MPPS), that uses program synthesis to automatically generate the guiding programs. It trains a generative model to predict the unobserved portions of the world, and then synthesizes a program based on samples from this model in a way that is robust to its uncertainty. In our experiments, we show that our approach significantly outperforms non-program-guided approaches on a set of challenging benchmarks, including a 2D Minecraft-inspired environment where the agent must complete a complex sequence of subtasks to achieve its goal, and achieves a similar performance as using handcrafted programs to guide the agent. Our results demonstrate that our approach can obtain the benefits of program-guided reinforcement learning without requiring the user to provide a new guiding program for every new task.
Human-in-the-Loop Synthesis for Partially Observable Markov Decision Processes
The University of Texas at Austin（得克萨斯大学奥斯汀分校）；Radboud University（拉德堡德大学）；Albert-Ludwigs-Universität Freiburg（弗莱堡大学）；Worcester Polytechnic Institute (WPI)（伍斯特理工学院）
Real-World Knowledge-Guided Change Data Synthesis for Remote Sensing
School of Artificial Intelligence, Wuhan University（武汉大学人工智能学院）；Wuhan AI Research（武汉人工智能研究院）；Institute of Automation, University of Chinese Academy of Sciences（中国科学院大学自动化研究所）；Institute for Math & AI, Wuhan（武汉数学与人工智能研究院）
CDS: Knowledge Component-Driven Data Synthesis Guided by Cognitive Diagnosis Theory
College of Computer Science and Artificial Intelligence, Fudan University（复旦大学计算机科学与人工智能学院）；Shanghai Institute of Artificial Intelligence for Education, East China Normal University（华东师范大学教育人工智能研究所）；School of Data Science, Fudan University（复旦大学数据科学学院）；Huawei Noah’s Ark Lab（华为诺亚实验室）
Towards Compositional Generalization of LLMs via Skill Taxonomy Guided Data Synthesis
State Key Laboratory of Complex & Critical Software Environment, Beihang University（复杂与关键软件环境国家重点实验室，北京航空航天大学）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；Beijing Institute of Technology（北京理工大学）；Institute of Computing Technology, CAS（中国科学院计算技术研究所）
Progressive Boundary Guided Anomaly Synthesis for Industrial Anomaly Detection
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Pose Guided Human Image Synthesis with Partially Decoupled GAN
Ping An Technology (Shenzhen) Co., Ltd.（平安科技（深圳）有限公司）；University of Science and Technology of China（中国科学技术大学）；Shanghai International Studies University（上海外国语大学）
Learning to Guarantee Type Correctness in Code Generation through Type-Guided Program Synthesis
Peking University（北京大学）；University of Michigan（密歇根大学）；Shanghai Jiao Tong University（上海交通大学）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；Tencent（腾讯）
CLAIM: Clinically-Guided LGE Augmentation for Realistic and Diverse Myocardial Scar Synthesis and Segmentation
Insigneo Institute for in-silico, Medicine, University of Sheffield, Sheffield, UK（Insigneo医学研究所（in-silico），谢菲尔德大学）；School of Computer Science, University of Sheffield, Sheffield, UK（谢菲尔德大学计算机科学学院）；Department of Cardiology, Nottingham University Hospitals NHS Trust, Nottingham, UK（诺丁汉大学医院心脏科）；School of Medicine, University of Nottingham, Nottingham, UK（诺丁汉大学医学院）
