Compositional Flows for 3D Molecule and Synthesis Pathway Co-design
School of Computing Science, Simon Fraser University, Canada（计算科学学院，西蒙弗雷泽大学，加拿大）；Department of Chemistry, KAIST, Republic of Korea（化学系，韩国科学技术院）；Department of Computer Science（计算机科学系）；Engineering, Chalmers University of Technology, Sweden（工程学院，查尔姆斯理工大学，瑞典）；Department of Computer Science, University of Oxford, UK（计算机科学系，牛津大学，英国）；NVIDIA
Tony Shen, Seonghwan Seo, Ross Irwin, Kieran Didi, Simon Olsson, Woo Youn Kim, Martin Ester
Many generative applications, such as synthesis-based 3D molecular design, involve constructing compositional objects with continuous features. Here, we introduce Compositional Generative Flows (CGFlow), a novel framework that extends flow matching to generate objects in compositional steps while modeling continuous states. Our key insight is that modeling compositional state transitions can be formulated as a straightforward extension of the flow matching interpolation process. We further build upon the theoretical foundations of generative flow networks (GFlowNets), enabling reward-guided sampling of compositional structures. We apply CGFlow to synthesizable drug design by jointly designing the molecule's synthetic pathway with its 3D binding pose. Our approach achieves state-of-the-art binding affinity on all 15 targets from the LIT-PCBA benchmark, and 5.8$\times$ improvement in sampling efficiency compared to 2D synthesis-based baseline. To our best knowledge, our method is also the first to achieve state of-art-performance in both Vina Dock (-9.38) and AiZynth success rate (62.2\%) on the CrossDocked benchmark.
GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphs
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；ETH Zürich（苏黎世联邦理工学院）；University of Tübingen（蒂宾根大学）；Tübingen AI Center（蒂宾根人工智能中心）；University of Cambridge（剑桥大学）
Many aspects of human reasoning, including language, require learning rules from very little data. Humans can do this, often learning systematic rules from very few examples, and combining these rules to form compositional rule-based systems. Current neural architectures, on the other hand, often fail to generalize in a compositional manner, especially when evaluated in ways that vary systematically from training. In this work, we present a neuro-symbolic model which learns entire rule systems from a small set of examples. Instead of directly predicting outputs from inputs, we train our model …
Agentic Proposing: Enhancing Large Language Model Reasoning via Compositional Skill Synthesis
AI DATA, Alibaba Group Holding Limited（阿里数据，阿里巴巴集团控股有限公司）；EPIC Lab, Shanghai Jiao Tong University（上海交通大学EPIC实验室）；Shanghai University of Finance（上海财经大学）；Wuhan University（武汉大学）
Tel Aviv University（特拉维夫大学）；UC Berkeley（加州大学伯克利分校）；UC San Diego（加州大学圣地亚哥分校）；NVIDIA Research（英伟达研究院）；Bar-Ilan University（巴伊兰大学）
Videos of actions are complex signals containing rich compositional structure in space and time. Current video generation methods lack the ability to condition the generation on multiple coordinated and potentially simultaneous timed actions. To address this challenge, we propose to represent the actions in a graph structure called Action Graph and present the new ``Action Graph To Video'' synthesis task. Our generative model for this task (AG2Vid) disentangles motion and appearance features, and by incorporating a scheduling mechanism for actions facilitates a timely and coordinated video gen…
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；University of Tübingen（蒂宾根大学）
Deep generative models allow for photorealistic image synthesis at high resolutions. But for many applications, this is not enough: content creation also needs to be controllable. While several recent works investigate how to disentangle underlying factors of variation in the data, most of them operate in 2D and hence ignore that our world is three-dimensional. Further, only few works consider the compositional nature of scenes. Our key hypothesis is that incorporating a compositional 3D scene representation into the generative model leads to more controllable image synthesis. Representing sce…
German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心（DFKI））；Saarland Informatics Campus（萨尔兰信息学园区）；Max-Planck Institute for Informatics（马克斯·普朗克信息学研究所）
"How can we animate 3D-characters from a movie script or move robots by simply telling them what we would like them to do?" "How unstructured and complex can we make a sentence and still generate plausible movements from it?" These are questions that need to be answered in the long-run, as the field is still in its infancy. Inspired by these problems, we present a new technique for generating compositional actions, which handles complex input sentences. Our output is a 3D pose sequence depicting the actions in the input sentence. We propose a hierarchical two-stream sequential model to explore…
Hybrid systems are a compact and natural mechanism with which to address problems in robotics. This work introduces an approach to learning hybrid systems from demonstrations, with an emphasis on extracting models that are explicitly verifiable and easily interpreted by robot operators. We fit a sequence of controllers using sequential importance sampling under a generative switching proportional controller task model. Here, we parameterise controllers using a proportional gain and a visually verifiable joint angle goal. Inference under this model is challenging, but we address this by introdu…
Gradient-Based Program Synthesis with Neurally Interpreted Languages
AMLab, University of Amsterdam（阿姆斯特丹大学AMLab）；Ndea；Department of Computing Science, University of Alberta（阿尔伯塔大学计算机科学系）；Alberta Machine Intelligence Institute (Amii), Edmonton, Canada（阿尔伯塔机器智能研究所（Amii），加拿大爱德蒙顿）
Molecular design and synthesis planning are two critical steps in the process of molecular discovery that we propose to formulate as a single shared task of conditional synthetic pathway generation. We report an amortized approach to generate synthetic pathways as a Markov decision process conditioned on a target molecular embedding. This approach allows us to conduct synthesis planning in a bottom-up manner and design synthesizable molecules by decoding from optimized conditional codes, demonstrating the potential to solve both problems of design and synthesis simultaneously. The approach lev…
Harbin Engineering University（哈尔滨工程大学）；NC State University（北卡罗来纳州立大学）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
This paper proposes to learn hierarchical compositional AND-OR model for interpretable image synthesis by sparsifying the generator network. The proposed method adopts the scene-objects-parts-subparts-primitives hierarchy in image representation. A scene has different types (i.e., OR) each of which consists of a number of objects (i.e., AND). This can be recursively formulated across the scene-objects-parts-subparts hierarchy and is terminated at the primitive level (e.g., wavelets-like basis). To realize this AND-OR hierarchy in image synthesis, we learn a generator network that consists of t…
Finding an embedding space for a linear approximation of a nonlinear dynamical system enables efficient system identification and control synthesis. The Koopman operator theory lays the foundation for identifying the nonlinear-to-linear coordinate transformations with data-driven methods. Recently, researchers have proposed to use deep neural networks as a more expressive class of basis functions for calculating the Koopman operators. These approaches, however, assume a fixed dimensional state space; they are therefore not applicable to scenarios with a variable number of objects. In this pape…
