Distribution matching methods for language model alignment such as Generation with Distributional Control (GDC) and Distributional Policy Gradient (DPG) have not received the same level of attention in reinforcement learning from human feedback (RLHF) as contrastive methods such as Sequence Likelihood Calibration (SLiC), Direct Preference Optimization (DPO) and its variants. We identify high variance of the gradient estimate as the primary reason for the lack of success of these methods and propose a self-normalized baseline to reduce the variance. We further generalize the target distribution in DPG, GDC and DPO by using Bayes' rule to define the reward-conditioned posterior. The resulting approach, referred to as BRAIn - Bayesian Reward-conditioned Amortized Inference acts as a bridge between distribution matching methods and DPO and significantly outperforms prior art in summarization and Antropic HH tasks.
Molecular design and synthesis planning are two critical steps in the process of molecular discovery that we propose to formulate as a single shared task of conditional synthetic pathway generation. We report an amortized approach to generate synthetic pathways as a Markov decision process conditioned on a target molecular embedding. This approach allows us to conduct synthesis planning in a bottom-up manner and design synthesizable molecules by decoding from optimized conditional codes, demonstrating the potential to solve both problems of design and synthesis simultaneously. The approach lev…
FalconBC: Flow matching for Amortized inference of Latent-CONditioned physiologic Boundary Conditions
Department of Mechanical Engineering, Stanford University（斯坦福大学机械工程系）；Institute for Computational and Mathematical Engineering, Stanford University（斯坦福大学计算与数学工程研究所）；Bioengineering, Stanford University（斯坦福大学生物工程系）；Pediatric Cardiology, Stanford University（斯坦福大学儿科心脏病学）；Department of Applied and Computational Mathematics and Statistics, University of Notre Dame（诺丁汉大学应用与计算数学与统计学系）
Frequently, population studies feature pyramidally-organized data represented using Hierarchical Bayesian Models (HBM) enriched with plates. These models can become prohibitively large in settings such as neuroimaging, where a sample is composed of a functional MRI signal measured on 300 brain locations, across 4 measurement sessions, and 30 subjects, resulting in around 1 million latent parameters.Such high dimensionality hampers the usage of modern, expressive flow-based techniques.To infer parameter posterior distributions in this challenging class of problems, we designed a novel methodolo…
We propose a novel amortized variational inference scheme for an empirical Bayes meta-learning model, where model parameters are treated as latent variables. We learn the prior distribution over model parameters conditioned on limited training data using a variational autoencoder approach. Our framework proposes sharing the same amortized inference network between the conditional prior and variational posterior distributions over the model parameters. While the posterior leverages both the labeled support and query data, the conditional prior is based only on the labeled support data. We show …
Amortized Simulation-Based Inference in Generalized Bayes via Neural Posterior Estimation
Department of Statistics, University of Oxford, Oxford, United Kingdom（英国牛津大学统计系）；Department of Statistics, University of Auckland, Auckland, New Zealand（新西兰奥克兰大学统计系）
INRIA Saclay, \'Equipe Parietal Neurospin, B\ at 145, CEA Saclay, 91191 Gif sur Yvette, France
The total variation (TV) penalty, as many other analysis-sparsity problems, does not lead to separable factors or a proximal operatorwith a closed-form expression, such as soft thresholding for the $\ell\_1$ penalty. As a result, in a variational formulation of an inverse problem or statisticallearning estimation, it leads to challenging non-smooth optimization problemsthat are often solved with elaborate single-step first-order methods. When thedata-fit term arises from empirical measurements, as in brain imaging, it isoften very ill-conditioned and without simple structure. In this situation…
Exploiting learning algorithms under scarce data regimes is a limitation and a reality of the medical imaging field. In an attempt to mitigate the problem, we propose a data augmentation protocol based on generative adversarial networks. We condition the networks at a pixel-level (segmentation mask) and at a global-level information (acquisition environment or lesion type). Such conditioning provides immediate access to the image-label pairs while controlling global class specific appearance of the synthesized images. To stimulate synthesis of the features relevant for the segmentation task, a…
University of Sussex（萨塞克斯大学）；Brighton and Sussex Medical School（布莱顿和萨塞克斯医学院）；St. Vincent’s University Hospital（圣文森特大学医院）；University of Nottingham（诺丁汉大学）；Cardiff University（卡迪夫大学）
Streamlined qBOLD acquisitions enable experimentally straightforward observations of brain oxygen metabolism. $R_2^\prime$ maps are easily inferred; however, the Oxygen extraction fraction (OEF) and deoxygenated blood volume (DBV) are more ambiguously determined from the data. As such, existing inference methods tend to yield very noisy and underestimated OEF maps, while overestimating DBV. This work describes a novel probabilistic machine learning approach that can infer plausible distributions of OEF and DBV. Initially, we create a model that produces informative voxelwise prior distribution…
Stable Neural Decoding Across Sessions via Task-Conditioned Latent Alignment for Brain-Machine Interfaces
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Pittsburgh（匹兹堡大学）；Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
Functional MRI Time Series Generation via Wavelet-Based Image Transform and Spectral Flow Matching for Brain Disorder Identification
School of Information Technology, Monash University Malaysia（墨尔本大学马来西亚分校信息科技学院）；School of Engineering, Monash University Malaysia（墨尔本大学马来西亚分校工程学院）；Statistics Program, King Abdullah University of Science and Technology（国王阿卜杜勒·阿齐兹大学科学与技术学院统计学项目）
Due to the low signal-to-noise ratio and limited resolution of functional MRI data, and the high complexity of natural images, reconstructing a visual stimulus from human brain fMRI measurements is a challenging task. In this work, we propose a novel approach for this task, which we call Cortex2Image, to decode visual stimuli with high semantic fidelity and rich fine-grained detail. In particular, we train a surface-based convolutional network model that maps from brain response to semantic image features first (Cortex2Semantic). We then combine this model with a high-quality image generator (…
Adaptive Defense against Harmful Fine-Tuning for Large Language Models via Bayesian Data Scheduler
Nanyang Technological University（南洋理工大学）；Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；University of Central Florida（佛罗里达大学）；Tsinghua University（清华大学）
Monash University Malaysia（莫纳什大学马来西亚分校）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；University College London（伦敦大学学院）
Common measures of brain functional connectivity (FC) including covariance and correlation matrices are semi-positive definite (SPD) matrices residing on a cone-shape Riemannian manifold. Despite its remarkable success for Euclidean-valued data generation, use of standard generative adversarial networks (GANs) to generate manifold-valued FC data neglects its inherent SPD structure and hence the inter-relatedness of edges in real FC. We propose a novel graph-regularized manifold-aware conditional Wasserstein GAN (GR-SPD-GAN) for FC data generation on the SPD manifold that can preserve the globa…
Flow Matching with In-Context Priors for Out-of-Distribution Brain Dynamics
Hertie Institute for AI in Brain Health, University of Tübingen（赫蒂人工智能脑健康研究所，图宾根大学）；Tübingen AI Center, University of Tübingen（图宾根人工智能中心，图宾根大学）；Charité – Universitätsmedizin Berlin, Department of Psychiatry and Psychotherapy（柏林夏里特医学院，精神病学与心理治疗系）；German Center for Mental Health (DZPG), partner site Tübingen（德国心理健康中心（DZPG），图宾根合作站点）
Rochester Institute of Technology（罗切斯特理工学院）；The University of Utah（犹他大学）；Dalhousie University（达尔豪斯大学）
Clinical adoption of personalized virtual heart simulations faces challenges in model personalization and expensive computation. While an ideal solution is an efficient neural surrogate that at the same time is personalized to an individual subject, the state-of-the-art is either concerned with personalizing an expensive simulation model, or learning an efficient yet generic surrogate. This paper presents a completely new concept to achieve personalized neural surrogates in a single coherent framework of meta-learning (metaPNS). Instead of learning a single neural surrogate, we pursue the proc…
T2I-Diff: fMRI Signal Generation via Time-Frequency Image Transform and Classifier-Free Denoising Diffusion Models
School of Information Technology, Monash University Malaysia（莫纳什大学马来西亚分校信息科技学院）；Statistics Program, King Abdullah University of Science and Technology（国王阿卜杜勒阿齐兹大学科学与技术学院统计项目）
A Demographic-Conditioned Variational Autoencoder for fMRI Distribution Sampling and Removal of Confounds
Tulane University（杜兰大学）；Boys Town National Research Hospital（Boys Town国家研究医院）；Mind Research Network（Mind研究网络）；Georgia State University（佐治亚州立大学）；Georgia Institute of Technology（佐治亚理工学院）；Emory University（埃默里大学）
We propose a Model Predictive Control (MPC) method for collision-free navigation that uses amortized variational inference to approximate the distribution of optimal control sequences by training a normalizing flow conditioned on the start, goal and environment. This representation allows us to learn a distribution that accounts for both the dynamics of the robot and complex obstacle geometries. We can then sample from this distribution to produce control sequences which are likely to be both goal-directed and collision-free as part of our proposed FlowMPPI sampling-based MPC method. However, …
