对齐少步生成模型具有挑战性，因为现有的对齐框架通常依赖于限制性假设：可处理的似然、特定的ODE/SDE求解器或特定的模型族。我们引入了FAV（Few-step Generative Models Alignment via Sample-based Variational Inference），这是一个通用的对齐框架，仅需要对生成器和参考分布的样本访问。我们将对齐视为从倾斜于参考分布的奖励倾斜分布中采样。我们利用Stein变分梯度下降作为基于样本的变分推断方案，并通过固定点回归将粒子更新摊销到生成器参数中。我们在两个领域评估了FAV：机器人操作和图像生成器对齐。在机器人操作的生成策略对齐中，FAV在56个离线RL任务和30个离线到在线RL任务中优于现有的策略提取基线。对于图像生成器对齐，FAV微调了多种少步骨干模型，包括GAN、漂移模型、一致性模型和流映射，从ImageNet-$256$扩展到1024$^2$文本到图像合成。代码可在https://github.com/Jaewoopudding/FAV获取。
英文摘要：
Aligning a few-step generative model is challenging, since existing alignment frameworks typically rely on restrictive assumptions: a tractable likelihood, a specific ODE/SDE solver, or a particular model family. We introduce FAV, Few-step Generative Models Alignment via Sample-based Variational Inference, a general alignment framework that requires only sample access to the generator and the reference distribution. We cast alignment as sampling from a reward-tilted distribution anchored to a reference distribution. We leverage Stein Variational Gradient Descent as a sample-based variational inference scheme and amortize its particle updates into the generator parameters via fixed-point regression. We evaluate FAV on two domains: robotics manipulation and image generator alignment. On generative policy alignment for robotic manipulation, FAV outperforms prevailing policy extraction baselines across 56 offline and 30 offline-to-online RL tasks. For image generator alignment, FAV fine-tunes diverse few-step backbones, including GAN, drifting model, consistency models, and flow maps, scaling from ImageNet-$256$ to 1024$^2$ text-to-image synthesis. Code is available at https://github.com/Jaewoopudding/FAV.
Fast and Robust Likelihood-Guided Diffusion Posterior Sampling with Amortized Variational Inference
Institute of Foundation Models, MBZUAI（基础模型研究所，MBZUAI）；Huawei Lagrange Mathematics and Computing Research Center, Paris, France（华为拉格朗日数学与计算研究中心，法国巴黎）；Mohamed Bin Zayed University of AI, Chair of the ML Department, EPITA, Laboratoire Recherche de l'EPITA（穆罕默德·本·扎耶德人工智能大学，机器学习系主任，EPITA，EPITA研究实验室）
Federated Variational Preference Alignment with Gumbel-Softmax Prior for Personalized User Preferences
Graduate School of AI, POSTECH, Pohang, Republic of Korea（POSTECH人工智能研究生院）；Department of CSE, POSTECH, Pohang, Republic of Korea（POSTECH计算机科学与工程系）；National AI Research Lab, Seoul, Republic of Korea（首尔国家人工智能研究实验室）
Domain Generalization under Conditional and Label Shifts via Variational Bayesian Inference
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Beth Israel Deaconess Medical Center（贝斯以色列女执事医疗中心）；National University of Singapore（新加坡国立大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
VaCDA: Variational Contrastive Alignment-based Scalable Human Activity Recognition
Department of Computer Science and Engineering, Bangladesh University of Engineering and Technology（计算机科学与工程系，孟加拉国工程与技术大学）；Department of Computer Science, Bowie State University（计算机科学系，布里沃州立大学）
Mix- and MoE-DPO: A Variational Inference Approach to Direct Preference Optimization
Department of Applied Mathematics and Statistics（应用数学与统计学系）；Stony Brook University（石溪大学）；Institute for Advanced Computational Science（先进计算科学研究所）；Center of Excellence in Wireless and Information Technology (CEWIT)（无线与信息技术卓越中心）；AI Innovation Institute（人工智能创新研究院）；Bloomberg（彭博）；Toronto, ON M5J 2S1（多伦多，ON M5J 2S1）；San Francisco, CA 94105（旧金山，CA 94105）；Bloomberg New York, NY 10022（纽约，NY 10022）
