arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.16570cs.LGcs.DS
收藏

通过二次奖励引导扩散模型:细粒度分析

Steering diffusion models with quadratic rewards: a fine-grained analysis

Ankur Moitra, Andrej Risteski, Dhruv Rohatgi

AI总结:

本文通过二次奖励引导扩散模型,分析了不同奖励函数下的采样效率及计算复杂性。

摘要:

在推理时间算法中,预训练模型被用作子程序来解决下游任务,已成为一种新兴范式。此类算法已被提出用于从逆问题、引导图像生成到推理等任务。然而,目前实际部署的方法都是启发式方法,具有各种失败模式——我们对这些启发式方法何时可以高效改进的理解非常有限。在本文中,我们考虑从奖励倾斜的扩散模型中采样——即从p*(x) ∝ p(x) exp(r(x))中采样——给定一个奖励函数r和预训练的扩散 oracle 用于p。我们对二次奖励r(x) = x^T A x + b^T x的任务的计算可处理性 进行细粒度分析。我们证明线性奖励倾斜总是可以高效采样——这是一个简单的结果,似乎在文献中未被注意到。我们使用这个作为基础块,结合一个概念上新的成分——Hubbard-Stratonovich变换——来提供一种高效算法,用于从低秩正定二次倾斜中采样,即r(x) = x^T A x,其中A是正定且秩为O(1)。对于负定倾斜,即r(x) = -x^T A x,其中A是正定的,我们证明即使A的秩为1(尽管具有指数级大的条目),该问题也是不可行的。

英文摘要:

Inference-time algorithms are an emerging paradigm in which pre-trained models are used as subroutines to solve downstream tasks. Such algorithms have been proposed for tasks ranging from inverse problems and guided image generation to reasoning. However, the methods currently deployed in practice are heuristics with a variety of failure modes -- and we have very little understanding of when these heuristics can be efficiently improved. In this paper, we consider the task of sampling from a reward-tilted diffusion model -- that is, sampling from $p^{\star}(x) \propto p(x) \exp(r(x))$ -- given a reward function $r$ and pre-trained diffusion oracle for $p$. We provide a fine-grained analysis of the computational tractability of this task for quadratic rewards $r(x) = x^\top A x + b^\top x$. We show that linear-reward tilts are always efficiently sampleable -- a simple result that seems to have gone unnoticed in the literature. We use this as a building block, along with a conceptually new ingredient -- the Hubbard-Stratonovich transform -- to provide an efficient algorithm for sampling from low-rank positive-definite quadratic tilts, i.e. $r(x) = x^\top A x$ where $A$ is positive-definite and of rank $O(1)$. For negative-definite tilts, i.e. $r(x) = - x^\top A x$ where $A$ is positive-definite, we prove that the problem is intractable even if $A$ is of rank 1 (albeit with exponentially-large entries).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑