用于图像生成的条件动力系统

中文摘要 AI 辅助

图像生成领域长期由运行在GPU上的深度生成模型主导，该范式的计算和能源成本引发了日益增长的可持续性担忧。新兴的非冯·诺依曼计算载体，包括量子、存算一体、光子和热力学平台，有望实现更高的效率，但现有大部分工作只是将传统神经架构移植到这些载体上，主要加速矩阵乘法等操作。这并未充分利用许多新兴计算载体的固有能力：向低能态的弛豫本身就能以可忽略的成本执行计算。我们开发了一类用于图像生成的连续动力系统，围绕这一基本操作构建，以更好地利用其计算能力。所提出的生成器在具有显式李雅普诺夫能量的动力学下演化内部状态，随后通过紧凑的、类别无关的解码器渲染所得状态。对于条件生成，我们引入能量倾斜：编程的成对交互在类别间保持固定且共享，而依赖类别的线性场在不重新编程交互阵列的情况下重塑能量。受伊辛模型启发的设计在CIFAR-10数据集上使用4096个自旋变量达到了9.71的干净FID。这些结果表明，降能动力学可直接作为生成式计算，并为超越GPU的高效生成任务提供了一条有前景的路径。

英文摘要

Image generation is dominated by deep generative models on GPUs, whose computational and energy costs raise sustainability concerns. Emerging non-von Neumann substrates, including quantum, compute-in-memory, photonic, and thermodynamic platforms, promise greater efficiency, yet most work ports conventional neural network architectures onto them and mainly accelerates operations such as matrix multiplication. This does not fully exploit a native capability of many hardware substrates: relaxation toward low-energy states can itself compute at little cost. Building on this primitive, we develop dynamical systems for image generation based on Ising spins and Kuramoto oscillators. The internal states of both models evolve under dynamics with an explicit Lyapunov energy, and a compact decoder renders the final states as images. For conditional generation, we introduce local conditioning: programmed interactions remain fixed, and one class-dependent bias per node reshapes the landscape, avoiding additional reprogramming and calibration costs. On CIFAR-10, the Ising and Kuramoto models reach clean-FID scores of $6.40$ and $5.37$, respectively, with $4096$ nodes. With a CMOS-compatible hardware realization, the dynamical-system core is projected to cost ${\sim}110.6$~nJ per image, and the full computation, even with the decoder on a GPU, outperforms all GPU baselines in latency and energy. These results suggest that energy-descending dynamics on non-von Neumann substrates can serve as generative computation, offering an efficient path beyond GPUs.

发表机构

• University of Rochester（罗切斯特大学）
• Rice University（莱斯大学）

机构由 AI 辅助整理，请以论文原文为准。