HyperAlign:用于扩散模型高效测试时对齐的超网络
HyperAlign: Hypernetwork for Efficient Test-Time Alignment of Diffusion Models
AI总结:
HyperAlign通过训练超网络实现扩散模型测试时对齐,平衡对齐质量与计算效率,提升语义一致性和视觉质量。
摘要:
扩散模型对齐旨在通过增强文本提示的语义一致性和整体视觉质量来弥合生成输出与人类偏好的差距。现有对齐方法面临艰难的权衡:测试时方法虽能实现输入特定的适应性,但引入了显著的计算开销且容易欠优化;而微调方法则有风险奖励过优化和损失生成多样性。为解决这一问题,我们提出了HyperAlign,一个训练超网络以实现高效且有效的测试时对齐的框架。HyperAlign不直接修改潜在状态,而是动态生成输入和状态条件的低秩适应权重,以调节去噪轨迹向目标奖励方向发展。我们引入了多种粒度不同的HyperAlign变体以平衡对齐质量和计算效率。超网络通过偏好数据正则化的奖励目标进行优化,以缓解奖励黑客问题。我们评估了HyperAlign在多个生成范式中的表现,包括Stable Diffusion和FLUX,在语义一致性和视觉质量方面显著优于现有对齐方法。
英文摘要:
Diffusion model alignment aims to bridge the gap between generated outputs and human preferences by enhancing both semantic consistency with textual prompts and overall visual quality. Existing alignment methods face a challenging trade-off: test-time approaches enable input-specific adaptability but introduce significant computational overhead and tend to under-optimize, while fine-tuning approaches risk reward over-optimization and loss of generation diversity. To bridge this gap, we propose HyperAlign, a framework that trains a hypernetwork for efficient and effective test-time alignment. Instead of modifying latent states directly, HyperAlign dynamically generates input-and-state-conditioned low-rank adaptation weights to modulate the denoising trajectory toward target rewards. We introduce multiple HyperAlign variants of varying granularity to balance alignment quality and computational efficiency. The hypernetwork is optimized with a reward objective regularized by preference data to mitigate reward hacking. We evaluate HyperAlign across multiple generative paradigms, including Stable Diffusion and FLUX, where it significantly outperforms existing alignment methods in semantic consistency and visual quality.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。