UltraDiff:将高保真先验迁移至压缩潜在空间以实现高分辨率图像生成
UltraDiff: Transferring High-Fidelity Priors to Compressed Latent Spaces for High-Resolution Image Generation
AI总结:
UltraDiff框架通过联合潜在扩散学习与边缘分数蒸馏,借助MixTeacher双分支DiT及向量场蒸馏,在f32压缩潜在空间实现接近f8的图像质量,效率较f8基准提升40倍以上。
摘要:
基于潜在扩散模型(LDM)的高分辨率合成面临计算成本与生成保真度之间的基本权衡,该权衡主要由潜在压缩因子f(例如f8表示f=8,f32表示f=32)决定。尽管激进压缩(f32)显著提升了效率,但与标准f8模型相比,其存在严重的信息丢失与模糊问题。本文提出UltraDiff框架,将该挑战重新表述为联合潜在扩散学习后接边缘分数蒸馏。我们引入MixTeacher,一种双分支DiT,其对联合f8-f32分布进行建模,利用共享注意力借助高保真f8先验解决压缩空间的模糊问题。关键地,我们在原生f32空间中提出向量场蒸馏,理论上这迫使学生学习教师轨迹的条件期望,有效过滤压缩伪影。在评估的1K至4K设置中,UltraDiff接近f8级别的质量,同时保留f32的效率。在单块A100 GPU上进行20次采样步骤时,其生成2K图像耗时6.2秒,比我们的f8基准快40倍以上。
英文摘要:
High-resolution synthesis with Latent Diffusion Models (LDMs) faces a fundamental trade-off between computational cost and generative fidelity. This trade off is largely governed by the latent compression factor f (e.g. f8 denotes f=8 and f32 f=32). While aggressive compression (f32) significantly improves efficiency, it suffers from severe information loss and ambiguity compared to standard f8 models. We propose UltraDiff, a framework that reformulates this challenge as joint latent diffusion learning followed by marginal score distillation. We introduce MixTeacher, a dual branch DiT that models the joint f8-f32 distribution, utilizing shared attention to resolve the ambiguity of the compressed space with high fidelity f8 priors. Crucially, we propose vector field distillation in the native f32 space. Theoretically, this compels the student to learn the conditional expectation of the teacher trajectory, effectively filtering out compression artifacts. Across the evaluated 1K to 4K settings, UltraDiff approaches f8 level quality while preserving f32 efficiency. With 20 sampling steps on a single A100 GPU, it generates a 2K image in 6.2 seconds over 40 times faster than our f8 baseline.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。