基于扩散先验的无噪声单步LoRA用于任务驱动的图像恢复
Noise-Free One-Step LoRA for Task-Driven Image Restoration with Diffusion Priors
浏览论文内容
中文总结 AI 辅助
研究针对退化图像影响下游任务的问题，提出基于扩散先验的无噪声单步LoRA方法用于任务驱动的图像恢复，通过特定适配模块及新训练策略，经实验验证该方法在多任务上优于先前方法且具泛化能力。
中文摘要 AI 辅助
退化图像不仅降低视觉质量，还损害下游高级视觉任务。任务驱动的图像恢复（TDIR）通过联合优化恢复质量和任务性能来解决此问题。近期工作表明预训练的扩散先验有益于TDIR，但基于扩散的恢复本质上是随机的，因为采样过程依赖随机噪声项，这会破坏任务一致性。本文表明，使用预训练扩散先验进行确定性、无噪声的单步前向传播可显著改善TDIR，但益处关键取决于适配模块：LoRA能带来持续增益，而ControlNet风格的条件设定则不能。这使得单步前向传播超越传统多步扩散TDIR基线。此外，我们引入一种保持任务的GAN训练策略，在不牺牲任务性能的情况下提高感知质量。在分类、分割和检测上的大量实验表明，相对于先前的TDIR方法有持续增益，我们还在真实世界退化图像和OCR上验证了泛化能力。
英文摘要
Degraded images not only reduce visual quality but also impair downstream high-level vision tasks. Task-driven image restoration (TDIR) addresses this issue by jointly optimizing restoration quality and task performance. Recent works show that pretrained diffusion priors benefit TDIR, yet diffusion-based restoration is inherently stochastic, as the sampling process depends on a random noise term, which can undermine task consistency. In this paper, we show that a deterministic, noise-free one-step forward pass with pretrained diffusion priors can substantially improve TDIR, but the benefit critically depends on the adaptation module: LoRA yields consistent gains, whereas ControlNet-style conditioning does not. This enables one-step forwarding that surpasses conventional multi-step diffusion TDIR baselines. Furthermore, we introduce a task-preserving GAN training strategy that improves perceptual quality without sacrificing task performance. Extensive experiments on classification, segmentation, and detection demonstrate consistent gains over prior TDIR methods, and we further validate generalization on real-world degraded images and OCR.
发表机构
- Seoul National University（首尔国立大学）
- IPAI, Seoul National University（首尔国立大学IPAI）
机构由 AI 辅助整理，请以论文原文为准。
