arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2411.08034cs.CVcs.AI
收藏

扩散模型在感知任务中的缩放特性

Scaling Properties of Diffusion Models for Perceptual Tasks

University of California, Berkeley(加州大学伯克利分校)

Rahul Ravishankar, Zeeshan Patel, Jathushan Rajasegaran, Jitendra Malik

AI总结:

本文提出扩散模型的迭代计算可用于视觉感知任务,在图像到图像翻译框架下统一深度估计等任务,并制定计算最优训练与推理方案,以更少数据和计算达到先进性能。

摘要:

本文认为,扩散模型的迭代计算不仅为生成任务,也为视觉感知任务提供了强大的范式。我们在图像到图像翻译的框架下统一了深度估计、光流和amodal segmentation等任务,并展示了扩散模型在这些感知任务中如何从扩展训练和测试时计算中受益。通过对这些缩放特性的仔细分析,我们制定了计算最优的训练和推理方案,以扩展用于视觉感知任务的扩散模型。我们的模型在使用显著更少的数据和计算量的情况下,取得了与最先进方法相媲美的性能。要访问我们的代码和模型,请参见 https://scaling-diffusion-perception.github.io 。

英文摘要:

In this paper, we argue that iterative computation with diffusion models offers a powerful paradigm for not only generation but also visual perception tasks. We unify tasks such as depth estimation, optical flow, and amodal segmentation under the framework of image-to-image translation, and show how diffusion models benefit from scaling training and test-time compute for these perceptual tasks. Through a careful analysis of these scaling properties, we formulate compute-optimal training and inference recipes to scale diffusion models for visual perception tasks. Our models achieve competitive performance to state-of-the-art methods using significantly less data and compute. To access our code and models, see https://scaling-diffusion-perception.github.io .

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑