In this paper, we argue that iterative computation with diffusion models offers a powerful paradigm for not only generation but also visual perception tasks. We unify tasks such as depth estimation, optical flow, and amodal segmentation under the framework of image-to-image translation, and show how diffusion models benefit from scaling training and test-time compute for these perceptual tasks. Through a careful analysis of these scaling properties, we formulate compute-optimal training and inference recipes to scale diffusion models for visual perception tasks. Our models achieve competitive performance to state-of-the-art methods using significantly less data and compute. To access our code and models, see https://scaling-diffusion-perception.github.io .
ACPO: Anchor-Constrained Perceptual Optimization for Diffusion Models with No-Reference Quality Guidance
School of Electronic and Information Engineering, Anhui University(安徽大学电子与信息工程学院);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥国家综合科学中心人工智能研究院);School of Cyber Science and Technology, University of Science and Technology of China(中国科学技术大学网络科学与技术学院)
Perceptual Evaluation of GANs and Diffusion Models for Generating X-rays
Pontificia Universidad Católica de Chile(智利天主教大学);iHealth - Instituto Milenio en Ingeniería e Inteligencia Artificial para la Salud(iHealth - 毫米级工程与人工智能健康研究所);CENIA – Centro Nacional de Inteligencia Artificial(国家人工智能中心)
Active Sampling for Ultra-Low-Bit-Rate Video Compression via Conditional Controlled Diffusion
Department of Electrical and Computer Engineering, University of California San Diego(电子与计算机工程系,加州大学圣地亚哥分校);Department of Electrical and Systems Engineering, University of Pennsylvania(电子与系统工程系,宾夕法尼亚大学)
Generative Neural Video Compression via Video Diffusion Prior
School of Information and Communication Engineering, Communication University of China(信息与通信工程学院,通信大学);School of Computer Science, Peking University(计算机学院,北京大学)
Synthesis and Perceptual Scaling of High Resolution Naturalistic Images Using Stable Diffusion
Department of Psychology, Humboldt-Universität zu Berlin(柏林洪堡大学心理学系);Bernstein Center for Computational Neuroscience Berlin and Berlin Center for Advanced Neuroimaging, Charité Universitätsmedizin Berlin(柏林伯恩斯坦计算神经科学中心和柏林先进神经成像中心,查理医学院);Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany(马克斯·普朗克人类认知与脑科学研究所,莱比锡,德国);Max Planck School of Cognition, Leipzig, Germany(马克斯·普朗克认知学校,莱比锡,德国);Kavli Institute for Systems Neuroscience, Centre for Neural Computation, The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, Jebsen Centre for Alzheimer’s Disease, Norwegian University of Science and Technology, Trondheim, Norway(凯斯琳系统神经科学研究所,神经计算中心,埃吉和保罗·布雷athen及弗雷德·卡维利微电路中心,朱贝森阿尔茨海默病中心,挪威科学与技术大学,特罗姆瑟,挪威);Clinic of Neurology, Charité-Universitätsmedizin Berlin, Berlin, Germany(柏林查理医学院神经科诊所);Berlin Center for Advanced Neuroimaging, Charité-Universitätsmedizin Berlin, Berlin, Germany(柏林先进神经成像中心,查理医学院);Research Cluster of Excellence “Science of Intelligence”, Technische(卓越研究集群“智能科学”,技术)
Semantic Mismatch and Perceptual Degradation: A New Perspective on Image Editing Immunity
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences (CAS)(人工智能安全国家重点实验室,计算技术研究所,中国科学院);University of China Academy of Sciences(中国科学院大学);Center for Machine Vision and Signal Analysis, University of Oulu(信号分析中心,奥卢大学);School of Computer Science, China University of Geosciences(计算机科学学院,中国地质大学)
Recolour What Matters: Region-Aware Colour Editing via Token-Level Diffusion
Beijing University of Posts and Telecommunications(北京邮电大学);Beijing Key Laboratory of Multimodal Data Intelligent Perception and Governance(北京多模态数据智能感知与治理重点实验室)
End-to-End PET Image Reconstruction via a Posterior-Mean Diffusion Model
Department of Electrical and Computer Engineering, Rice University, Houston, USA(电气与计算机工程系,里士大学,美国休斯顿);Department of Imaging Physics, University of Texas MD Anderson Cancer Center, Houston, USA(影像物理系,德克萨斯大学MD安德森癌症中心,美国休斯顿)
High-Quality Exposure Correction with Diffusion-Based Image Generation Priors
Huazhong University of Science and Technology(华中科技大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);National University of Defense Technology(国防科技大学);Xiaomi Communications Company Ltd.(小米通讯有限公司)