This study introduces a novel method for inpainting normal maps using a generative adversarial network (GAN). Normal maps, often derived from a lightstage, are crucial in performance capture but can have obscured areas due to movement (e.g., by arms, hair, or props). Inpainting fills these missing areas with plausible data. Our approach extends previous general image inpainting techniques, employing a bow tie-like generator network and a discriminator network, with alternating training phases. The generator aims to synthesize images aligning with the ground truth and deceive the discriminator, which differentiates between real and processed images. Periodically, the discriminator undergoes retraining to enhance its ability to identify processed images. Importantly, our method adapts to the unique characteristics of normal map data, necessitating modifications to the loss function. We utilize a cosine loss instead of mean squared error loss for generator training. Limited training data availability, even with synthetic datasets, demands significant augmentation, considering the specific nature of the input data. This includes appropriate image flipping and in-plane rotations to accurately alter normal vectors. Throughout training, we monitored key metrics such as average loss, Structural Similarity Index Measure (SSIM), and Peak Signal-to-Noise Ratio (PSNR) for the generator, along with average loss and accuracy for the discriminator. Our findings suggest that the proposed model effectively generates high-quality, realistic inpainted normal maps, suitable for performance capture applications. These results establish a foundation for future research, potentially involving more advanced networks and comparisons with inpainting of source images used to create the normal maps.
University of Science and Technology of China（中国科学技术大学）
Feature Normalization (FN) is an important technique to help neural network training, which typically normalizes features across spatial dimensions. Most previous image inpainting methods apply FN in their networks without considering the impact of the corrupted regions of the input image on normalization, e.g. mean and variance shifts. In this work, we show that the mean and variance shifts caused by full-spatial FN limit the image inpainting network training and we propose a spatial region-wise normalization named Region Normalization (RN) to overcome the limitation. RN divides spatial pixel…
We present an end-to-end deep learning framework for indoor panoramic image inpainting. Although previous inpainting methods have shown impressive performance on natural perspective images, most fail to handle panoramic images, particularly indoor scenes, which usually contain complex structure and texture content. To achieve better inpainting quality, we propose to exploit both the global and local context of indoor panorama during the inpainting process. Specifically, we take the low-level layout edges estimated from the input panorama as a prior to guide the inpainting model for recovering …
Image inpainting has achieved fundamental advances with deep learning. However, almost all existing inpainting methods aim to process natural images, while few target Thermal Infrared (TIR) images, which have widespread applications. When applied to TIR images, conventional inpainting methods usually generate distorted or blurry content. In this paper, we propose a novel task -- Thermal Infrared Image Inpainting, which aims to reconstruct missing regions of TIR images. Crucially, we propose a novel deep-learning-based model TIR-Fill. We adopt the edge generator to complete the canny edges of b…
Department of Computer Science Norwegian University of Science and Technology（挪威科技大学计算机科学系）；Norwegian University of Science and Technology（挪威科技大学）
A regular convolution layer applying a filter in the same way over known and unknown areas causes visual artifacts in the inpainted image. Several studies address this issue with feature re-normalization on the output of the convolution. However, these models use a significant amount of learnable parameters for feature re-normalization, or assume a binary representation of the certainty of an output. We propose (layer-wise) feature imputation of the missing input values to a convolution. In contrast to learned feature re-normalization, our method is efficient and introduces a minimal number of…
Image inpainting is an old problem in computer vision that restores occluded regions and completes damaged images. In the case of facial image inpainting, most of the methods generate only one result for each masked image, even though there are other reasonable possibilities. To prevent any potential biases and unnatural constraints stemming from generating only one image, we propose a novel framework for diverse facial inpainting exploiting the embedding space of StyleGAN. Our framework employs pSp encoder and SeFa algorithm to identify semantic components of the StyleGAN embeddings and feed …
Harbin Institute of Technology（哈尔滨工业大学）；Peng Cheng Lab（鹏城实验室）；DAMO Academy at Alibaba Group（阿里巴巴达摩院）
For image inpainting, the convolutional neural networks (CNN) in previous methods often adopt standard convolutional operator, which treats valid pixels and holes indistinguishably. As a result, they are limited in handling irregular holes and tend to produce color-discrepant and blurry inpainting result. Partial convolution (PConv) copes with this issue by conducting masked convolution and feature re-normalization conditioned only on valid pixels, but the mask-updating is handcrafted and independent with image structural information. In this paper, we present an edge-guided learnable bidirect…
Facebook Artificial Intelligence（脸书人工智能）；University of Southern California（南加州大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
Deformable registration of magnetic resonance images between patients with brain tumors and healthy subjects has been an important tool to specify tumor geometry through location alignment and facilitate pathological analysis. Since tumor region does not match with any ordinary brain tissue, it has been difficult to deformably register a patients brain to a normal one. Many patient images are associated with irregularly distributed lesions, resulting in further distortion of normal tissue structures and complicating registration's similarity measure. In this work, we follow a multi-step contex…
The Chinese University of Hong Kong（香港中文大学）；YouTu Lab, Tencent（腾讯优图实验室）；SmartMore
Blind inpainting is a task to automatically complete visual contents without specifying masks for missing areas in an image. Previous works assume missing region patterns are known, limiting its application scope. In this paper, we relax the assumption by defining a new blind inpainting setting, making training a blind inpainting neural system robust against various unknown missing region patterns. Specifically, we propose a two-stage visual consistency network (VCN), meant to estimate where to fill (via masks) and generate what to fill. In this procedure, the unavoidable potential mask predic…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Adobe Research（奥多比研究院）；ByteDance AI Lab（字节跳动人工智能实验室）
We present a generative image inpainting system to complete images with free-form mask and guidance. The system is based on gated convolutions learned from millions of images without additional labelling efforts. The proposed gated convolution solves the issue of vanilla convolution that treats all input pixels as valid ones, generalizes partial convolution by providing a learnable dynamic feature selection mechanism for each channel at each spatial location across all layers. Moreover, as free-form masks may appear anywhere in images with any shape, global and local GANs designed for a single…
MAGIC: Few-Shot Mask-Guided Anomaly Inpainting with Prompt Perturbation, Spatially Adaptive Guidance, and Context Awareness
Department of AI Semiconductor Engineering, Hanyang University, Seoul, Korea（汉阳大学人工智能半导体工程系）；Department of Electronic Engineering, Hanyang University, Seoul, Korea（汉阳大学电子工程系）
Image inpainting aims to complete the missing or corrupted regions of images with realistic contents. The prevalent approaches adopt a hybrid objective of reconstruction and perceptual quality by using generative adversarial networks. However, the reconstruction loss and adversarial loss focus on synthesizing contents of different frequencies and simply applying them together often leads to inter-frequency conflicts and compromised inpainting. This paper presents WaveFill, a wavelet-based inpainting network that decomposes images into multiple frequency bands and fills the missing regions in e…
Harbin Institute of Technology（哈尔滨工业大学）；Peng Cheng Laboratory（鹏城实验室）；Baidu Inc.（百度公司）；Nankai University（南开大学）
Most convolutional network (CNN)-based inpainting methods adopt standard convolution to indistinguishably treat valid pixels and holes, making them limited in handling irregular holes and more likely to generate inpainting results with color discrepancy and blurriness. Partial convolution has been suggested to address this issue, but it adopts handcrafted feature re-normalization, and only considers forward mask-updating. In this paper, we present a learnable attention map module for learning feature renormalization and mask-updating in an end-to-end manner, which is effective in adapting to i…
Baidu Inc.（百度公司）；Institute of Automation, Chinese Academy of Sciences (CASIA)（中国科学院自动化研究所）
Inpainting arbitrary missing regions is challenging because learning valid features for various masked regions is nontrivial. Though U-shaped encoder-decoder frameworks have been witnessed to be successful, most of them share a common drawback of mask unawareness in feature extraction because all convolution windows (or regions), including those with various shapes of missing pixels, are treated equally and filtered with fixed learned kernels. To this end, we propose our novel mask-aware inpainting solution. Firstly, a Mask-Aware Dynamic Filtering (MADF) module is designed to effectively learn…
Huya Inc（虎牙公司）；City University of Hong Kong（香港城市大学）；Hunan University（湖南大学）；Tencent AI Lab（腾讯AI实验室）
We propose PD-GAN, a probabilistic diverse GAN for image inpainting. Given an input image with arbitrary hole regions, PD-GAN produces multiple inpainting results with diverse and visually realistic content. Our PD-GAN is built upon a vanilla GAN which generates images based on random noise. During image generation, we modulate deep features of input random noise from coarse-to-fine by injecting an initially restored image and the hole regions in multiple scales. We argue that during hole filling, the pixels near the hole boundary should be more deterministic (i.e., with higher probability tru…
Thinking inside the Convolution for Image Inpainting: Reconstructing Texture via Structure under Global and Local Side
School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机科学与信息工程学院）；Department of Automation, Tsinghua University（清华大学自动化系）；Lenovo Research（联想研究院）
Sun Yat-sen University（中山大学）；Petuum Inc（Petuum公司）；ByteDance AI Lab（字节跳动AI实验室）
Interactive fashion image manipulation, which enables users to edit images with sketches and color strokes, is an interesting research problem with great application value. Existing works often treat it as a general inpainting task and do not fully leverage the semantic structural information in fashion images. Moreover, they directly utilize conventional convolution and normalization layers to restore the incomplete image, which tends to wash away the sketch and color information. In this paper, we propose a novel Fashion Editing Generative Adversarial Network (FE-GAN), which is capable of ma…
The Chinese University of Hong Kong（香港中文大学）；MEGVII Technology（旷视科技）
Traditional convolution-based generative adversarial networks synthesize images based on hierarchical local operations, where long-range dependency relation is implicitly modeled with a Markov chain. It is still not sufficient for categories with complicated structures. In this paper, we characterize long-range dependence with attentive normalization (AN), which is an extension to traditional instance normalization. Specifically, the input feature map is softly divided into several regions based on its internal semantic similarity, which are respectively normalized. It enhances consistency bet…
Peking University（北京大学）；Microsoft Research Asia（微软亚洲研究院）；Microsoft Azure AI（微软Azure人工智能）
3D photography renders a static image into a video with appealing 3D visual effects. Existing approaches typically first conduct monocular depth estimation, then render the input frame to subsequent frames with various viewpoints, and finally use an inpainting model to fill those missing/occluded regions. The inpainting model plays a crucial role in rendering quality, but it is normally trained on out-of-domain data. To reduce the training and inference gap, we propose a novel self-supervised diffusion model as the inpainting module. Given a single input image, we automatically construct a tra…
Centre for Research and Technology Hellas（希腊研究与技术中心）
The task of transforming a furnished room image into a background-only is extremely challenging since it requires making large changes regarding the scene context while still preserving the overall layout and style. In order to acquire photo-realistic and structural consistent background, existing deep learning methods either employ image inpainting approaches or incorporate the learning of the scene layout as an individual task and leverage it later in a not fully differentiable semantic region-adaptive normalization module. To tackle these drawbacks, we treat scene layout generation as a fea…
X-ray Insights Unleashed: Pioneering the Enhancement of Multi-Label Long-Tail Data
Tencent Jarvis Lab（腾讯 Jarvis实验室）；School of Artificial Intelligence, Shenzhen University（深圳大学人工智能学院）；National University of Singapore（新加坡国立大学）；Westlake University（西湖大学）
Anomaly detection in computer vision is the task of identifying images which deviate from a set of normal images. A common approach is to train deep convolutional autoencoders to inpaint covered parts of an image and compare the output with the original image. By training on anomaly-free samples only, the model is assumed to not being able to reconstruct anomalous regions properly. For anomaly detection by inpainting we suggest it to be beneficial to incorporate information from potentially distant regions. In particular we pose anomaly detection as a patch-inpainting problem and propose to so…
Diff4MMLiTS: Advanced Multimodal Liver Tumor Segmentation via Diffusion-Based Image Synthesis and Alignment
Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen, China（电子与电气工程系，南方科技大学，深圳，中国）；Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, China（电气与电子工程系，香港大学，香港特别行政区，中国）；Department of Radiology, Zhongda Hospital, Medical School, Southeast University, Nanjing, China（放射科，中大医院，医学院，东南大学，南京，中国）；Jiaxing Research Institute, Southern University of Science and Technology, Jiaxing, China（嘉兴研究所，南方科技大学，嘉兴，中国）
Conditional diffusion models for guided anomaly detection in brain images using fluid-driven anomaly randomization
Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital and Harvard Medical School, Boston, USA（Athinoula A. Martinos 生物医学成像中心、麻省总医院和哈佛医学院，波士顿，美国）；Danish Research Centre for Magnetic Resonance, Department of Radiology and Nuclear Medicine（丹麦磁共振研究中心、放射学与核医学部门）；Copenhagen University Hospital– Amager and Hvidovre（哥本哈根大学医院–阿马格和赫维多尔）
CADD: Context aware disease deviations via restoration of brain images using normative conditional diffusion models
Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital and Harvard Medical School（阿提诺拉A.马丁纳生物医学成像中心，麻省总医院和哈佛医学院）；Hawkes Institute, University College London（霍克斯研究所，伦敦大学学院）；Mecha Health（Mecha健康）；Computer Science & Artificial Intelligence Lab, Massachusetts Institute of Technology（计算机科学与人工智能实验室，麻省理工学院）；Department of Radiology, the University of Tokyo（东京大学放射科）；Medical Engineering, Chiba University（千叶大学医学工程）；Department of Computational Diagnostic Radiology and Preventive Medicine, the University of Tokyo Hospital（东京大学医院计算诊断放射学与预防医学科）
Institute for Medical Imaging Technology, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200030, China（中国上海200030，上海交通大学生物医学工程学院医学影像技术研究所）；Department of Radiology and Biomedical Research Imaging Center (BRIC), University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA（美国北卡罗来纳州教堂山27599，北卡罗来纳大学教堂山分校放射学与生物医学研究成像中心（BRIC））；North China Electric Power University, Beijing, China（中国北京华北电力大学）；Shanghai United Imaging Intelligence Co., Ltd, Shanghai, China（中国上海上海联影智能科技有限公司）；Shanghai Cancer Center, Fudan University, Shanghai, China（中国上海复旦大学附属肿瘤医院）
Thermal ablation is a minimally invasive procedure for treat-ing small or unresectable tumors. Although CT is widely used for guiding ablation procedures, the contrast of tumors against surrounding normal tissues in CT images is often poor, aggravating the difficulty in accurate thermal ablation. In this paper, we propose a fast MR-CT image registration method to overlay a pre-procedural MR (pMR) image onto an intra-procedural CT (iCT) image for guiding the thermal ablation of liver tumors. By first using a Cycle-GAN model with mutual information constraint to generate synthesized CT (sCT) ima…
Autoencoder reconstructions are widely used for the task of unsupervised anomaly localization. Indeed, an autoencoder trained on normal data is expected to only be able to reconstruct normal features of the data, allowing the segmentation of anomalous pixels in an image via a simple comparison between the image and its autoencoder reconstruction. In practice however, local defects added to a normal image can deteriorate the whole reconstruction, making this segmentation challenging. To tackle the issue, we propose in this paper a new approach for projecting anomalous data on a autoencoder-lear…
The goal of our work is to complete the depth channel of an RGB-D image. Commodity-grade depth cameras often fail to sense depth for shiny, bright, transparent, and distant surfaces. To address this problem, we train a deep network that takes an RGB image as input and predicts dense surface normals and occlusion boundaries. Those predictions are then combined with raw depth observations provided by the RGB-D camera to solve for depths for all pixels, including those missing in the original observation. This method was chosen over others (e.g., inpainting depths directly) as the result of exten…
Diffusion-Based Data Augmentation for Medical Image Segmentation
School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）；Dept. of Engineering for Innovation Medicine, University of Verona（威尼斯大学创新医学工程系）
Euclid Quick Data Release (Q1). Active galactic nuclei identification using diffusion-based inpainting of Euclid VIS images
School of Physics, HH Wills Physics Laboratory, University of Bristol, Tyndall Avenue, Bristol, BS8 1TL, UK aff1 Max-Planck-Institut f\"ur Astronomie, K\"onigstuhl 17, 69117 Heidelberg, Germany aff2 SRON Netherlands Institute for Space Research, Landleven 12, 9747 AD, Groningen, The Netherlands aff3 Instituto de Astrof\' sica de Canarias, V\' a L\'actea, 38205 La Laguna, Tenerife, Spain aff4 Instituto de Astrof\'isica de Canarias (IAC)；Departamento de Astrof\'isica, Universidad de La Laguna (ULL), 38200, La Laguna, Tenerife, Spain aff5 Universit\'e PSL, Observatoire de Paris, Sorbonne Universit\'e, CNRS, LERMA, 75014, Paris, France aff6 Universit\'e Paris-Cit\'e, 5 Rue Thomas Mann, 75013, Paris, France aff7 School of Physics, Astronomy；Mathematics, University of Hertfordshire, College Lane, Hatfield AL10 9AB, UK aff8 Aspia Space, Falmouth, TR10 9TA, UK aff9 David A. Dunlap Department of Astronomy \& Astrophysics, University of Toronto, 50 St George Street, Toronto, Ontario M5S 3H4, Canada aff10 Jodrell Bank Centre for Astrophysics, Department of Physics；Astronomy, University of Manchester, Oxford Road, Manchester M13 9PL, UK aff11 Max Planck Institute for Extraterrestrial Physics, Giessenbachstr. 1, 85748 Garching, Germany aff12 Institute of Space Sciences (ICE, CSIC), Campus UAB, Carrer de Can Magrans, s/n, 08193 Barcelona, Spain aff13 Institut d'Estudis Espacials de Catalunya (IEEC), Edifici RDIT, Campus UPC, 08860 Castelldefels, Barcelona, Spain aff14 Centro de Astrof\' sica da Universidade do Porto, Rua das Estrelas, 4150-762 Porto, Portugal aff15 Instituto de Astrof\'isica e Ci\ encias do Espa c o, Universidade do Porto, CAUP, Rua das Estrelas, PT4150-762 Porto, Portugal aff16 Dipartimento di Fisica e Astronomia "Augusto Righi" - Alma Mater Studiorum Universit\`a di Bologna, via Piero Gobetti 93/2, 40129 Bologna, Italy aff17 INAF-Osservatorio di Astrofisica e Scienza dello Spazio di Bologna, Via Piero Gobetti 93/3, 40129 Bologna, Italy aff18 Department of Mathematics；Physics, Roma Tre University, Via della Vasca Navale 84, 00146 Rome, Italy aff19 INAF-Osservatorio Astronomico di Roma, Via Frascati 33, 00078 Monteporzio Catone, Italy aff20 Universit\'e Paris-Saclay, CNRS, Institut d'astrophysique spatiale, 91405, Orsay, France aff21 ESAC/ESA, Camino Bajo del Castillo, s/n., Urb. Villafranca del Castillo, 28692 Villanueva de la Ca\ nada, Madrid, Spain aff22 INAF-Osservatorio Astronomico di Brera, Via Brera 28, 20122 Milano, Italy aff23 Universit\'e Paris-Saclay, Universit\'e Paris Cit\'e, CEA, CNRS, AIM, 91191, Gif-sur-Yvette, France aff24 IFPU, Institute for Fundamental Physics of the Universe, via Beirut 2, 34151 Trieste, Italy aff25 INAF-Osservatorio Astronomico di Trieste, Via G. B. Tiepolo 11, 34143 Trieste, Italy aff26 INFN, Sezione di Trieste, Via Valerio 2, 34127 Trieste TS, Italy aff27 SISSA, International School for Advanced Studies, Via Bonomea 265, 34136 Trieste TS, Italy aff28 Dipartimento di Fisica e Astronomia, Universit\`a di Bologna, Via Gobetti 93/2, 40129 Bologna, Italy aff29 INFN-Sezione di Bologna, Viale Berti Pichat 6/2, 40127 Bologna, Italy aff30 Space Science Data Center, Italian Space Agency, via del Politecnico snc, 00133 Roma, Italy aff31 Dipartimento di Fisica, Universit\`a di Genova, Via Dodecaneso 33, 16146, Genova, Italy aff32 INFN-Sezione di Genova, Via Dodecaneso 33, 16146, Genova, Italy aff33 Department of Physics "E. Pancini", University Federico II, Via Cinthia 6, 80126, Napoli, Italy aff34 INAF-Osservatorio Astronomico di Capodimonte, Via Moiariello 16, 80131 Napoli, Italy aff35 Faculdade de Ci\ encias da Universidade do Porto, Rua do Campo de Alegre, 4150-007 Porto, Portugal aff36 Dipartimento di Fisica, Universit\`a degli Studi di Torino, Via P. Giuria 1, 10125 Torino, Italy aff37 INFN-Sezione di Torino, Via P. Giuria 1, 10125 Torino, Italy aff38 INAF-Osservatorio Astrofisico di Torino, Via Osservatorio 20, 10025 Pino Torinese (TO), Italy aff39 European Space Agency/ESTEC, Keplerlaan 1, 2201 AZ Noordwijk, The Netherlands aff40 Institute Lorentz, Leiden University, Niels Bohrweg 2, 2333 CA Leiden, The Netherlands aff41 Leiden Observatory, Leiden University, Einsteinweg 55, 2333 CC Leiden, The Netherlands aff42 INAF-IASF Milano, Via Alfonso Corti 12, 20133 Milano, Italy aff43 Centro de Investigaciones Energ\'eticas, Medioambientales y Tecnol\'ogicas (CIEMAT), Avenida Complutense 40, 28040 Madrid, Spain aff44 Port d'Informaci\' o Cient\' i fica, Campus UAB, C. Albareda s/n, 08193 Bellaterra (Barcelona), Spain aff45 INFN section of Naples, Via Cinthia 6, 80126, Napoli, Italy aff46 Institute for Astronomy, University of Hawaii, 2680 Woodlawn Drive, Honolulu, HI 96822, USA aff47 Dipartimento di Fisica e Astronomia "Augusto Righi" - Alma Mater Studiorum Universit\`a di Bologna, Viale Berti Pichat 6/2, 40127 Bologna, Italy aff48 Institute for Astronomy, University of Edinburgh, Royal Observatory, Blackford Hill, Edinburgh EH9 3HJ, UK aff49 European Space Agency/ESRIN, Largo Galileo Galilei 1, 00044 Frascati, Roma, Italy aff50 Universit\'e Claude Bernard Lyon 1, CNRS/IN2P3, IP2I Lyon, UMR 5822, Villeurbanne, F-69100, France aff51 Aix-Marseille Universit\'e, CNRS, CNES, LAM, Marseille, France aff52 Institut de Ci\` e ncies del Cosmos (ICCUB), Universitat de Barcelona (IEEC-UB), Mart\' i i Franqu\` e s 1, 08028 Barcelona, Spain aff53 Instituci\'o Catalana de Recerca i Estudis Avan c ats (ICREA), Passeig de Llu\' s Companys 23, 08010 Barcelona, Spain aff54 UCB Lyon 1, CNRS/IN2P3, IUF, IP2I Lyon, 4 rue Enrico Fermi, 69622 Villeurbanne, France aff55 Mullard Space Science Laboratory, University College London, Holmbury St Mary, Dorking, Surrey RH5 6NT, UK aff56 Departamento de F\'isica, Faculdade de Ci\ encias, Universidade de Lisboa, Edif\'icio C8, Campo Grande, PT1749-016 Lisboa, Portugal aff57 Instituto de Astrof\'isica e Ci\ encias do Espa c o, Faculdade de Ci\ encias, Universidade de Lisboa, Campo Grande, 1749-016 Lisboa, Portugal aff58 Department of Astronomy, University of Geneva, ch. d'Ecogia 16, 1290 Versoix, Switzerland aff59 INFN-Padova, Via Marzolo 8, 35131 Padova, Italy aff60 Aix-Marseille Universit\'e, CNRS/IN2P3, CPPM, Marseille, France aff61 INAF-Istituto di Astrofisica e Planetologia Spaziali, via del Fosso del Cavaliere, 100, 00100 Roma, Italy aff62 Universit\"ats-Sternwarte M\"unchen, Fakult\"at f\"ur Physik, Ludwig-Maximilians-Universit\"at M\"unchen, Scheinerstrasse 1, 81679 M\"unchen, Germany aff63 INAF-Osservatorio Astronomico di Padova, Via dell'Osservatorio 5, 35122 Padova, Italy aff64 Institute of Theoretical Astrophysics, University of Oslo, P.O. Box 1029 Blindern, 0315 Oslo, Norway aff65 Department of Physics, Lancaster University, Lancaster, LA1 4YB, UK aff66 Felix Hormuth Engineering, Goethestr. 17, 69181 Leimen, Germany aff67 Technical University of Denmark, Elektrovej 327, 2800 Kgs. Lyngby, Denmark aff68 Cosmic Dawn Center (DAWN), Denmark aff69 Institut d'Astrophysique de Paris, UMR 7095, CNRS；Sorbonne Universit\'e, 98 bis boulevard Arago, 75014 Paris, France aff70 NASA Goddard Space Flight Center, Greenbelt, MD 20771, USA aff71 Department of Physics；Helsinki Institute of Physics, Gustaf H\"allstr\"omin katu 2, 00014 University of Helsinki, Finland aff72 Jet Propulsion Laboratory, California Institute of Technology, 4800 Oak Grove Drive, Pasadena, CA, 91109, USA aff73 Department of Physics, P.O. Box 64, 00014 University of Helsinki, Finland aff74 Helsinki Institute of Physics, Gustaf H \"a llstr \"o min katu 2, University of Helsinki, Helsinki, Finland aff75 Centre de Calcul de l'IN2P3/CNRS, 21 avenue Pierre de Coubertin 69627 Villeurbanne Cedex, France aff76 Laboratoire d'etude de l'Univers et des phenomenes eXtremes, Observatoire de Paris, Universit\'e PSL, Sorbonne Universit\'e, CNRS, 92190 Meudon, France aff77 SKA Observatory, Jodrell Bank, Lower Withington, Macclesfield, Cheshire SK11 9FT, UK aff78 Dipartimento di Fisica "Aldo Pontremoli", Universit\`a degli Studi di Milano, Via Celoria 16, 20133 Milano, Italy aff79 INFN-Sezione di Milano, Via Celoria 16, 20133 Milano, Italy aff80 Universit\"at Bonn, Argelander-Institut f\"ur Astronomie, Auf dem H\"ugel 71, 53121 Bonn, Germany aff81 INFN-Sezione di Roma, Piazzale Aldo Moro, 2 - c/o Dipartimento di Fisica, Edificio G. Marconi, 00185 Roma, Italy aff82 Department of Physics, Institute for Computational Cosmology, Durham University, South Road, Durham, DH1 3LE, UK aff83 Universit\'e C\ o te d'Azur, Observatoire de la C\ o te d'Azur, CNRS, Laboratoire Lagrange, Bd de l'Observatoire, CS 34229, 06304 Nice cedex 4, France aff84 Universit\'e Paris Cit\'e, CNRS, Astroparticule et Cosmologie, 75013 Paris, France aff85 CNRS-UCB International Research Laboratory, Centre Pierre Binetruy, IRL2007, CPB-IN2P3, Berkeley, USA aff86 University of Applied Sciences；Arts of Northwestern Switzerland, School of Engineering, 5210 Windisch, Switzerland aff87 Institute of Physics, Laboratory of Astrophysics, Ecole Polytechnique F\'ed\'erale de Lausanne (EPFL), Observatoire de Sauverny, 1290 Versoix, Switzerland aff88 Aurora Technology for European Space Agency (ESA), Camino bajo del Castillo, s/n, Urbanizacion Villafranca del Castillo, Villanueva de la Ca\ nada, 28692 Madrid, Spain aff89 Institut de F\' i sica d'Altes Energies (IFAE), The Barcelona Institute of Science；Technology, Campus UAB, 08193 Bellaterra (Barcelona), Spain aff90 DARK, Niels Bohr Institute, University of Copenhagen, Jagtvej 155, 2200 Copenhagen, Denmark aff91 Waterloo Centre for Astrophysics, University of Waterloo, Waterloo, Ontario N2L 3G1, Canada aff92 Department of Physics；Astronomy, University of Waterloo, Waterloo, Ontario N2L 3G1, Canada aff93 Perimeter Institute for Theoretical Physics, Waterloo, Ontario N2L 2Y5, Canada aff94 Centre National d'Etudes Spatiales -- Centre spatial de Toulouse, 18 avenue Edouard Belin, 31401 Toulouse Cedex 9, France aff95 Institute of Space Science, Str. Atomistilor, nr. 409 M a gurele, Ilfov, 077125, Romania aff96 Consejo Superior de Investigaciones Cientificas, Calle Serrano 117, 28006 Madrid, Spain aff97 Universidad de La Laguna, Departamento de Astrof\' sica, 38206 La Laguna, Tenerife, Spain aff98 Dipartimento di Fisica e Astronomia "G. Galilei", Universit\`a di Padova, Via Marzolo 8, 35131 Padova, Italy aff99 Departamento de F\'isica, FCFM, Universidad de Chile, Blanco Encalada 2008, Santiago, Chile aff100 Universit\"at Innsbruck, Institut f\"ur Astro- und Teilchenphysik, Technikerstr. 25/8, 6020 Innsbruck, Austria aff101 Satlantis, University Science Park, Sede Bld 48940, Leioa-Bilbao, Spain aff102 Centre for Electronic Imaging, Open University, Walton Hall, Milton Keynes, MK7 6AA, UK aff103 Instituto de Astrof\'isica e Ci\ encias do Espa c o, Faculdade de Ci\ encias, Universidade de Lisboa, Tapada da Ajuda, 1349-018 Lisboa, Portugal aff104 Cosmic Dawn Center (DAWN) aff105 Niels Bohr Institute, University of Copenhagen, Jagtvej 128, 2200 Copenhagen, Denmark aff106 Universidad Polit\'ecnica de Cartagena, Departamento de Electr\'onica y Tecnolog\'ia de Computadoras, Plaza del Hospital 1, 30202 Cartagena, Spain aff107 Institut de Recherche en Astrophysique et Plan\'etologie (IRAP), Universit\'e de Toulouse, CNRS, UPS, CNES, 14 Av. Edouard Belin, 31400 Toulouse, France aff108 INFN-Bologna, Via Irnerio 46, 40126 Bologna, Italy aff109 Kapteyn Astronomical Institute, University of Groningen, PO Box 800, 9700 AV Groningen, The Netherlands aff110 Infrared Processing；Analysis Center, California Institute of Technology, Pasadena, CA 91125, USA aff111 Dipartimento di Fisica e Scienze della Terra, Universit\`a degli Studi di Ferrara, Via Giuseppe Saragat 1, 44122 Ferrara, Italy aff112 Istituto Nazionale di Fisica Nucleare, Sezione di Ferrara, Via Giuseppe Saragat 1, 44122 Ferrara, Italy aff113 INAF, Istituto di Radioastronomia, Via Piero Gobetti 101, 40129 Bologna, Italy aff114 Department of Physics, Oxford University, Keble Road, Oxford OX1 3RH, UK aff115 INAF - Osservatorio Astronomico di Brera, via Emilio Bianchi 46, 23807 Merate, Italy aff116 INAF-Osservatorio Astronomico di Brera, Via Brera 28, 20122 Milano, Italy；INFN-Sezione di Genova, Via Dodecaneso 33, 16146, Genova, Italy aff117 Institut d'Astrophysique de Paris, 98bis Boulevard Arago, 75014, Paris, France aff118 ICL, Junia, Universit\'e Catholique de Lille, LITL, 59000 Lille, France aff119 Instituto de F\'isica Te\'orica UAM-CSIC, Campus de Cantoblanco, 28049 Madrid, Spain aff120 CERCA/ISO, Department of Physics, Case Western Reserve University, 10900 Euclid Avenue, Cleveland, OH 44106, USA aff121 Technical University of Munich, TUM School of Natural Sciences, Physics Department, James-Franck-Str. 1, 85748 Garching, Germany aff122 Max-Planck-Institut f\"ur Astrophysik, Karl-Schwarzschild-Str. 1, 85748 Garching, Germany aff123 Laboratoire Univers et Th\'eorie, Observatoire de Paris, Universit\'e PSL, Universit\'e Paris Cit\'e, CNRS, 92190 Meudon, France aff124 Departamento de F \' sica Fundamental. Universidad de Salamanca. Plaza de la Merced s/n. 37008 Salamanca, Spain aff125 Universit\'e de Strasbourg, CNRS, Observatoire astronomique de Strasbourg, UMR 7550, 67000 Strasbourg, France aff126 Center for Data-Driven Discovery, Kavli IPMU (WPI), UTIAS, The University of Tokyo, Kashiwa, Chiba 277-8583, Japan aff127 Dipartimento di Fisica - Sezione di Astronomia, Universit\`a di Trieste, Via Tiepolo 11, 34131 Trieste, Italy aff128 ICSC - Centro Nazionale di Ricerca in High Performance Computing, Big Data e Quantum Computing, Via Magnanelli 2, Bologna, Italy aff129 California Institute of Technology, 1200 E California Blvd, Pasadena, CA 91125, USA aff130 Departamento F\'isica Aplicada, Universidad Polit\'ecnica de Cartagena, Campus Muralla del Mar, 30202 Cartagena, Murcia, Spain aff131 Instituto de F\'isica de Cantabria, Edificio Juan Jord\'a, Avenida de los Castros, 39005 Santander, Spain aff132 INFN, Sezione di Lecce, Via per Arnesano, CP-193, 73100, Lecce, Italy aff133 Department of Mathematics；Physics E. De Giorgi, University of Salento, Via per Arnesano, CP-I93, 73100, Lecce, Italy aff134 INAF-Sezione di Lecce, c/o Dipartimento Matematica e Fisica, Via per Arnesano, 73100, Lecce, Italy aff135 CEA Saclay, DFR/IRFU, Service d'Astrophysique, Bat. 709, 91191 Gif-sur-Yvette, France aff136 Institute of Cosmology；Gravitation, University of Portsmouth, Portsmouth PO1 3FX, UK aff137 Instituto de Astrof\' sica de Canarias, c/ Via Lactea s/n, La Laguna 38200, Spain. Departamento de Astrof\' sica de la Universidad de La Laguna, Avda. Francisco Sanchez, La Laguna, 38200, Spain aff138 Caltech/IPAC, 1200 E. California Blvd., Pasadena, CA 91125, USA aff139 Ruhr University Bochum, Faculty of Physics；Astronomy, Astronomical Institute (AIRUB), German Centre for Cosmological Lensing (GCCL), 44780 Bochum, Germany aff140 Department of Physics；Astronomy, Vesilinnantie 5, 20014 University of Turku, Finland aff141 Serco for European Space Agency (ESA), Camino bajo del Castillo, s/n, Urbanizacion Villafranca del Castillo, Villanueva de la Ca\ nada, 28692 Madrid, Spain aff142 ARC Centre of Excellence for Dark Matter Particle Physics, Melbourne, Australia aff143 Centre for Astrophysics \& Supercomputing, Swinburne University of Technology, Hawthorn, Victoria 3122, Australia aff144 Department of Physics；Astronomy, University of the Western Cape, Bellville, Cape Town, 7535, South Africa aff145 DAMTP, Centre for Mathematical Sciences, Wilberforce Road, Cambridge CB3 0WA, UK aff146 Kavli Institute for Cosmology Cambridge, Madingley Road, Cambridge, CB3 0HA, UK aff147 Department of Astrophysics, University of Zurich, Winterthurerstrasse 190, 8057 Zurich, Switzerland aff148 Department of Physics, Centre for Extragalactic Astronomy, Durham University, South Road, Durham, DH1 3LE, UK aff149 Institute for Theoretical Particle Physics；Cosmology (TTK), RWTH Aachen University, 52056 Aachen, Germany aff150 IRFU, CEA, Universit\'e Paris-Saclay 91191 Gif-sur-Yvette Cedex, France aff151 Univ. Grenoble Alpes, CNRS, Grenoble INP, LPSC-IN2P3, 53, Avenue des Martyrs, 38000, Grenoble, France aff152 INAF-Osservatorio Astrofisico di Arcetri, Largo E. Fermi 5, 50125, Firenze, Italy aff153 Dipartimento di Fisica, Sapienza Universit\`a di Roma, Piazzale Aldo Moro 2, 00185 Roma, Italy aff154 HE Space for European Space Agency (ESA), Camino bajo del Castillo, s/n, Urbanizacion Villafranca del Castillo, Villanueva de la Ca\ nada, 28692 Madrid, Spain aff155 Theoretical astrophysics, Department of Physics；Astronomy, Uppsala University, Box 515, 751 20 Uppsala, Sweden aff156 Mathematical Institute, University of Leiden, Einsteinweg 55, 2333 CA Leiden, The Netherlands aff157 School of Physics \& Astronomy, University of Southampton, Highfield Campus, Southampton SO17 1BJ, UK aff158 Institute of Astronomy, University of Cambridge, Madingley Road, Cambridge CB3 0HA, UK aff159 Department of Astrophysical Sciences, Peyton Hall, Princeton University, Princeton, NJ 08544, USA aff160 Center for Computational Astrophysics, Flatiron Institute, 162 5th Avenue, 10010, New York, NY, USA aff161
Deep neural networks, in particular convolutional neural networks, have become highly effective tools for compressing images and solving inverse problems including denoising, inpainting, and reconstruction from few and noisy measurements. This success can be attributed in part to their ability to represent and generate natural images well. Contrary to classical tools such as wavelets, image-generating deep neural networks have a large number of parameters---typically a multiple of their output dimension---and need to be trained on large datasets. In this paper, we propose an untrained simple i…
The University of Queensland（昆士兰大学）；Australian e-Health Research Centre（澳大利亚电子健康研究中心）；Commonwealth Scientific and Industrial Research Organisation（联邦科学与工业研究组织）
In medical image analysis, automated segmentation of multi-component anatomical structures, which often have a spectrum of potential anomalies and pathologies, is a challenging task. In this work, we develop a multi-step approach using U-Net-based neural networks to initially detect anomalies (bone marrow lesions, bone cysts) in the distal femur, proximal tibia and patella from 3D magnetic resonance (MR) images of the knee in individuals with varying grades of osteoarthritis. Subsequently, the extracted data are used for downstream tasks involving semantic segmentation of individual bone and c…
