The past decade has witnessed transformative applications of deep learning in various computational imaging, sensing and microscopy tasks. Due to the supervised learning schemes employed, these methods mostly depend on large-scale, diverse, and labeled training data. The acquisition and preparation of such training image datasets are often laborious and costly, also leading to biased estimation and limited generalization to new sample types. Here, we report a self-supervised learning model, termed GedankenNet, that eliminates the need for labeled or experimental training data, and demonstrate its effectiveness and superior generalization on hologram reconstruction tasks. Without prior knowledge about the sample types to be imaged, the self-supervised learning model was trained using a physics-consistency loss and artificial random images that are synthetically generated without any experiments or resemblance to real-world samples. After its self-supervised training, GedankenNet successfully generalized to experimental holograms of various unseen biological samples, reconstructing the phase and amplitude images of different types of objects using experimentally acquired test holograms. Without access to experimental data or knowledge of real samples of interest or their spatial features, GedankenNet's self-supervised learning achieved complex-valued image reconstructions that are consistent with the Maxwell's equations, and its output inference and object solutions accurately represent the wave propagation in free-space. GedankenNet framework also exhibits resilience to random, unknown perturbations in the physical forward model, including changes in the hologram distances, pixel size and illumination wavelength. This self-supervised learning of image reconstruction tasks creates new opportunities for various inverse problems in holography, microscopy and computational imaging fields.
Sparsity-Driven Parallel Imaging Consistency for Improved Self-Supervised MRI Reconstruction
Department of Electrical \& Computer Engineering, University of Minnesota, MN, USA Center for Magnetic Resonance Research, University of Minnesota, MN, USA
Self-supervised physics-informed generative networks for phase retrieval from a single X-ray hologram
NSLS-II, Brookhaven National Laboratory(布鲁克海文国家实验室国家同步辐射光源II);Institute of Materials Physics, Helmholtz-Zentrum Hereon(亥姆霍兹联合会Hereon材料物理研究所);Center for X-ray and Nano Science CXNS, Deutsches Elektronen-Synchrotron DESY(德国电子同步加速器X射线与纳米科学中心);Department of Physics, Universität Hamburg(汉堡大学物理系);Helmholtz Imaging, Deutsches Elektronen-Synchrotron DESY(德国电子同步加速器亥姆霍兹成像);Institute of Metallic Biomaterials, Helmholtz-Zentrum Hereon(亥姆霍兹联合会Hereon金属生物材料研究所)
Enhancing Digital Hologram Reconstruction Using Reverse-Attention Loss for Untrained Physics-Driven Deep Learning Models with Uncertain Distance
School of Computing(计算机学院);Clemson University(克莱姆森大学);School of Electrical and Computer Engineering(电气与计算机工程学院);Georgia Institute of Technology(佐治亚理工学院);The University of Arizona(亚利桑那大学);Medical University of South Carolina(南卡罗来纳医科大学)
An Intra- and Cross-frame Topological Consistency Scheme for Semi-supervised Atherosclerotic Coronary Plaque Segmentation
Xiamen University(厦门大学);University of Washington(华盛顿大学);Institute of Artificial Intelligence, Xiamen University(厦门大学人工智能研究院);National Institute for Data Science in Health and Medicine, Xiamen University(厦门大学健康与医学数据科学国家研究院)
Pseudo Label-Guided Data Fusion and Output Consistency for Semi-Supervised Medical Image Segmentation
college of physics and information engineering, University of Fuzhou University(福州大学物理与信息工程学院);Faculty of Applied Science, University of Macao Polytechnic University(澳门理工大学应用科学学院);University of Toronto University(多伦多大学)
Consistency-Based Semi-supervised Evidential Active Learning for Diagnostic Radiograph Classification
National University of Singapore(新加坡国立大学);Institute for Infocomm Research, A*STAR(新加坡科技研究局资讯通信研究院);Singapore University of Technology and Design(新加坡科技设计大学)
All-Around Real Label Supervision: Cyclic Prototype Consistency Learning for Semi-supervised Medical Image Segmentation
The Chinese University of Hong Kong(香港中文大学);Tencent Jarvis Lab(腾讯医疗健康(深圳)有限公司(腾讯天衍实验室));Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);The University of Hong Kong(香港大学);Tsinghua University(清华大学);Brigham and Women’s Hospital(布莱根妇女医院);Harvard Medical School(哈佛医学院)
半监督学习大大提高了医学图像分割,因为它减轻了获取昂贵的专家检查注释的沉重负担。特别是,基于一致性的方法已经吸引了更多的关注,其优越的性能,其中真正的标签只被用来监督他们的成对图像通过监督损失,而未标记的图像被利用,通过实施基于扰动的\textit{"unsupervised"}一致性没有显式的指导,从这些真正的标签。不过,直观来看,经过专家检验的真实标签包含了更可靠的监管信号。观察到这一点,我们提出了一个未探索但有趣的问题:我们可以通过显式真实标签监督来利用未标记的数据进行半监督训练吗?为此,我们放弃了以前的扰动为基础的一致性,但吸收非参数原型学习的本质。基于原型网络,提出了一种新的循环原型一致性学习框架,该框架由一个标记到未标记(L2U)原型前向过程和一个未标记到标记(U2L)原型后向过程构成.这两个过程通过鼓励更具鉴别力和紧凑的特征来协同增强分割网络。通过这种方式,我们的框架将以前的\textit{"unsupervised"}一致性转换为新的\textit{"supervised"}一致性,从而获得我们方法的\textit{"all-around real label supervision"}属性。对MRI脑肿瘤分割和CT肾脏分割的实验表明,我们的CPCL可以有效地利用未标记的数据,并优于其他最先进的半监督医学图像分割方法。
Mind the Context: Attention-Guided Weak-to-Strong Consistency for Enhanced Semi-Supervised Medical Image Segmentation
College of Engineering Huazhong Agricultural University Wuhan, China(华中农业大学工程学院);College of Informatics Huazhong Agricultural University Wuhan, China(华中农业大学信息学院)
Uncertainty-Guided Mutual Consistency Learning for Semi-Supervised Medical Image Segmentation
Beihang University(北京航空航天大学);Hefei Innovation Research Institute, Beihang University(北京航空航天大学合肥创新研究院);Beijing Advanced Innovation Centre for Biomedical Engineering(北京生物医学工程高精尖创新中心)
Dual-Consistency Semi-Supervised Learning with Uncertainty Quantification for COVID-19 Lesion Segmentation from CT Images
Imsight AI Research Lab(Imsight人工智能研究实验室);The Chinese University of Hong Kong(香港中文大学);The Hong Kong University of Science and Technology(香港科技大学);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院)
An Efficient Self-supervised Seismic Data Reconstruction Method Based on Self-Consistency Learning
The Key Laboratory of Earth Exploration & Information Techniques of Ministry Education(教育部地球探测与信息处理重点实验室);School of Geophysics, Chengdu University of Technology(成都理工大学地球物理学院);Chengdu University of Technology(成都理工大学)
Spatio-Temporal Structure Consistency for Semi-supervised Medical Image Classification
Shenzhen Research Institute of Big Data, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)深圳大数据研究院);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Dual-Decoder Consistency via Pseudo-Labels Guided Data Augmentation for Semi-Supervised Medical Image Segmentation
Fuzhou University(福州大学);Fujian Medical University Union Hospital(福建医科大学附属协和医院);Fujian Medical Ultrasound Research Institute(福建省医学超声研究所);Macao Polytechnic University(澳门理工大学)
SimMatchV2: Semi-Supervised Learning with Graph Consistency
The University of Sydney(悉尼大学);SenseTime Research(商汤科技研究院);The University of Tokyo(东京大学);State Grid Anhui Electric Power Research Institute(国网安徽电力研究院);University of Science and Technology of China(中国科学技术大学)
Dual Agreement Consistency Learning for Semi-Supervised Fetal Ultrasound Segmentation
Research Ireland Centre for Research Training in Machine Learning(爱尔兰研究机器学习研究中心);School of Medicine, University College Dublin(都柏林大学医学院);School of Computer Science, University College Dublin(都柏林大学计算机科学学院);School of Computer Science and Digital Technologies, Aston University(阿斯顿大学计算机科学与数字技术学院)