Recently, Neural Radiance Field (NeRF) has shown great success in rendering novel-view images of a given scene by learning an implicit representation with only posed RGB images. NeRF and relevant neural field methods (e.g., neural surface representation) typically optimize a point-wise loss and make point-wise predictions, where one data point corresponds to one pixel. Unfortunately, this line of research failed to use the collective supervision of distant pixels, although it is known that pixels in an image or scene can provide rich structural information. To the best of our knowledge, we are the first to design a nonlocal multiplex training paradigm for NeRF and relevant neural field methods via a novel Stochastic Structural SIMilarity (S3IM) loss that processes multiple data points as a whole set instead of process multiple inputs independently. Our extensive experiments demonstrate the unreasonable effectiveness of S3IM in improving NeRF and neural surface representation for nearly free. The improvements of quality metrics can be particularly significant for those relatively difficult tasks: e.g., the test MSE loss unexpectedly drops by more than 90% for TensoRF and DVGO over eight novel view synthesis tasks; a 198% F-score gain and a 64% Chamfer $L_{1}$ distance reduction for NeuS over eight surface reconstruction tasks. Moreover, S3IM is consistently robust even with sparse inputs, corrupted images, and dynamic scenes.
Objective Quality Assessment of Point Clouds Using Multi-scale Implicit Structural Similarity
School of Electronics and Information, Northwestern Polytechnical University(电子与信息学院,西北工业大学);Department of Computer Science, City University of Hong Kong(计算机科学系,香港城市大学);School of Telecommunication Engineering, Xidian University(电信工程学院,西安电子科技大学)
MSDS: Deep Structural Similarity with Multiscale Representation
Fujian Key Lab for Intelligent Processing and Wireless Communication of Media Information, College of Physics and Information Engineering, Fuzhou University(福建省智能信息处理与无线通信重点实验室,物理与信息工程学院,福州大学);School of Computing and Information Science, Fuzhou Institute of Technology(计算机与信息科学学院,福州理工学院)
Detection of Alzheimer's Disease Using Graph-Regularized Convolutional Neural Network Based on Structural Similarity Learning of Brain Magnetic Resonance Images
Lakehead University(湖首大学);University of Calgary(卡尔加里大学)
Application of Structural Similarity Analysis of Visually Salient Areas and Hierarchical Clustering in the Screening of Similar Wireless Capsule Endoscopic Images
借助于注意力机制自适应地对图像特征进行加权,最近的基于深度学习的高级模型鼓励预测结果以尽可能大的可预测区域逼近地面真实掩模,从而实现最先进的性能。然而,这些方法没有足够的注意到小的区域容易出现错误预测。以这种方式,由于存在具有不可区分的前景和背景的区域以及具有复杂或精细结构的区域,因此仍然难以准确地定位显著对象。为了解决这些问题,我们提出了一种新的卷积神经网络的净化机制和结构相似性损失。具体来说,为了更好地定位初步显著对象,首先引入了基于空间和通道注意力机制的促进注意,以促进对显著区域的注意。随后,为了恢复可以被视为一个模型的易出错区域的不可区分区域,我们提出了校正注意力,它是从错误预测的区域中学习的,并引导网络将注意力集中在易出错区域,从而纠正错误。通过这两个方面的关注,我们使用净化机制,对整个显著对象的不同区域施加严格的权重,并从难以区分的区域中净化结果,从而准确地预测显著对象的位置和细节。除了对这些难以区分的区域给予不同的关注外,我们还考虑了复杂区域的结构约束,提出了结构相似性损失。在实验中,所提出的方法优于19个国家的最先进的方法在六个数据集上的显着利润率超过27 FPS的单个NVIDIA 1080 Ti GPU。
Variational Autoencoders with a Structural Similarity Loss in Time of Flight MRAs
Image Sciences Institute, University Medical Center Utrecht(乌得勒支大学医学中心影像科学研究所);Department of Radiology, University Medical Center Utrecht(乌得勒支大学医学中心放射科);Department of Neurology and Neurosurgery, UMC Utrecht Brain Center, University Medical Center Utrecht(乌得勒支大学医学中心神经病学与神经外科系,乌得勒支大学医学中心脑中心)
Structural Similarity-Inspired Unfolding for Lightweight Image Super-Resolution
School of Computer Science and Technology and the Key Laboratory of Embedded System and Service Computing (Ministry of Education), Tongji University(计算机科学与技术学院和嵌入式系统与服务计算重点实验室,同济大学);Department of Computer Science, City University of Hong Kong(计算机科学系,城市大学);Pengcheng Laboratory(鹏城实验室);School of Data Science, Lingnan University(数据科学学院,岭大大学)
Modality-Agnostic Structural Image Representation Learning for Deformable Multi-Modality Medical Image Registration
DAMO Academy, Alibaba Group(阿里巴巴达摩院);Hupan Lab(湖畔实验室);College of Computer Science and Technology, Zhejiang University, China(浙江大学计算机科学与技术学院);Shengjing Hospital of China Medical University, China(中国医科大学附属盛京医院)
Color image restoration based on nonlocal saturation-value similarity
School of Mathematical Sciences, Key Laboratory of Intelligent Computing and Applications (Ministry of Education), Tongji University(数学科学学院、智能计算与应用重点实验室(教育部)、同济大学)
Investigating Fine- and Coarse-grained Structural Correspondences Between Deep Neural Networks and Human Object Image Similarity Judgments Using Unsupervised Alignment
Graduate School of Arts and Science, University of Tokyo(东京大学艺术与科学研究生院)
Macro2Micro: A Rapid and Precise Cross-modal Magnetic Resonance Imaging Synthesis using Multi-scale Structural Brain Similarity
Department of Brain and Cognitive Science, Seoul National University, Seoul, Republic of Korea(脑科学与认知科学系,首尔国立大学);Department of Applied Bioengineering, Seoul National University, Seoul, Republic of Korea(应用生物工程系,首尔国立大学);Department of Psychology, Seoul National University, Seoul, Republic of Korea(心理学系,首尔国立大学);Interdisciplinary Program in Artificial Intelligence, Seoul National University, Seoul, Republic of Korea(人工智能跨学科项目,首尔国立大学);Computational Science Initiative, Brookhaven National Laboratory, Upton, NY, USA(计算科学计划,布鲁赫斯国家实验室);School of Economics, Sogang University(经济学院,成均馆大学);Brookhaven National Laboratory(布鲁赫斯国家实验室)
A Novel Image Similarity Metric for Scene Composition Structure
School of Information Technology, Deakin University(信息技术学院,德堪大学);School of Computing, Mathematics and Engineering, Charles Sturt University(计算科学、数学与工程学院,查尔斯·斯图尔特大学)
Spatio-Temporal Structure Consistency for Semi-supervised Medical Image Classification
Shenzhen Research Institute of Big Data, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)深圳大数据研究院);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))