With the limited availability of labeled data with various atmospheric conditions in remote sensing images, it seems useful to work with self-supervised algorithms. Few pretext-based algorithms, including from rotation, spatial context and jigsaw puzzles are not appropriate for satellite images. Often, satellite images have a higher temporal frequency. So, the temporal dimension of remote sensing data provides natural augmentation without requiring us to create artificial augmentation of images. Here, we propose S3-TSS, a novel method of self-supervised learning technique that leverages natural augmentation occurring in temporal dimension. We compare our results with current state-of-the-art methods and also perform various experiments. We observed that our method was able to perform better than baseline SeCo in four downstream datasets. Code for our work can be found here: https://github.com/hewanshrestha/Why-Self-Supervision-in-Time
Morph-SSL: Self-Supervision with Longitudinal Morphing to Predict AMD Progression from OCT
Medical University of Vienna（维也纳医科大学）；Imperial College London（伦敦帝国学院）；Technical University Munich（慕尼黑工业大学）；Moorfields Eye Hospital NHS Foundation Trust（摩尔菲尔兹眼科医院NHS基金会信托）；University of Basel（巴塞尔大学）；University of Southampton（南安普顿大学）
Self-Supervised Representation Learning for Geospatial Objects: A Survey
College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）；Lund University（隆德大学）；School of Computer Science, University of Auckland（奥克兰大学计算机科学学院）
Understanding Self-supervised Learning with Dual Deep Networks
Facebook AI Research（脸书人工智能研究院）；Stanford University（斯坦福大学）
我们提出了一个新的理论框架来理解对比自监督学习（SSL）方法，该方法采用双对深度ReLU网络（例如，Simplified）。首先，我们证明，在每个SGD更新Simplified与各种损失函数，包括简单的对比损失，软三重损失和InfoNCE损失，在每一层的权重更新的\emph{covariance operator}，专门放大初始随机选择性，不同的数据样本，但生存的平均值超过数据增强。为了进一步研究协方差算子扮演的角色以及在这样的过程中学习到的特征，我们通过\emph{hierarchical latent tree model}（HLTM）对数据生成和增强过程进行建模，并证明深度ReLU网络的隐藏神经元可以学习HLTM中的潜变量，尽管网络从这些未观察到的潜变量中接收\emph{no direct supervision}。这导致通过对比SSL放大最初的随机选择性来证明分层特征的出现。大量的数值研究证明了我们的理论发现。代码发布在https://github.com/facebookresearch/luckmatters/tree/master/ssl。
A Survey of the Self Supervised Learning Mechanisms for Vision Transformers
Pattern Recognition Lab, DCIS, PIEAS（模式识别实验室，DCIS，PIEAS）；PIEAS Artificial Intelligence Center (PAIC)（PIEAS人工智能中心）；Center of Secure Cyber-Physical Security Systems, Khalifa University（安全网络物理安全系统中心，卡比拉大学）；IBM Research（IBM研究院）；Department of Computer Science（计算机科学系）；Air University（空军大学）；Department of Computer Science and Engineering（计算机科学与工程系）；Kyung Hee University (Global Campus)（庆熙大学（全球校区））；Department of Electrical Engineering and Automation（电气工程与自动化系）；Aalto University（艾尔沃斯大学）；Faculty of Engineering and Green Technology（工程与绿色技术学院）；Universiti Tunku Abdul Rahman（陶布鲁克大学）；Computer Vision Department（计算机视觉部门）；Mohamed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Foundation for Advancement of Science and Technology (FAST)（科学与技术进步基金会）
Self-supervised Learning of Echocardiographic Video Representations via Online Cluster Distillation
Department of Engineering Science, University of Oxford（工程科学系，牛津大学）；Nuffield Department of Women’s and Reproductive Health, University of Oxford（妇女与生殖健康系，牛津大学）；Fundamental AI Lab, University of Technology Nuremberg（基础人工智能实验室，纽伦堡技术大学）；University of Amsterdam（阿姆斯特丹大学）
Modulation Consistency-based Contrastive Learning for Self-Supervised Automatic Modulation Classification
School of Artificial Intelligence, Xidian University（西安电子科技大学人工智能学院）；Unmanned System Research Institute, Northwestern Polytechnical University（西北工业大学无人系统研究院）；School of Electronics And Information, Northwestern Polytechnical University（西北工业大学电子与信息学院）；Department of Electrical and Computer Engineering, Aristotle University of Thessaloniki（希腊亚里士多德大学电气与计算机工程系）
DA-SSL: self-supervised domain adaptor to leverage foundational models in turbt histopathology slides
National Cancer Institute, National Institutes of Health（美国国立卫生研究院国家癌症研究所）；Fred Hutchinson Cancer Center（弗雷德·哈钦森癌症中心）；University of California, San Francisco（加州大学旧金山分校）；Yale University（耶鲁大学）；University of British Columbia（不列颠哥伦比亚大学）；University of Washington（华盛顿大学）
