12 pages, 2 figures, 7 tables. This article is a pre-print and is currently under review at a conference
发表信息：
In: Lian C., Cao X., Rekik I., Xu X., Yan P. (eds) Machine Learning in Medical Imaging. MLMI 2021. Lecture Notes in Computer Science, vol 12966. Springer, Cham
Learning robust representations to discriminate cell phenotypes based on microscopy images is important for drug discovery. Drug development efforts typically analyse thousands of cell images to screen for potential treatments. Early works focus on creating hand-engineered features from these images or learn such features with deep neural networks in a fully or weakly-supervised framework. Both require prior knowledge or labelled datasets. Therefore, subsequent works propose unsupervised approaches based on generative models to learn these representations. Recently, representations learned with self-supervised contrastive loss-based methods have yielded state-of-the-art results on various imaging tasks compared to earlier unsupervised approaches. In this work, we leverage a contrastive learning framework to learn appropriate representations from single-cell fluorescent microscopy images for the task of Mechanism-of-Action classification. The proposed work is evaluated on the annotated BBBC021 dataset, and we obtain state-of-the-art results in NSC, NCSB and drop metrics for an unsupervised approach. We observe an improvement of 10% in NCSB accuracy and 11% in NSC-NSCB drop over the previously best unsupervised method. Moreover, the performance of our unsupervised approach ties with the best supervised approach. Additionally, we observe that our framework performs well even without post-processing, unlike earlier methods. With this, we conclude that one can learn robust cell representations with contrastive learning.
PhenoProfiler: Advancing Phenotypic Learning for Image-based Drug Discovery
University of Macau（澳门大学）；Beijing University of Technology（北京工业大学）；University of Florida（佛罗里达大学）；Macao Polytechnic University（澳门理工大学）；Sun Yat-sen University（中山大学）；Wake Forest School of Medicine（维克森林医学院）
Image-based brain cancer prediction models, based on radiomics, quantify the radiologic phenotype from magnetic resonance imaging (MRI). However, these features are difficult to reproduce because of variability in acquisition and preprocessing pipelines. Despite evidence of intra-tumor phenotypic heterogeneity, the spatial diversity between different slices within an MRI scan has been relatively unexplored using such methods. In this work, we propose a deep anchor attention aggregation strategy with a Vision Transformer to predict survival risk for brain cancer patients. A Deep Anchor Attentio…
Contrastive Balancing Representation Learning for Heterogeneous Dose-Response Curves Estimation
Zhejiang University（浙江大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Peking University（北京大学）；Emory University（埃默里大学）；Tsinghua University（清华大学）；Didi Chuxing（滴滴出行）
University of Pittsburgh（匹兹堡大学）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）
Recently brain networks have been widely adopted to study brain dynamics, brain development and brain diseases. Graph representation learning techniques on brain functional networks can facilitate the discovery of novel biomarkers for clinical phenotypes and neurodegenerative diseases. However, current graph learning techniques have several issues on brain network mining. Firstly, most current graph learning models are designed for unsigned graph, which hinders the analysis of many signed network data (e.g., brain functional networks). Meanwhile, the insufficiency of brain network data limits …
F Hoffmann-La Roche AG（罗氏公司）；Genentech Inc（基因泰克公司）；ETH Zurich（苏黎世联邦理工学院）；Uppsala University（乌普萨拉大学）
Accurate segmentation of retinal fluids in 3D Optical Coherence Tomography images is key for diagnosis and personalized treatment of eye diseases. While deep learning has been successful at this task, trained supervised models often fail for images that do not resemble labeled examples, e.g. for images acquired using different devices. We hereby propose a novel semi-supervised learning framework for segmentation of volumetric images from new unlabeled domains. We jointly use supervised and contrastive learning, also introducing a contrastive pairing scheme that leverages similarity between nea…
Facebook AI Research（脸书人工智能研究院）；Columbia University（哥伦比亚大学）；MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）；ITS, CUNY Graduate Center（纽约市立大学研究生中心智能交通系统实验室）
Self-supervised learning has recently begun to rival supervised learning on computer vision tasks. Many of the recent approaches have been based on contrastive instance discrimination (CID), in which the network is trained to recognize two augmented versions of the same instance (a query and positive) while discriminating against a pool of other instances (negatives). The learned representation is then used on downstream tasks such as image classification. Using methodology from MoCo v2 (Chen et al., 2020), we divided negatives by their difficulty for a given query and studied which difficulty…
This paper presents a novel positive and negative set selection strategy for contrastive learning of medical images based on labels that can be extracted from clinical data. In the medical field, there exists a variety of labels for data that serve different purposes at different stages of a diagnostic and treatment process. Clinical labels and biomarker labels are two examples. In general, clinical labels are easier to obtain in larger quantities because they are regularly collected during routine clinical care, while biomarker labels require expert analysis and interpretation to obtain. With…
Southern University of Science and Technology（南方科技大学）；The University of Hong Kong（香港大学）
Glaucoma is one of the ophthalmic diseases that may cause blindness, for which early detection and treatment are very important. Fundus images and optical coherence tomography (OCT) images are both widely-used modalities in diagnosing glaucoma. However, existing glaucoma grading approaches mainly utilize a single modality, ignoring the complementary information between fundus and OCT. In this paper, we propose an efficient multi-modality supervised contrastive learning framework, named COROLLA, for glaucoma grading. Through layer segmentation as well as thickness calculation and projection, re…
Contrastive learning is a representation learning method performed by contrasting a sample to other similar samples so that they are brought closely together, forming clusters in the feature space. The learning process is typically conducted using a two-stage training architecture, and it utilizes the contrastive loss (CL) for its feature learning. Contrastive learning has been shown to be quite successful in handling imbalanced datasets, in which some classes are overrepresented while some others are underrepresented. However, previous studies have not specifically modified CL for imbalanced …
GS-EMA: Integrating Gradient Surgery Exponential Moving Average with Boundary-Aware Contrastive Learning for Enhanced Domain Generalization in Aneurysm Segmentation
University of Leeds（利兹大学）；University of Manchester（曼彻斯特大学）
Determining causal effects of temporal multi-intervention assists decision-making. Restricted by time-varying bias, selection bias, and interactions of multiple interventions, the disentanglement and estimation of multiple treatment effects from individual temporal data is still rare. To tackle these challenges, we propose a comprehensive framework of temporal counterfactual forecasting from an individual multiple treatment perspective (TCFimt). TCFimt constructs adversarial tasks in a seq2seq framework to alleviate selection and time-varying bias and designs a contrastive learning-based block…
We present a new technique called contrastive principal component analysis (cPCA) that is designed to discover low-dimensional structure that is unique to a dataset, or enriched in one dataset relative to other data. The technique is a generalization of standard PCA, for the setting where multiple datasets are available -- e.g. a treatment and a control group, or a mixed versus a homogeneous population -- and the goal is to explore patterns that are specific to one of the datasets. We conduct a wide variety of experiments in which cPCA identifies important dataset-specific patterns that are mi…
Parotid gland tumor is a common type of head and neck tumor. Segmentation of the parotid glands and tumors by MR images is important for the treatment of parotid gland tumors. However, segmentation of the parotid glands is particularly challenging due to their variable shape and low contrast with surrounding structures. Recently deep learning has developed rapidly, which can handle complex problems. However, most of the current deep learning methods for processing medical images are still based on supervised learning. Compared with natural images, medical images are difficult to acquire and co…
University of California, Los Angeles（加州大学洛杉矶分校）；National University of Singapore（新加坡国立大学）
Brain tumor is a common and fatal form of cancer which affects both adults and children. The classification of brain tumors into different types is hence a crucial task, as it greatly influences the treatment that physicians will prescribe. In light of this, medical imaging techniques, especially those applying deep convolutional networks followed by a classification layer, have been developed to make possible computer-aided classification of brain tumor types. In this paper, we present a novel approach of directly learning deep embeddings for brain tumor types, which can be used for downstrea…
We propose WS-DINO as a novel framework to use weak label information in learning phenotypic representations from high-content fluorescent images of cells. Our model is based on a knowledge distillation approach with a vision transformer backbone (DINO), and we use this as a benchmark model for our study. Using WS-DINO, we fine-tuned with weak label information available in high-content microscopy screens (treatment and compound) and achieve state-of-the-art performance in not-same-compound mechanism of action prediction on the BBBC021 dataset (98%), and not-same-compound-and-batch performance…
ETH Zürich（苏黎世联邦理工学院）；Institute of Molecular Systems Biology（分子系统生物学研究所）
Recent advances in computer vision and robotics enabled automated large-scale biological image analysis. Various machine learning approaches have been successfully applied to phenotypic profiling. However, it remains unclear how they compare in terms of biological feature extraction. In this study, we propose a simple CNN architecture and implement 4 different representation learning approaches. We train 16 deep learning setups on the 770k cancer cell images dataset under identical conditions, using different augmenting and cropping strategies. We compare the learned representations by evaluat…
Northeastern University（东北大学）；Stevens Institute of Technology（史蒂文斯理工学院）；Cancer Hospital of China Medical University（中国医科大学附属肿瘤医院）；Chengdu University of Information Technology（成都信息工程大学）
Breast cancer is one of the most common and deadliest cancers among women. Since histopathological images contain sufficient phenotypic information, they play an indispensable role in the diagnosis and treatment of breast cancers. To improve the accuracy and objectivity of Breast Histopathological Image Analysis (BHIA), Artificial Neural Network (ANN) approaches are widely used in the segmentation and classification tasks of breast histopathological images. In this review, we present a comprehensive overview of the BHIA techniques based on ANNs. First of all, we categorize the BHIA systems int…
University of Glasgow（格拉斯哥大学）；New York University（纽约大学）
Deep learning based analysis of histopathology images shows promise in advancing the understanding of tumor progression, tumor micro-environment, and their underpinning biological processes. So far, these approaches have focused on extracting information associated with annotations. In this work, we ask how much information can be learned from the tissue architecture itself. We present an adversarial learning model to extract feature representations of cancer tissue, without the need for manual annotations. We show that these representations are able to identify a variety of morphological char…
NeuroCLIP: A Multimodal Contrastive Learning Method for rTMS-treated Methamphetamine Addiction Analysis
CenBRAIN Neurotech Center of Excellence, School of Engineering, Westlake University（西溪大学工程学院先进神经技术中心）；School of Data Science, Xiamen University Malaysia（马来西亚厦门大学数据科学学院）；Department of Neurosurgery, Second Affiliated Hospital, School of Medicine, Zhejiang University（浙江大学医学院附属第二医院神经外科）；Department of Education and Correction, Zhejiang Gongchen Compulsory Isolated Detoxification Center（浙江省公检强制隔离戒毒所教育矫正部）；Zhejiang Liangzhu Compulsory Isolated Detoxification Center（浙江省良渚强制隔离戒毒所）
MB-DSMIL-CL-PL: Scalable Weakly Supervised Ovarian Cancer Subtype Classification and Localisation Using Contrastive and Prototype Learning with Frozen Patch Features
School of Computing Sciences, University of East Anglia（东安格利亚大学计算机科学学院）；Norfolk and Norwich University Hospital（诺福克和诺里奇大学医院）；University of East Anglia（东安格利亚大学）
Enhancing drug and cell line representations via contrastive learning for improved anti-cancer drug prioritization
The Ohio State University（俄亥俄州立大学）；Biomedical Informatics Department, The Ohio State University（俄亥俄州立大学生物医学信息学系）；Computer Science and Engineering Department, The Ohio State University（俄亥俄州立大学计算机科学与工程系）；Translational Data Analytics Institute, The Ohio State University（俄亥俄州立大学转化数据分析研究所）
University of Oxford（牛津大学）；University Hospitals Coventry and Warwickshire（考文垂与沃里克郡大学医院）；University of Warwick（华威大学）；Hamad Medical Corporation（哈马德医疗公司）；International Association for Cancer Research（国际癌症研究协会）
Distant metastasis is the major cause of death in colorectal cancer (CRC). Patients at high risk of developing distant metastasis could benefit from appropriate adjuvant and follow-up treatments if stratified accurately at an early stage of the disease. Studies have increasingly recognized the role of diverse cellular components within the tumor microenvironment in the development and progression of CRC tumors. In this paper, we show that a new method of automated analysis of digitized images from colorectal cancer tissue slides can provide important estimates of distant metastasis-free surviv…
CoDCL: Counterfactual-Inspired Augmentation Contrastive Learning for Temporal Link Prediction in Social Networks
Jiangsu Key Laboratory of Net-Worked Collective Intelligence, School of Cyber Science and Engineering, Southeast University（江苏网络集体智能重点实验室，网络安全科学与工程学院，东南大学）；School of Mathematics, Southeast University（数学学院，东南大学）；Frontiers Science Center for Mobile Information Communication and Security, School of Mathematics, Southeast University（移动信息通信与安全前沿科学中心，数学学院，东南大学）；Purple Mountain Laboratories（紫金山实验室）
Integrating chemical structures as treatments improves representations of microscopy images for morphological profiling
Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）；Microsoft Research（微软研究院）；Department of Computer Science, Zhejiang University（浙江大学计算机科学系）
University of Michigan（密歇根大学）；Western Michigan University（西密歇根大学）；Brown University（布朗大学）；Invenio Imaging（Invenio影像公司）；New York University（纽约大学）
Background: Accurate diagnosis of skull base tumors is essential for providing personalized surgical treatment strategies. Intraoperative diagnosis can be challenging due to tumor diversity and lack of intraoperative pathology resources. Objective: To develop an independent and parallel intraoperative pathology workflow that can provide rapid and accurate skull base tumor diagnoses using label-free optical imaging and artificial intelligence. Method: We used a fiber laser-based, label-free, non-consumptive, high-resolution microscopy method ($<$ 60 sec per 1 $\times$ 1 mm$^\text{2}$), called s…
EVOKE: Elevating Chest X-ray Report Generation via Multi-View Contrastive Learning and Patient-Specific Knowledge
Xidian University（西安电子科技大学）；Shanghai Jiao Tong University School of Medicine（上海交通大学医学院）；Ruijin Hospital（瑞金医院）；Shanghai Institute of Traumatology and Orthopedics（上海市伤骨科研究所）；Harbin Medical University（哈尔滨医科大学）；The First Affiliated Hospital of Harbin Medical University（哈尔滨医科大学附属第一医院）；Brown University（布朗大学）
Joint Masked Reconstruction and Contrastive Learning for Mining Interactions Between Proteins
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）；Institute of Artificial Intelligence, Huazhong University of Science and Technology（华中科技大学人工智能研究院）；Hubei Key Laboratory of Brain-Inspired Intelligent Systems, Huazhong University of Science and Technology（华中科技大学湖北省类脑智能系统重点实验室）；Key Laboratory of Image Processing and Intelligent Control (Huazhong University of Science and Technology), Ministry of Education（华中科技大学图像信息处理与智能控制教育部重点实验室）
Autism Spectrum Disorder Classification with Interpretability in Children based on Structural MRI Features Extracted using Contrastive Variational Autoencoder
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Shenzhen Key Laboratory of Intelligent Bioinformatics（深圳市智能生物信息学重点实验室）；Center for High Performance Computing, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院高性能计算中心）
Cross-Modal Contrastive Learning from Histopathology and CT for Automated Renal Cell Carcinoma Grading
Dartmouth College（达特茅斯学院）；Geisel School of Medicine at Dartmouth（达特茅斯盖泽尔医学院）；Massachusetts General Hospital（麻省总医院）；Dartmouth-Hitchcock Medical Center（达特茅斯-希契科克医疗中心）；Memorial Sloan Kettering Cancer Center（纪念斯隆-凯特琳癌症中心）
