Spatial arrangement of cells of various types, such as tumor infiltrating lymphocytes and the advancing edge of a tumor, are important features for detecting and characterizing cancers. However, convolutional neural networks (CNNs) do not explicitly extract intricate features of the spatial arrangements of the cells from histopathology images. In this work, we propose to classify cancers using graph convolutional networks (GCNs) by modeling a tissue section as a multi-attributed spatial graph of its constituent cells. Cells are detected using their nuclei in H&E stained tissue image, and each cell's appearance is captured as a multi-attributed high-dimensional vertex feature. The spatial relations between neighboring cells are captured as edge features based on their distances in a graph. We demonstrate the utility of this approach by obtaining classification accuracy that is competitive with CNNs, specifically, Inception-v3, on two tasks-cancerous versus non-cancerous and in situ versus invasive-on the BACH breast cancer dataset.
University of Waterloo（滑铁卢大学）；Vector Institute（向量研究所）
Representation learning for Whole Slide Images (WSIs) is pivotal in developing image-based systems to achieve higher precision in diagnostic pathology. We propose a two-stage framework for WSI representation learning. We sample relevant patches using a color-based method and use graph neural networks to learn relations among sampled patches to aggregate the image information into a single vector representation. We introduce attention via graph pooling to automatically infer patches with higher relevance. We demonstrate the performance of our approach for discriminating two sub-types of lung ca…
GRAPHITE: Graph-Based Interpretable Tissue Examination for Enhanced Explainability in Breast Cancer Histopathology
School of Computer Science and Engineering（计算机科学与工程学院）；University of New South Wales（新南威尔士大学）；St. George and Sutherland Clinical School（圣乔治与萨瑟兰临床学院）；Department of Anatomical Pathology（解剖病理学部）；NSW Health Pathology（新南威尔士州卫生病理学）；Cancer Care Centre（癌症护理中心）
University of Michigan（密歇根大学）；Imbio LLC（Imbio有限责任公司）
Early detection of lung cancer is critical for improvement of patient survival. To address the clinical need for efficacious treatments, genetically engineered mouse models (GEMM) have become integral in identifying and evaluating the molecular underpinnings of this complex disease that may be exploited as therapeutic targets. Assessment of GEMM tumor burden on histopathological sections performed by manual inspection is both time consuming and prone to subjective bias. Therefore, an interplay of needs and challenges exists for computer-aided diagnostic tools, for accurate and efficient analys…
Feature-interactive Siamese graph encoder-based image analysis to predict STAS from histopathology images in lung cancer
Hunan University（湖南大学）；The Second Xiangya Hospital, Central South University（中南大学湘雅二医院）；Central South University（中南大学）；Hunan Clinical Medical Research Center for Cancer Pathogenic Genes Testing and Diagnosis（湖南省肿瘤致病基因检测与诊断临床医学研究中心）；Shanghai Cancer Institute（上海市肿瘤研究所）；Renji Hospital, School of Medicine, Shanghai Jiaotong University（上海交通大学医学院附属仁济医院）；Shanghai Jiaotong University（上海交通大学）
University of Connecticut（康涅狄格大学）；University of Connecticut Health Center（康涅狄格大学健康中心）
Prostate cancer is the most common cancer in men worldwide and the second leading cause of cancer death in the United States. One of the prognostic features in prostate cancer is the Gleason grading of histopathology images. The Gleason grade is assigned based on tumor architecture on Hematoxylin and Eosin (H&E) stained whole slide images (WSI) by the pathologists. This process is time-consuming and has known interobserver variability. In the past few years, deep learning algorithms have been used to analyze histopathology images, delivering promising results for grading prostate cancer. Howev…
Modern histopathological image analysis relies on the segmentation of cell structures to derive quantitative metrics required in biomedical research and clinical diagnostics. State-of-the-art deep learning approaches predominantly apply convolutional layers in segmentation and are typically highly customized for a specific experimental configuration; often unable to generalize to unknown data. As the model capacity of classical convolutional layers is limited by a finite set of learned kernels, our approach uses a graph representation of the image and focuses on the node transitions in multipl…
IBM Research（IBM研究院）；EPFL（洛桑联邦理工学院）；ETH（苏黎世联邦理工学院）
Advances in entity-graph based analysis of histopathology images have brought in a new paradigm to describe tissue composition, and learn the tissue structure-to-function relationship. Entity-graphs offer flexible and scalable representations to characterize tissue organization, while allowing the incorporation of prior pathological knowledge to further support model interpretability and explainability. However, entity-graph analysis requires prerequisites for image-to-graph translation and knowledge of state-of-the-art machine learning algorithms applied to graph-structured data, which can po…
ProBAG: Prototype-Guided Boundary-Aware Graph Diffusion for Weakly Supervised Histopathology Segmentation
AI VIETNAM Lab（AI越南实验室）；Washington University School of Medicine（华盛顿大学医学院）；Jeonbuk National University（全北国立大学）；Perelman School of Medicine, University of Pennsylvania（宾夕法尼亚大学佩雷尔曼医学院）
Shenzhen Research Institute of Big Data（深圳大数据研究院）；The Chinese University of Hong Kong at Shenzhen（香港中文大学（深圳））；Pazhou Lab（琶洲实验室）；School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Artificial Intelligence Thrust, The Hong Kong University of Science and Technology at Guangzhou（香港科技大学广州人工智能部）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Guangdong Provincial Key Laboratory of Big Data Computing（广东省大数据计算重点实验室）；Guangdong Provincial Key Laboratory of Colorectal and Pelvic Floor Diseases（广东省结直肠和盆底疾病重点实验室）
Nuclei classification provides valuable information for histopathology image analysis. However, the large variations in the appearance of different nuclei types cause difficulties in identifying nuclei. Most neural network based methods are affected by the local receptive field of convolutions, and pay less attention to the spatial distribution of nuclei or the irregular contour shape of a nucleus. In this paper, we first propose a novel polygon-structure feature learning mechanism that transforms a nucleus contour into a sequence of points sampled in order, and employ a recurrent neural netwo…
Slide-based Graph Collaborative Training for Histopathology Whole Slide Image Analysis
University of Science and Technology of China（中国科学技术大学）；the First Affiliated Hospital of USTC（中国科学技术大学第一附属医院）；Beijing Advanced Innovation Center on Biomedical Engineering（北京生物医学工程高精尖创新中心）
Multi-Resolution Histopathology Patch Graphs for Ovarian Cancer Subtyping
Centre for Computational Imaging and Simulation Technologies in Biomedicine (CISTIB)（生物医学计算成像与模拟技术中心）；School of Computing, University of Leeds（利兹大学计算机学院）；Leeds Institute of Medical Research at St James’s, School of Medicine, University of Leeds（利兹大学医学院圣詹姆斯利兹医学研究所）；Leeds Cancer Centre, St James’s University Hospital（圣詹姆斯大学医院利兹癌症中心）
CellPilot: A unified approach to automatic and interactive segmentation in histopathology
School of Computation and Information Technology, Technical University of Munich（慕尼黑工业大学计算与信息技术学院）；Helmholtz Zentrum München - German Research Center for Environmental Health（亥姆霍兹慕尼黑中心——德国环境健康研究中心）；School of Biomedical Engineering and Imaging Sciences, King’s College London（伦敦国王学院生物医学工程与影像科学学院）
Leveraging Medical Foundation Model Features in Graph Neural Network-Based Retrieval of Breast Histopathology Images
Artificial Intelligence Department, Faculty of Computer Engineering, University of Isfahan, Isfahan, Iran；Department of Computer Engineering Techniques, Mazaya University College, Nasiriyah, Iraq；Department of Medicine, Danube Private University, Krems an der Donau, Austria；Austrian Center for Medical Innovation；Research Center for Medical Image Analysis；Artificial Intelligence, Department of Medicine, Danube Private University, Krems an der Donau, Austria
SIGMMA: Hierarchical Graph-Based Multi-Scale Multi-modal Contrastive Alignment of Histopathology Image and Spatial Transcriptome
Wellcome Sanger Institute（沃森桑格研究所）；Cambridge Centre for AI in Medicine（剑桥人工智能医学中心）；Institute of AI for Health（人工智能与健康研究所）；Cambridge Stem Cell Institute（剑桥干细胞研究所）
Digital Environment Research Institute（数字环境研究所）；Queen Mary University of London（伦敦玛丽女王大学）；William Harvey Research Institute（威廉·哈维研究所）；The Hong Kong Polytechnic University（香港理工大学）
Semantic-Aware Subgraph State Space Model for WSI Classification in Histopathology
National College for Excellent Engineers（卓越工程师学院）；Beihang University（北京航空航天大学）；School of Biological Science and Medical Engineering（生物与医学工程学院）；College of Engineering（工学院）；Peking University（北京大学）
Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University（北京航空航天大学生物医学工程北京高级创新中心）；School of Software, Hefei University of Technology（合肥工业大学软件学院）；Tianjin Institute of Industrial Biotechnology, Chinese Academy of Sciences（中国科学院天津工业生物技术研究所）
Content-based histopathological image retrieval (CBHIR) has become popular in recent years in the domain of histopathological image analysis. CBHIR systems provide auxiliary diagnosis information for pathologists by searching for and returning regions that are contently similar to the region of interest (ROI) from a pre-established database. While, it is challenging and yet significant in clinical applications to retrieve diagnostically relevant regions from a database that consists of histopathological whole slide images (WSIs) for a query ROI. In this paper, we propose a novel framework for …
GAFR-Net: A Graph Attention and Fuzzy-Rule Network for Interpretable Breast Cancer Image Classification
1 Department of IT Engineering, Mokwon University, Daejeon 35349, South Korea；2 School of Digital Arts, Jiangxi Arts \& Ceramics Technology Institute, Jingdezhen 333001, China
Dealing with the application of grading colorectal cancer images, this work proposes a 3 step pipeline for prediction of cancer levels from a histopathology image. The overall model performs better compared to other state of the art methods on the colorectal cancer grading data set and shows excellent performance for the extended colorectal cancer grading set. The performance improvements can be attributed to two main factors: The feature selection and graph augmentation method described here are spatially aware, but overall pixel position independent. Further, the graph size in terms of nodes…
In pathology, tissue samples are assessed using multiple staining techniques to enhance contrast in unique histologic features. In this paper, we introduce a multimodal CNN-GNN based graph fusion approach that leverages complementary information from multiple non-registered histopathology images to predict pathologic scores. We demonstrate this approach in nonalcoholic steatohepatitis (NASH) by predicting CRN fibrosis stage and NAFLD Activity Score (NAS). Primary assessment of NASH typically requires liver biopsy evaluation on two histological stains: Trichrome (TC) and hematoxylin and eosin (…
Image-based characterization and disease understanding involve integrative analysis of morphological, spatial, and topological information across biological scales. The development of graph convolutional networks (GCNs) has created the opportunity to address this information complexity via graph-driven architectures, since GCNs can perform feature aggregation, interaction, and reasoning with remarkable flexibility and efficiency. These GCNs capabilities have spawned a new wave of research in medical imaging analysis with the overarching goal of improving quantitative disease understanding, mon…
Indian Institute of Technology, Bombay（印度理工学院孟买分校）
With the increase in the use of deep learning for computer-aided diagnosis in medical images, the criticism of the black-box nature of the deep learning models is also on the rise. The medical community needs interpretable models for both due diligence and advancing the understanding of disease and treatment mechanisms. In histology, in particular, while there is rich detail available at the cellular level and that of spatial relationships between cells, it is difficult to modify convolutional neural networks to point out the relevant visual features. We adopt an approach to model histology ti…
HIEGNet: A Heterogenous Graph Neural Network Including the Immune Environment in Glomeruli Classification
LIX, École Polytechnique, IP Paris, France（巴黎理工学院LIX研究所）；ICube, Université de Strasbourg, France（斯特拉斯堡大学ICube研究所）；Institute of Pathology, Hannover Medical School, Germany（汉诺威医学院病理研究所）；Fachhochschule Wedel, Germany（韦德尔应用科学大学）
A Graph-Based Framework for Interpretable Whole Slide Image Analysis
School of Computation, Information and Technology, Technical University of Munich（慕尼黑技术大学计算、信息与技术学院）；Department of Computing, Imperial College London（伦敦帝国学院计算机系）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Munich Data Science Institute, Technical University of Munich（慕尼黑数据科学研究所）；Institute of Pathology, TUM School of Medicine and Health, Technical University of Munich（慕尼黑技术大学医学与健康学院病理学研究所）；Weill Cornell Medicine, Cornell University（韦尔医学院，康奈尔大学）
PND-Net: Plant Nutrition Deficiency and Disease Classification using Graph Convolutional Network
BITS Pilani（毕拉尼理工学院）；Jadavpur University（jadavpur大学）；University of Hradec Kralove（赫拉德茨-克拉洛韦大学）；Skoda Auto University（斯柯达汽车大学）；Universiti Teknologi Malaysia（马来西亚理工大学）
Northeastern University（东北大学）；China Medical University（中国医科大学）；University of Luebeck（吕贝克大学）
The gold standard for gastric cancer detection is gastric histopathological image analysis, but there are certain drawbacks in the existing histopathological detection and diagnosis. In this paper, based on the study of computer aided diagnosis system, graph based features are applied to gastric cancer histopathology microscopic image analysis, and a classifier is used to classify gastric cancer cells from benign cells. Firstly, image segmentation is performed, and after finding the region, cell nuclei are extracted using the k-means method, the minimum spanning tree (MST) is drawn, and graph …
