Nuclei classification is a critical step in computer-aided diagnosis with histopathology images. In the past, various methods have employed graph neural networks (GNN) to analyze cell graphs that model inter-cell relationships by considering nuclei as vertices. However, they are limited by the GNN mechanism that only passes messages among local nodes via fixed edges. To address the issue, we develop a cell graph transformer (CGT) that treats nodes and edges as input tokens to enable learnable adjacency and information exchange among all nodes. Nevertheless, training the transformer with a cell graph presents another challenge. Poorly initialized features can lead to noisy self-attention scores and inferior convergence, particularly when processing the cell graphs with numerous connections. Thus, we further propose a novel topology-aware pretraining method that leverages a graph convolutional network (GCN) to learn a feature extractor. The pre-trained features may suppress unreasonable correlations and hence ease the finetuning of CGT. Experimental results suggest that the proposed cell graph transformer with topology-aware pretraining significantly improves the nuclei classification results, and achieves the state-of-the-art performance. Code and models are available at https://github.com/lhaof/CGT
Accurate Tracking of Arabidopsis Root Cortex Cell Nuclei in 3D Time-Lapse Microscopy Images Based on Genetic Algorithm
Graduate School of Science and Technology, Nara Institute of Science and Technology(奈良科学技术大学研究生院);Bioresource Engineering Laboratory, Ishikawa Prefectural University(石川县立大学生物资源工程实验室);Department of Engineering Informatics, Osaka Electro-Communication University(大阪电讯大学工程信息学系);Exploratory Research Center on Life and Living Systems, National Institutes of Natural Sciences(国家自然科学研究院生命与生活系统探索研究中心);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院)
A Multi-Branch Hybrid Transformer Networkfor Corneal Endothelial Cell Segmentation
Southern University of Science and Technology(南方科技大学);University of Technology Sydney(悉尼科技大学);Tomey Corporation(Tomey公司);Inception Institute of Artificial Intelligence(起源人工智能研究院);Baidu(百度);Chinese Academy of Sciences(中国科学院)
Max Planck Institute of Molecular Cell Biology and Genetics(马克斯·普朗克分子细胞生物学与遗传学研究所);Center for Systems Biology Dresden(德累斯顿系统生物学中心);Technical University Dresden(德累斯顿工业大学)
Tissue Aware Nuclei Detection and Classification Model for Histopathology Images
TIA Centre, Department of Computer Science, University of Warwick, UK(沃里克大学计算机科学系TIA中心,英国);GSK, Artificial Intelligence and Machine Learning, London, UK(葛兰素史克人工智能与机器学习,英国)
CellViT: Vision Transformers for Precise Cell Segmentation and Classification
Institute for AI in Medicine (IKIM), University Hospital Essen (AöR)(埃森大学医院(AöR)医学人工智能研究所(IKIM));Cancer Research Center Cologne Essen (CCCE), West German Cancer Center Essen, University Hospital Essen (AöR)(埃森大学医院(AöR)西德埃森癌症中心科隆埃森癌症研究中心(CCCE));Clinic for Nuclear Medicine, University Hospital Essen (AöR)(埃森大学医院(AöR)核医学科);Institute of Pathology, University Hospital Essen (AöR)(埃森大学医院(AöR)病理研究所);Institute of Interventional and Diagnostic Radiology and Neuroradiology, University Hospital Essen (AöR)(埃森大学医院(AöR)介入与诊断放射学及神经放射学研究所);Department of Dermatology, University Hospital Essen (AöR)(埃森大学医院(AöR)皮肤科);German Cancer Consortium (DKTK, Partner site Essen)(德国癌症联盟(DKTK,埃森合作站点))
From Histopathology Images to Cell Clouds: Learning Slide Representations with Hierarchical Cell Transformer
University of Science and Technology Beijing(北京科技大学);DAMO Academy, Alibaba Group(阿里巴巴达摩院);Zhejiang University(浙江大学);Fudan University(复旦大学);Changhai Hospital(长海医院);Peking University(北京大学)
Segmentation-free integration of nuclei morphology and spatial transcriptomics for retinal images
Dept. of Information Technology(信息技术系);Dept. of Molecular, Cell and Developmental Biology(分子、细胞与发育生物学系);University of California, Los Angeles(加利福尼亚大学洛杉矶分校)
Institute of Imaging and Computer Vision, RWTH Aachen University(亚琛工业大学成像与计算机视觉研究所);Heinrich Heine University Düsseldorf(杜塞尔多夫海因里希·海涅大学);Institute of Medical Sciences, Kermanshah University(克尔曼沙阿大学医学研究所)
Automated Histopathologic Assessment of Hirschsprung Disease Using a Multi-Stage Vision Transformer Framework
Department of Systems and Computer Engineering, Carleton University(系统与计算机工程系,卡尔顿大学);Department of Clinical Science and Translational Medicine, University of Ottawa(临床科学与转化医学系,渥太华大学);School of Epidemiology and Public Health, University of Ottawa(流行病学与公共卫生学院,渥太华大学);Department of Methodological and Implementation Research, Ottawa Hospital Research Institute(方法学与实施研究系,渥太华医院研究所);Children’s Hospital of Eastern Ontario (CHEO)(东部 Ontario儿童医院(CHEO))
Prompt-based Grouping Transformer for Nucleus Detection and Classification
Sun Yat-sen University(中山大学);Shenzhen Research Institute of Big Data(深圳大数据研究院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Guangdong University of Technology(广东工业大学)
Trackastra: Transformer-based cell tracking for live-cell microscopy
Institute of Bioengineering, School of Life Sciences, EPFL(EPFL生命科学学院生物工程研究所);Center for Scalable Data Analytics and AI (ScaDS.AI)(可扩展数据分析与人工智能中心(ScaDS.AI));Faculty of Computer Science, TU Dresden(德累斯顿工业大学计算机科学学院)
Combining Microscopy Data and Metadata for Reconstruction of Cellular Traction Forces Using a Hybrid Vision Transformer-U-Net
Fakultät Physik, Technische Universität Dortmund(物理系,德意志理工大学多特蒙德分校);Faculty of Physics and Center for NanoScience, Ludwig-Maximilians-Universität München(物理系和纳米科学中心,路德维希-马克西米利安大学慕尼黑分校);Department of Veterinary Sciences, Ludwig-Maximilians-Universität München(兽医学系,路德维希-马克西米利安大学慕尼黑分校)
Context-aware Skin Cancer Epithelial Cell Classification with Scalable Graph Transformers
Faculty of Mathematics and Natural Sciences, University of Cologne(数学与自然科学学院,科隆大学);Excellence Cluster on Cellular Stress Responses in Aging-Associated Diseases (CECAD), University of Cologne(细胞应激反应与衰老相关疾病卓越集群(CECAD),科隆大学)
EmbryosFormer: Deformable Transformer and Collaborative Encoding-Decoding for Embryos Stage Development Classification
FPT Software AI Center(FPT软件人工智能中心);John von Neumann Institute, Vietnam National University(越南国家大学冯·诺依曼研究院);My Duc Phu Nhuan Hospital(美德富润医院);Vinmec Central Park International Hospital(Vinmec中央公园国际医院);Arkansas Economic Development Commission(阿肯色州经济发展委员会);University of Science, VNU-HCM(越南国家大学胡志明市分校科学大学);University of Arkansas(阿肯色大学);My Duc Hospital(美德医院)
Star-convex Polyhedra for 3D Object Detection and Segmentation in Microscopy
Institute of Bioengineering, École Polytechnique Fédérale de Lausanne (EPFL)(洛桑联邦理工学院生物工程研究所);Max Planck Institute of Molecular Cell Biology and Genetics (MPI-CBG)(马克斯·普朗克分子细胞生物学与遗传学研究所);Center for Systems Biology Dresden (CSBD)(德累斯顿系统生物学中心);Institut de Génomique Fonctionnelle de Lyon (IGFL), École Normale Supérieure de Lyon(里昂高等师范学院里昂功能基因组学研究所);Centre National de la Recherche Scientifique (CNRS)(法国国家科学研究中心)
Weak-to-Strong Generalization Enables Fully Automated De Novo Training of Multi-head Mask-RCNN Model for Segmenting Densely Overlapping Cell Nuclei in Multiplex Whole-slice Brain Images
Department of Electrical and Computer Engineering, University of Houston(电子与计算机工程系,休斯顿大学);National Institute of Neurological Disorders and Stroke(国家神经疾病与中风研究所);Department of Neurobiology and Anatomy, UT McGovern Medical School(神经生物学与解剖学系,UT麦戈文医学学校);Canva Pty Ltd.
3D Segmentation of Neuronal Nuclei and Cell-Type Identification using Multi-channel Information
Universidad Politécnica de Madrid(马德里理工大学);Center for Computational Simulation(计算模拟中心);Instituto Cajal (CSIC)(卡哈尔研究所(西班牙国家研究委员会));Laboratorio Cajal de Circuitos Corticales (CTB)(卡哈尔皮层回路实验室(CTB))
Is the aspect ratio of cells important in deep learning? A robust comparison of deep learning methods for multi-scale cytopathology cell image classification: from convolutional neural networks to visual transformers
Northeastern University(东北大学);Chengdu University of Information Technology(成都信息工程大学);Shengjing Hospital, China Medical University(中国医科大学附属盛京医院);Suzhou Ruiguan Technology Company Ltd.(苏州瑞观科技有限公司)
宫颈癌是一种常见的女性致命癌症。细胞病理学图像通常用于筛查这种癌症。鉴于人工筛查过程中可能会出现许多错误,开发了一种基于深度学习的计算机辅助诊断系统。深度学习方法要求输入图像的维度是固定的,但临床医学图像的维度是不一致的。图像的宽高比受到影响,而直接将其隐藏。在临床上,细胞病理学图像中细胞的长宽比为医生诊断癌症提供了重要信息。因此,很难直接调整大小。然而,许多现有的研究直接调整图像的大小,并获得了高度鲁棒的分类结果。为了确定一个合理的解释,我们进行了一系列的比较实验。首先,SIPaKMeD数据集的原始数据进行预处理,以获得标准和缩放的数据集。然后,将数据集的大小调整为224 x 224像素。最后,使用22个深度学习模型对标准和缩放数据集进行分类。研究结果表明,深度学习模型对宫颈细胞病理学图像中细胞纵横比的变化具有鲁棒性。这一结论也通过Herlev数据集得到了验证。
A Survey on Cell Nuclei Instance Segmentation and Classification: Leveraging Context and Attention
INESC TEC - Institute for Systems and Computer Engineering, Technology and Science(INESC TEC系统与计算机工程、技术与科学研究所);University of Porto - Faculty of Engineering(波尔图大学工程学院);IMP Diagnostics(IMP诊断公司);Portuguese Oncology Institute of Porto (IPO Porto)(波尔图葡萄牙肿瘤研究所);University of Porto (ICBAS-UP)(波尔图大学(ICBAS-UP));FCTUC - Faculty of Science and Technology, University of Coimbra(科英布拉大学科学与技术学院(FCTUC))
Learning Nuclei Representations with Masked Image Modelling
University of Cologne(科隆大学);University Hospital Cologne(科隆大学医院);Institute for Biomedical Informatics(生物医学信息学研究所);Center for Molecular Medicine Cologne (CMMC)(科隆分子医学中心(CMMC));Institute of Pathology(病理学研究所);Cologne Excellence Cluster on Cellular Stress Responses in Aging-Associated Diseases (CECAD)(科隆衰老相关疾病细胞应激反应卓越集群(CECAD))
NucMM Dataset: 3D Neuronal Nuclei Instance Segmentation at Sub-Cubic Millimeter Scale
Harvard University(哈佛大学);NIT Karnataka(卡纳塔克国家技术学院);Technical University of Munich(慕尼黑工业大学);Donostia International Physics Center (DIPC)(多诺斯蒂亚国际物理中心);University of the Basque Country (UPV/EHU)(巴斯克大学);Ikerbasque, Basque Foundation for Science(巴斯克科学基金会)
CircleFormer: Circular Nuclei Detection in Whole Slide Images with Circle Queries and Attention
Institute of Intelligent Machines, HFIPS, Chinese Academy of Sciences(中国科学院合肥物质科学研究院智能机械研究所);Institutes of Physical Science and Information Technology, Anhui University(安徽大学物质科学与信息技术研究院);School of Computer and Artificial Intelligence, Zhengzhou University(郑州大学计算机与人工智能学院)