iSight: Towards expert-AI co-assessment for improved immunohistochemistry staining interpretation
Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania（病理与实验室医学系，佩尔曼医学院，宾夕法尼亚大学）；Department of Electrical and Systems Engineering, University of Pennsylvania（电气与系统工程系，宾夕法尼亚大学）；Department of Biomedical Data Science, Stanford University School of Medicine（生物医学数据科学系，斯坦福大学医学院）；Department of Pathology, University of Maryland School of Medicine（病理学系，马里兰大学医学院）；Department of Pathology, Stanford University School of Medicine（病理学系，斯坦福大学医学院）
Jacob S. Leiby, Jialu Yao, Pan Lu, George Hu, Anna Davidian, Shunsuke Koga, Olivia Leung, Pravin Patel, Isabella Tondi Resta, Rebecca Rojansky, Derek Sung, Eric… 展开作者
Jacob S. Leiby, Jialu Yao, Pan Lu, George Hu, Anna Davidian, Shunsuke Koga, Olivia Leung, Pravin Patel, Isabella Tondi Resta, Rebecca Rojansky, Derek Sung, Eric Yang, Paul J. Zhang, Emma Lundberg, Dokyoon Kim, Serena Yeung-Levy, James Zou, Thomas Montine, Jeffrey Nirschl, Zhi Huang
免疫组化（IHC）为组织切片中的蛋白质表达提供信息，并常用于支持病理诊断和疾病分诊。尽管用于H&E染色切片的AI模型显示出潜力，但其在IHC中的适用性受到领域特定差异的限制。在此，我们介绍了HPA10M数据集，该数据集包含来自人类蛋白图谱的10,495,672张IHC图像，并包含详尽的元数据，涵盖45种正常组织类型和20种主要癌症类型。基于HPA10M，我们训练了iSight，一种多任务学习框架，用于自动化IHC染色评估。iSight通过令牌级注意力机制结合整张滑动图像的视觉特征和组织元数据，同时预测染色强度、位置、数量、组织类型和恶性状态。在保留数据上，iSight在位置、强度和数量上的准确率分别达到85.5%、76.6%和75.7%，优于微调的基础模型（PLIP、CONCH）2.5-10.2%。此外，iSight的预测具有良好的校准，预期校准误差为0.0150-0.0408。此外，在八名病理科医生评估来自两个数据集的200张图像的用户研究中，iSight在保留的HPA数据集上优于初始病理科医生评估（位置79% vs 68%，强度70% vs 57%，数量68% vs 52%）。在保留的HPA（Cohen's κ值从0.63增加到0.70）和斯坦福TMAD数据集（从0.74增加到0.76）中，跨病理科医生的一致性也有所提高，表明专家-人工智能协同评估可以提高IHC解释。这项工作为能够提高IHC诊断准确性的AI系统奠定了基础，并突显了将iSight整合到临床工作流程中以提高IHC评估一致性和可靠性的潜力。
英文摘要：
Immunohistochemistry (IHC) provides information on protein expression in tissue sections and is commonly used to support pathology diagnosis and disease triage. While AI models for H\&E-stained slides show promise, their applicability to IHC is limited due to domain-specific variations. Here we introduce HPA10M, a dataset that contains 10,495,672 IHC images from the Human Protein Atlas with comprehensive metadata included, and encompasses 45 normal tissue types and 20 major cancer types. Based on HPA10M, we trained iSight, a multi-task learning framework for automated IHC staining assessment. iSight combines visual features from whole-slide images with tissue metadata through a token-level attention mechanism, simultaneously predicting staining intensity, location, quantity, tissue type, and malignancy status. On held-out data, iSight achieved 85.5\% accuracy for location, 76.6\% for intensity, and 75.7\% for quantity, outperforming fine-tuned foundation models (PLIP, CONCH) by 2.5--10.2\%. In addition, iSight demonstrates well-calibrated predictions with expected calibration errors of 0.0150-0.0408. Furthermore, in a user study with eight pathologists evaluating 200 images from two datasets, iSight outperformed initial pathologist assessments on the held-out HPA dataset (79\% vs 68\% for location, 70\% vs 57\% for intensity, 68\% vs 52\% for quantity). Inter-pathologist agreement also improved after AI assistance in both held-out HPA (Cohen's $κ$ increased from 0.63 to 0.70) and Stanford TMAD datasets (from 0.74 to 0.76), suggesting expert--AI co-assessment can improve IHC interpretation. This work establishes a foundation for AI systems that can improve IHC diagnostic accuracy and highlights the potential for integrating iSight into clinical workflows to enhance the consistency and reliability of IHC assessment.
Automated segmentation of rheumatoid arthritis immunohistochemistry stained synovial tissue
Queen Mary University of London（伦敦玛丽女王大学）；Centre for Translational Bioinformatics（转化生物信息学中心）；Digital Environment Research Institute（数字环境研究所）；Centre for Experimental Medicine and Rheumatology（实验医学与风湿病学中心）
Deep Learning for Generating Computational PIN-4 Immunohistochemistry Staining from Prostate Biopsy H&E Images
Biomedical Engineering University of California, Irvine Irvine, CA, USA（生物医学工程 卡罗来纳大学伊文城分校 伊文城，加州，美国）；Laboratory Medicine Biomedical Engineering Electrical Engineering（实验室医学 生物医学工程 电气工程）；Computer Science University of California, Irvine Irvine, CA, USA（计算机科学 卡罗来纳大学伊文城分校 伊文城，加州，美国）
Diagnosis of Helicobacter pylori using AutoEncoders for the Detection of Anomalous Staining Patterns in Immunohistochemistry Images
Universitat Autònoma de Barcelona（巴塞罗那自治大学）；Computer Vision Center（计算机视觉中心）；Hospital Universitari General de Catalunya（加泰罗尼亚综合大学医院）；Josep Carreras Institute（何塞普·卡雷拉斯研究所）
VIMs: Virtual Immunohistochemistry Multiplex staining via Text-to-Stain Diffusion Trained on Uniplex Stains
Kahlert School of Computing, University of Utah（犹他大学卡勒特计算学院）；Scientific Computing and Imaging Institute, University of Utah（犹他大学科学计算与成像研究所）；The Catholic University of Korea College of Medicine（韩国天主教大学医学院）；Huntsman Cancer Institute, University of Utah Health（犹他大学健康亨茨曼癌症研究所）；Department of Pathology, University of Utah（犹他大学病理学系）
Roche Innovation Center Munich（慕尼黑罗氏创新中心）；DeePathology Ltd.（DeePathology有限公司）
Histopathological evaluation of tissue samples is a key practice in patient diagnosis and drug development, especially in oncology. Historically, Hematoxylin and Eosin (H&E) has been used by pathologists as a gold standard staining. However, in many cases, various target specific stains, including immunohistochemistry (IHC), are needed in order to highlight specific structures in the tissue. As tissue is scarce and staining procedures are tedious, it would be beneficial to generate images of stained tissue virtually. Virtual staining could also generate in-silico multiplexing of different stai…
Generalizing AI-driven Assessment of Immunohistochemistry across Immunostains and Cancer Types: A Universal Immunohistochemistry Analyzer
Lunit；Samsung Medical Center（三星医学中心）；Sungkyunkwan University School of Medicine（成均馆大学医学院）；Seoul National University Bundang Hospital（首尔大学盆唐医院）；Ajou University School of Medicine（亚洲大学医学院）；Ajou University Graduate School of Medicine（亚洲大学医学研究生院）
Artificial Intelligence-Assisted Prostate Cancer Diagnosis for Reduced Use of Immunohistochemistry
Stavanger University Hospital（斯塔万格大学医院）；University of Stavanger（斯塔万格大学）；Karolinska Institutet（卡罗林斯卡学院）；Synlab；University of Bergen（卑尔根大学）；Griffith University（格里菲斯大学）；SciLifeLab
Roche Innovation Center Munich（罗氏慕尼黑创新中心）；Technische Universität München（慕尼黑工业大学）；SagivTech LTD.（SagivTech有限公司）
A key challenge in cancer immunotherapy biomarker research is quantification of pattern changes in microscopic whole slide images of tumor biopsies. Different cell types tend to migrate into various tissue compartments and form variable distribution patterns. Drug development requires correlative analysis of various biomarkers in and between the tissue compartments. To enable that, tissue slides are manually annotated by expert pathologists. Manual annotation of tissue slides is a labor intensive, tedious and error-prone task. Automation of this annotation process can improve accuracy and cons…
Westlake University（西湖大学）；Westlake Institute for Advanced Study（西湖高等研究院）；Zhejiang University（浙江大学）
Ki67 is a significant biomarker in the diagnosis and prognosis of cancer, whose index can be evaluated by quantifying its expression in Ki67 immunohistochemistry (IHC) stained images. However, quantitative analysis on multi-source Ki67 images is yet a challenging task in practice due to cross-domain distribution differences, which result from imaging variation, staining styles, and lesion types. Many recent studies have made some efforts on domain generalization (DG), whereas there are still some noteworthy limitations. Specifically in the case of Ki67 images, learning invariant representation…
CRUK Beatson Institute（CRUK比特森研究所）；Institute of Cancer Sciences（癌症科学研究所）；University of Glasgow（格拉斯哥大学）
Immunohistochemistry is a valuable diagnostic tool for cancer pathology. However, it requires specialist labs and equipment, is time-intensive, and is difficult to reproduce. Consequently, a long term aim is to provide a digital method of recreating physical immunohistochemical stains. Generative Adversarial Networks have become exceedingly advanced at mapping one image type to another and have shown promise at inferring immunostains from haematoxylin and eosin. However, they have a substantial weakness when used with pathology images as they can fabricate structures that are not present in th…
Leveraging Adversarial Learning for Pathological Fidelity in Virtual Staining
INESC TEC, Faculdade de Engenharia, Universidade do Porto, Porto, Portugal（INESC TEC，工程学院， Porto 大学，Porto，葡萄牙）；The Netherlands Cancer Institute, Amsterdam, The Netherlands（荷兰癌症研究所，阿姆斯特丹，荷兰）；IMP Diagnostics, Porto, Portugal（IMP 诊断公司，Porto，葡萄牙）
SIMPLER: H&E-Informed Representation Learning for Structured Illumination Microscopy
Kahlert School of Computing, University of Utah, Salt Lake City, UT 84112, USA Scientific Computing \& Imaging Institute, University of Utah, Salt Lake City, UT 84112, USA Instapath Inc., Houston, TX 77021 Tulane University, Department of Biomedical Engineering, New Orleans, Louisiana, United States
Building Trust in Virtual Immunohistochemistry: Automated Assessment of Image Quality
Kahlert School of Computing, University of Utah（犹他大学卡勒特计算学院）；Department of Internal Medicine, Division of Epidemiology, University of Utah（犹他大学内科学系流行病学部门）；Department of Pathology, University of Utah（犹他大学病理学系）；ARUP Laboratories（ARUP实验室）
Segmentation-before-Staining Improves Structural Fidelity in Virtual IHC-to-Multiplex IF Translation
Interdisciplinary Program in Cancer Biology, Seoul National University College of Medicine（癌症生物学跨学科项目，首尔国立大学医学院）；Interdisciplinary Program in Bioengineering, Seoul National University（生物工程跨学科项目，首尔国立大学）；Department of Biomedical Sciences, Seoul National University（生物医学科学系，首尔国立大学）；Department of Radiology, Seoul National University Hospital（放射科，首尔国立大学医院）；Department of Radiology, Seoul National University College of Medicine（放射科，首尔国立大学医学院）；Healthcare AI Research Institute, Seoul National University Hospital（医疗人工智能研究 institute，首尔国立大学医院）
Histology Virtual Staining with Mask-Guided Adversarial Transfer Learning for Tertiary Lymphoid Structure Detection
Yu-Yue Pathology Research Center（渝越病理研究中心）；Jinfeng Laboratory（金凤实验室）；Institute of Pathology and Southwest Cancer Center, Southwest Hospital, Army Medical University (Third Military Medical University)（陆军军医大学（第三军医大学）西南医院病理科及西南癌症中心）
Score-based Diffusion Model for Unpaired Virtual Histology Staining
Department of Health Technology and Informatics, Hong Kong Polytechnic University, China（香港理工大学健康科技与信息学系）；Department of Clinical Neurosciences, University of Cambridge, UK（剑桥大学临床神经科学系）；School of Science and Engineering, University of Dundee, UK（邓迪大学科学与工程学院）；Department of Applied Mathematics and Theoretical Physics, University of Cambridge, UK（剑桥大学应用数学与理论物理系）
StainNet: Scaling Self-Supervised Foundation Models on Immunohistochemistry and Special Stains for Computational Pathology
Tsinghua Shenzhen International Graduate School, Tsinghua University, China（清华大学深圳国际研究生院）；Biomedical Engineering Programme, City University of Hong Kong (Dongguan)（香港城市大学（东莞）生物医学工程项目）；Department of Pathology, Affiliated Hospital of Nantong University（南通大学附属医院病理科）；Nantong University（南通大学）
Rank-Aware Agglomeration of Foundation Models for Immunohistochemistry Image Cell Counting
School of Software Engineering, Xi’an Jiaotong University（西安交通大学软件工程学院）；School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机科学学院）；Department of Gastrointestinal Surgery and the Gastric Cancer Center, Zhongshan Hospital, Fudan University（复旦大学中山医院胃肠外科及胃癌中心）；School of Computer Science and Technology, Xi’an Jiaotong University（西安交通大学计算机科学与技术学院）；The Comprehensive Breast Care Center, The Second Affiliated Hospital of Xi'an Jiaotong University（西安交通大学第二附属医院综合乳腺护理中心）；Department of Pathology, The Second Affiliated Hospital of Xi’an Jiaotong University（西安交通大学第二附属医院病理科）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Department of General Surgery, Zhongshan Hospital (Xiamen), Fudan University（复旦大学中山医院（厦门）外科科）
While hematoxylin and eosin (H&E) is a standard staining procedure, immunohistochemistry (IHC) staining further serves as a diagnostic and prognostic method. However, acquiring special staining results requires substantial costs. Hence, we proposed a strategy for ultra-high-resolution unpaired image-to-image translation: Kernelized Instance Normalization (KIN), which preserves local information and successfully achieves seamless stain transformation with constant GPU memory usage. Given a patch, corresponding position, and a kernel, KIN computes local statistics using convolution operation. In…
The Institute of Cancer Research（癌症研究院）；University College London（伦敦大学学院）
In digital pathology, cell detection and classification are often prerequisites to quantify cell abundance and explore tissue spatial heterogeneity. However, these tasks are particularly challenging for multiplex immunohistochemistry (mIHC) images due to high levels of variability in staining, expression intensity, and inherent noise as a result of preprocessing artefacts. We proposed a deep learning method to detect and classify cells in mIHC whole-tumour slide images of breast cancer. Inspired by inception-v3, we developed Cell COunt RegularizeD Convolutional neural Network (ConCORDe-Net) wh…
Adaptive H&E-IHC information fusion staining framework based on feature extra
HKUST(GZ)（香港科技大学（广州））；Deep Interdisciplinary Intelligence Lab（深度交叉智能实验室）；Institute of Deep Perception Technology, JITRI（JITRI深度感知技术研究所）；Shandong University（山东大学）
Deep learning-enabled virtual multiplexed immunostaining of label-free tissue for vascular invasion assessment
University of California, Los Angeles（加州大学洛杉矶分校）；California NanoSystems Institute（加州纳米系统研究所）；Hadassah Hebrew University Medical Center（哈达萨希伯来大学医学中心）；University of Southern California（南加州大学）
HistDiT: A Structure-Aware Latent Conditional Diffusion Model for High-Fidelity Virtual Staining in Histopathology
Edge Hill University（埃奇希尔大学）；University of Nottingham Malaysia（诺丁汉大学马来西亚分校）；University of Southampton Malaysia（南安普顿大学马来西亚分校）；Cancer Research Malaysia（马来西亚癌症研究中心）
Towards Reliable Stain Transfer: An Iterative Data-Model Co-Optimization Framework Based on Multimodal Expert-Guided Assessment
East China Normal University（华东师范大学）；Ruijin Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属瑞金医院）；Hangzhou Hyperspectral Imaging Technology Co., Ltd.（杭州高光谱成像技术有限公司）；Fudan University Shanghai Cancer Center（复旦大学附属肿瘤医院）
LGD-Net: Latent-Guided Dual-Stream Network for HER2 Scoring with Task-Specific Domain Knowledge
School of Mathematics and Statistics, Fujian Normal University（福建师范大学数学与统计学学院）；Department of Oncology, Mengchao Hepatobiliary Hospital of Fujian Medical University（福建医科大学 Mengchao 肝胆医院肿瘤科）
From Pixels to Pathology: Restoration Diffusion for Diagnostic-Consistent Virtual IHC
organization= Institute of Pathology, Technical University of Munich, TUM School of Medicine；organization= Computer Aided Medical Procedures (CAMP), TU Munich , city= Munich , country= Germany；organization= Munich Center for Machine Learning (MCML) , city= Munich , country= Germany；organization= TUM School of Computation, Information；organization= Department of Medical Informatics, School of Medicine, Mashhad University of Medical Sciences , city= Mashhad , country= Iran；organization= Munich Data Science Institute (MDSI) , city= Munich , country= Germany
The analysis of FFPE tissue sections stained with haematoxylin and eosin (H&E) or immunohistochemistry (IHC) is an essential part of the pathologic assessment of surgically resected breast cancer specimens. IHC staining has been broadly adopted into diagnostic guidelines and routine workflows to manually assess status and scoring of several established biomarkers, including ER, PGR, HER2 and KI67. However, this is a task that can also be facilitated by computational pathology image analysis methods. The research in computational pathology has recently made numerous substantial advances, often …
CentraleSupélec, Université Paris-Saclay（巴黎萨克雷大学中央高等电力学院）；Gustave Roussy Cancer Campus（古斯塔夫·鲁西癌症校区）；Therapanacea（Therapanacea公司）
In whole slide imaging, commonly used staining techniques based on hematoxylin and eosin (H&E) and immunohistochemistry (IHC) stains accentuate different aspects of the tissue landscape. In the case of detecting metastases, IHC provides a distinct readout that is readily interpretable by pathologists. IHC, however, is a more expensive approach and not available at all medical centers. Virtually generating IHC images from H&E using deep neural networks thus becomes an attractive alternative. Deep generative models such as CycleGANs learn a semantically-consistent mapping between two image domai…
Lymphoid Infiltration Assessment of the Tumor Margins in H&E Slides
Laboratoire d’Informatique Paris Descartes (LIPADE)（巴黎笛卡尔大学计算机实验室（LIPADE））；Université Paris Cité（巴黎大学）；University of Applied Sciences of Western Switzerland (HES-SO Valais)（瑞士西部应用科学大学（瓦莱州））；Ambroise-Paré Hospital（昂布鲁瓦兹-帕雷医院）
Advancing H&E-to-IHC Stain Translation in Breast Cancer: A Multi-Magnification and Attention-Based Approach
Fudan University（复旦大学）；College of Electronic Engineering, Ocean University of China（中国海洋大学电子工程学院）；Research Center for Frontier Fundamental Studies, Zhejiang Lab（之江实验室前沿基础研究中心）；Institutes of Biomedical Sciences, Fudan University（复旦大学生物医学研究院）
Fred Hutchinson Cancer Research Center（弗雷德·哈钦森癌症研究中心）；University of Minnesota Medical School（明尼苏达大学医学院）；Cedars-Sinai Medical Center（西达赛奈医疗中心）
Recent advances in tissue microarray technology have allowed immunohistochemistry to become a powerful medium-to-high throughput analysis tool, particularly for the validation of diagnostic and prognostic biomarkers. However, as study size grows, the manual evaluation of these assays becomes a prohibitive limitation; it vastly reduces throughput and greatly increases variability and expense. We propose an algorithm - Tissue Array Co-Occurrence Matrix Analysis (TACOMA) - for quantifying cellular phenotypes based on textural regularity summarized by local inter-pixel relationships. The algorithm…
Atlas H&E-TME: Scalable AI-Based Tissue Profiling at Expert Pathologist-Level Accuracy
Aignostics, Germany（Aignostics，德国）；Institute of Pathology, Charité – Universitätsmedizin Berlin, Germany（柏林夏里特医学院病理学研究所）；Berlin Institute of Health, Charité – Universitätsmedizin Berlin, Germany（柏林夏里特医学院柏林健康研究所）；Massachusetts General Hospital, Department of Pathology, Harvard Medical School, Boston, MA, US（哈佛医学院麻省总医院病理学系）；Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, US（梅奥诊所检验医学与病理学系）；Machine Learning Group, Technische Universität Berlin, Germany（柏林工业大学机器学习组）；BIFOLD – Berlin Institute for the Foundations of Learning and Data, Germany（柏林学习与数据基础研究所）；Department of Artificial Intelligence, Korea University, Republic of Korea（高丽大学人工智能系）；Max-Planck Institute for Informatics, Germany（马克斯·普朗克信息学研究所）；German Cancer Research Center (DKFZ) & German Cancer Consortium (DKTK), Berlin & Munich Partner Sites, Germany（德国癌症研究中心及德国癌症联盟柏林和慕尼黑合作站点）；Institute of Pathology, Ludwig-Maximilians-Universität München, Germany（慕尼黑大学病理学研究所）；Bavarian Cancer Research Center (BZKF), Germany（巴伐利亚癌症研究中心）
