SegTME-UNI2: 一种基于基础模型的可泛化多类细胞分割框架及LLM驱动的组织微环境表征在组织病理学中的应用
SegTME-UNI2: A Foundation Model-Based Framework for Generalisable Multiclass Cell Segmentation and LLM-Driven Tumour Microenvironment Characterisation in Histopathology
AI总结：
提出SegTME-UNI2框架，结合UNI2-H病理基础模型与双头UperNet解码器实现六类语义分割和核实例分割，通过三阶段伪标签课程学习解决标注不足问题，并利用LLM生成临床可解释的TME报告。
摘要：
从常规H&E染色组织学图像中表征肿瘤微环境（TME）需要同时进行细胞分割、特征提取和可解释的临床报告。我们提出了SEGTME-UNI2，一个统一框架来满足这些需求。其核心是UNI2-UPERHOVER，一个双头分割模型，将UNI2-H病理基础模型（ViT-Giant，在来自100K张切片的>100M张图块上预训练）与两个并行的UperNet解码器配对：一个用于六类语义分割，另一个用于水平-垂直梯度回归，从而实现基于分水岭的核实例分离。为了解决大型真实世界数据集中缺乏像素级标注的问题，UNI2-UPERHOVER经历了一个三阶段渐进式伪标签课程。每个阶段训练一个全新模型（无权重迁移），完全通过提高伪标签质量来驱动改进：阶段1：使用人工标注的PanNuke（7,901张图像，189,744个细胞核，0.25 um/像素）。阶段2：使用阶段1模型在271,711个TCGA-UT尺度0图块（0.5 um/像素）上生成的熵过滤伪标签。阶段3：使用阶段2模型在所有1,608,060个TCGA-UT图块（覆盖六个分辨率尺度，0.5-1.0 um/像素）上生成的伪标签。分割输出输入到一个结构化的TME特征提取流水线，计算每个图块的20多个组成、形态、空间熵和细胞间距离指标。这些指标编码为JSON，并传递给微调的NVIDIA BioNeMo GPT模型，以生成临床可解释的TME叙述。在保留的PanNuke和TCGA-UT分区上的初步验证证明了框架的可行性和内部一致性。公开释放了伪标注的TCGA-UT数据集和UNI2-UPERHOVER检查点，以支持大规模TME分析和空间生物学研究。
英文摘要：
Characterising the TME from routine H&E-stained histology images requires simultaneous cell segmentation, biological feature extraction, and interpretable clinical reporting. We present SegTME-UNI2, a unified framework addressing all three requirements end-to-end: a segmentation backbone that converts raw H\&E patches into per-nucleus class labels, a structured feature-extraction pipeline that turns those labels into quantitative TME descriptors, and a language-model narrative generator that turns those descriptors into clinician-readable text. At its core is UNI2-UperHoVer, a dual-head multiscale segmentation model that pairs UNI2 with two parallel UperNet decoders: one for six-class semantic segmentation and one for HV gradient regression enabling watershed-based nuclear instance separation. It is trained via a three-stage progressive pseudo-label curriculum, scaling from PanNuke (Stage 1, 0.25um/pixel) to TCGA-UT Scale-0 (Stage 2, 0.5um/pixel) and full 1.6M-patch, six-scale TCGA-UT (Stage 3, 0.5 to 1.0um/pixel). TCGA-UT's coarser, broader per-patch context than PanNuke's also permits a larger tile stride during whole-slide inference. This pipeline computes 22 per-patch compositional, morphological, spatial-entropy, and intercellular-distance metrics and translates them into six categorical phenotype labels and a standardised biological-token vocabulary, fine-tuned via NVIDIA BioNeMo that converts into clinically grounded narratives whose individual claims can be spot-checked directly against the underlying features. Qualitative validation on IGNITE NSCLC tiles shows the pipeline produces biologically coherent phenotype classifications and narratives despite inter-institutional stain variability and imperfect segmentation. The pseudo-labelled TCGA-UT dataset and UNI2-UperHoVer checkpoints are publicly released to support large-scale TME profiling and spatial biology research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
