Arges：用于内窥镜视频中溃疡性结肠炎严重程度评估的时空Transformer
Arges: Spatio-Temporal Transformer for Ulcerative Colitis Severity Assessment in Endoscopy Videos
AI总结：
本文提出Arges框架，利用带位置编码的Transformer整合时空信息，并在大规模多中心临床试验数据上预训练基础模型ArgesFM，显著提升内窥镜视频中UC严重程度评分的准确性和泛化能力。
摘要：
在溃疡性结肠炎（UC）中，从内窥镜视频准确评估疾病严重程度对于临床试验中的药物疗效评价至关重要。严重程度通常通过Mayo内窥镜子评分（MES）和溃疡性结肠炎内窥镜严重程度指数（UCEIS）评分来衡量。然而，专家进行MES/UCEIS标注耗时且易受评分者间差异影响，这些因素可以通过自动化来解决。由于临床试验中视频级标签的普遍存在，基于帧级标签的全监督解决方案在自动化尝试中面临挑战。基于CNN的弱监督模型（WSL）采用端到端（e2e）训练，缺乏对新疾病评分的泛化能力，并且忽略了对于准确评分至关重要的时空信息。为解决这些局限性，我们提出“Arges”，一个深度学习框架，利用带有位置编码的transformer来整合来自帧特征的时空信息，以估计内窥镜视频中的疾病严重程度评分。提取的特征来源于一个基础模型（ArgesFM），该模型在来自多个临床试验的大型多样化数据集（6100万帧，3927个视频）上进行了预训练。我们评估了四种UC疾病严重程度评分，包括MES和三个UCEIS组成评分。测试集评估表明有显著改进，与最先进方法相比，MES的F1分数提高了4.1%，三个UCEIS组成评分的F1分数分别提高了18.8%、6.6%和3.8%。在先前未见过的临床试验数据上进行的前瞻性验证进一步证明了该模型成功的泛化能力。
英文摘要：
Accurate assessment of disease severity from endoscopy videos in ulcerative colitis (UC) is crucial for evaluating drug efficacy in clinical trials. Severity is often measured by the Mayo Endoscopic Subscore (MES) and Ulcerative Colitis Endoscopic Index of Severity (UCEIS) score. However, expert MES/UCEIS annotation is time-consuming and susceptible to inter-rater variability, factors addressable by automation. Automation attempts with frame-level labels face challenges in fully-supervised solutions due to the prevalence of video-level labels in clinical trials. CNN-based weakly-supervised models (WSL) with end-to-end (e2e) training lack generalization to new disease scores and ignore spatio-temporal information crucial for accurate scoring. To address these limitations, we propose "Arges", a deep learning framework that utilizes a transformer with positional encoding to incorporate spatio-temporal information from frame features to estimate disease severity scores in endoscopy video. Extracted features are derived from a foundation model (ArgesFM), pre-trained on a large diverse dataset from multiple clinical trials (61M frames, 3927 videos). We evaluate four UC disease severity scores, including MES and three UCEIS component scores. Test set evaluation indicates significant improvements, with F1 scores increasing by 4.1% for MES and 18.8%, 6.6%, 3.8% for the three UCEIS component scores compared to state-of-the-art methods. Prospective validation on previously unseen clinical trial data further demonstrates the model's successful generalization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
