ColoDiff:整合动态一致性与内容感知用于结肠镜视频生成
ColoDiff: Integrating Dynamic Consistency With Content Awareness for Colonoscopy Video Generation
AI总结:
ColoDiff通过整合动态一致性与内容感知,生成高质量的结肠镜视频,以缓解数据短缺并辅助临床分析。
摘要:
结肠镜视频生成提供了动态且信息丰富的数据,对于诊断肠道疾病至关重要,特别是在数据稀缺的情况下。高质量视频生成需要时间一致性和对临床属性的精确控制,但面临挑战,包括不规则的肠道结构、多样的疾病表现和各种成像模式。为此,我们提出了ColoDiff,一种基于扩散的框架,旨在生成动态一致且内容感知的结肠镜视频,以缓解数据短缺并协助临床分析。在帧间层面,我们的TimeStream模块通过跨帧标记机制解耦时间依赖性,使在不规则肠道结构下能够实现复杂的动态建模。在帧内层面,我们的Content-Aware模块结合噪声注入嵌入和可学习原型,以实现对临床属性的精确控制,突破扩散模型的粗略指导。此外,ColoDiff采用非马尔可夫采样策略,将步骤减少超过90%以实现实时生成。ColoDiff在三个公开数据集和一个医院数据库上进行评估,基于生成指标和下游任务,包括疾病诊断、模态辨别、肠道准备评分和病变分割。大量实验表明,ColoDiff生成的视频具有平滑的过渡和丰富的动态。ColoDiff在可控的结肠镜视频生成中做出了努力,揭示了合成视频在补充真实表示和缓解临床设置中数据稀缺的潜力。
英文摘要:
Colonoscopy video generation delivers dynamic, information-rich data critical for diagnosing intestinal diseases, particularly in data-scarce scenarios. High-quality video generation demands temporal consistency and precise control over clinical attributes, but faces challenges from irregular intestinal structures, diverse disease representations, and various imaging modalities. To this end, we propose ColoDiff, a diffusion-based framework that generates dynamic-consistent and content-aware colonoscopy videos, aiming to alleviate data shortage and assist clinical analysis. At the inter-frame level, our TimeStream module decouples temporal dependency from video sequences through a cross-frame tokenization mechanism, enabling intricate dynamic modeling despite irregular intestinal structures. At the intra-frame level, our Content-Aware module incorporates noise-injected embeddings and learnable prototypes to realize precise control over clinical attributes, breaking through the coarse guidance of diffusion models. Additionally, ColoDiff employs a non-Markovian sampling strategy that cuts steps by over 90% for real-time generation. ColoDiff is evaluated across three public datasets and one hospital database, based on both generation metrics and downstream tasks including disease diagnosis, modality discrimination, bowel preparation scoring, and lesion segmentation. Extensive experiments show ColoDiff generates videos with smooth transitions and rich dynamics. ColoDiff presents an effort in controllable colonoscopy video generation, revealing the potential of synthetic videos in complementing authentic representation and mitigating data scarcity in clinical settings.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。