发表机构
Northeastern University; Memorial Sloan Kettering Cancer Center（东北大学; 纪念斯隆-凯特琳癌症中心）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
FloVMos是一个基于光流的深度学习视频拼接框架，通过合成数据微调实现跨七种生物医学成像模式的鲁棒实时拼接，在准确性、鲁棒性和速度上优于传统方法。
AI 中文摘要
生物医学成像模式通常需要在分辨率、视场（FOV）和采集速度之间进行权衡。视频拼接通过将连续的、高分辨率的帧计算拼接成宽视场复合图像，提供了一种克服这一限制的策略。然而，现有方法在处理非刚性变形以及临床和研究成像中出现的模态特定伪影方面存在困难。在此，我们提出了FloVMos，一个通用的、基于光流的深度学习框架，用于跨多种生物医学成像模式的实时视频拼接。FloVMos通过在具有真实变形场的合成训练数据上微调光流模型，实现了鲁棒的、像素级配准。我们引入了一个用于生成这些训练数据的流程，该流程模拟来自现有拼接图或原始视频的真实组织运动和成像畸变。我们基于这些数据的自动化合成数据生成和光流模型训练，使用户能够将FloVMos适应于不同的成像模式。为了展示这一点，我们将FloVMos应用于七种不同的成像模式：反射共聚焦显微镜、开放式光片显微镜、胎儿镜、腹腔镜、皮肤镜、稀疏光谱显微镜和内窥镜。在所有这些测试模式中，FloVMos在准确性、鲁棒性和速度方面均优于传统基线。这种适应性强且训练高效的框架能够实现实时性能的大面积可视化，并可能支持基于视频的生物医学成像在研究和临床工作流程中的更广泛应用。
英文摘要
Biomedical imaging modalities often require a trade-off among resolution, field of view (FOV), and acquisition speed. Video mosaicking offers a strategy to overcome this limitation by computationally stitching sequential high-resolution frames into a wide-FOV composite. However, existing methods struggle with non-rigid deformations, and modality-specific artifacts arising in clinical and research imaging. Here, we present FloVMos, a generalizable, optical-flow-based deep learning framework for real-time video mosaicking across diverse biomedical imaging modalities. FloVMos achieves robust, pixel-level registration by fine-tuning an optical flow model on synthetic training data with ground-truth deformation fields. We introduce a pipeline for generating this training data, simulating realistic tissue motion and imaging distortions from existing mosaics or raw videos. Our automated synthetic data generation and optical flow model training based on this data allow users to adapt FloVMos to different imaging modalities. To demonstrate this, we applied FloVMos to seven diverse imaging modalities: reflection confocal microscopy, open-top light-sheet microscopy, fetoscopy, laparoscopy, dermoscopy, sparse spectral microscopy, and endoscopy. FloVMos outperforms conventional baselines in accuracy, robustness, and speed for all the tested modalities. This adaptable and training-efficient framework enables large-area visualization with real-time performance and may support broader use of video-based biomedical imaging in research and clinical workflows.
