arXiv 2506.237592025-08-10eess.IVcs.CV

联邦学习中手术视频仪器分割的时空表示解耦与增强

Spatio-Temporal Representation Decoupling and Enhancement for Federated Instrument Segmentation in Surgical Videos

Department of Electrical and Computer Engineering, NUS, Singapore（新加坡国立大学电子与计算机工程系）；Department of Biomedical Engineering, NUS, Singapore（新加坡国立大学生物医学工程系）；Institute of High Performance Computing, A*STAR, Singapore（新加坡A*STAR高性能计算研究所）；Department of Computer Science and Engineering, The Chinese University of Hong Kong, HKSAR, China（香港中文大学（深圳）计算机科学与工程系）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China（中国科学院深圳先进技术研究所）

Zheng Fang, Xiaoming Qi, Chun-Mei Feng, Jialun Pei, Weixin Si, Yueming Jin

DOI：: 10.1109/TMI.2026.3651254

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出FedST方法，通过解耦和增强时空表示，提升联邦学习中手术视频仪器分割的性能。

摘要：

在联邦学习（FL）下进行手术仪器分割是一个有前景的方向，它使得多个手术部位能够协同训练模型而无需集中数据集。然而，手术数据科学中存在非常有限的FL工作，且针对其他模态的FL方法并未考虑手术领域内固有的特征：i）不同场景显示多样的解剖背景，但仪器表示却高度相似；ii）存在促进大规模合成数据生成的手术模拟器，只需最小的努力。在本文中，我们提出了一种新的个性化FL方案，即Spatio-Temporal Representation Decoupling and Enhancement（FedST），在本地站点和全局服务器训练过程中巧妙利用手术领域知识以提升分割性能。具体而言，我们的模型在本地站点训练中采用Representation Separation and Cooperation（RSC）机制，将查询嵌入层解耦以私有训练，以编码各自背景。同时，其他参数在全球范围内优化以捕捉仪器的一致表示，包括时间层以捕捉相似的运动模式。此外，进一步设计了文本引导的通道选择，以突出站点特定特征，促进模型适应每个站点。此外，在全局服务器训练中，我们提出Synthesis-based Explicit Representation Quantification（SERQ），该方法基于合成数据定义一个显式的表示目标，以在融合过程中同步模型收敛，从而提高模型的泛化能力。

英文摘要：

Surgical instrument segmentation under Federated Learning (FL) is a promising direction, which enables multiple surgical sites to collaboratively train the model without centralizing datasets. However, there exist very limited FL works in surgical data science, and FL methods for other modalities do not consider inherent characteristics in surgical domain: i) different scenarios show diverse anatomical backgrounds while highly similar instrument representation; ii) there exist surgical simulators which promote large-scale synthetic data generation with minimal efforts. In this paper, we propose a novel Personalized FL scheme, Spatio-Temporal Representation Decoupling and Enhancement (FedST), which wisely leverages surgical domain knowledge during both local-site and global-server training to boost segmentation. Concretely, our model embraces a Representation Separation and Cooperation (RSC) mechanism in local-site training, which decouples the query embedding layer to be trained privately, to encode respective backgrounds. Meanwhile, other parameters are optimized globally to capture the consistent representations of instruments, including the temporal layer to capture similar motion patterns. A textual-guided channel selection is further designed to highlight site-specific features, facilitating model adapta tion to each site. Moreover, in global-server training, we propose Synthesis-based Explicit Representation Quantification (SERQ), which defines an explicit representation target based on synthetic data to synchronize the model convergence during fusion for improving model generalization.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。