Global-Reasoned Multi-Task Learning Model for Surgical Scene Understanding
Biomedical Image Analysis Group, Imperial College London（伦敦帝国理工学院生物医学图像分析组）；Dept. of Biomedical Engineering, National University of Singapore（新加坡国立大学生物医学工程系）；Dept. of Electronics and Communication Engineering, National Institute of Technology, Tiruchirappalli（蒂鲁吉拉帕利国家技术学院电子与通信工程系）；Dept. of Electronic Engineering, The Chinese University of Hong Kong（香港中文大学电子工程系）
Lalithkumar Seenivasan, Sai Mitheran, Mobarakol Islam, Hongliang Ren
Global and local relational reasoning enable scene understanding models to perform human-like scene analysis and understanding. Scene understanding enables better semantic segmentation and object-to-object interaction detection. In the medical domain, a robust surgical scene understanding model allows the automation of surgical skill evaluation, real-time monitoring of surgeon's performance and post-surgical analysis. This paper introduces a globally-reasoned multi-task surgical scene understanding model capable of performing instrument segmentation and tool-tissue interaction detection. Here, we incorporate global relational reasoning in the latent interaction space and introduce multi-scale local (neighborhood) reasoning in the coordinate space to improve segmentation. Utilizing the multi-task model setup, the performance of the visual-semantic graph attention network in interaction detection is further enhanced through global reasoning. The global interaction space features from the segmentation module are introduced into the graph network, allowing it to detect interactions based on both node-to-node and global interaction reasoning. Our model reduces the computation cost compared to running two independent single-task models by sharing common modules, which is indispensable for practical applications. Using a sequential optimization technique, the proposed multi-task model outperforms other state-of-the-art single-task models on the MICCAI endoscopic vision challenge 2018 dataset. Additionally, we also observe the performance of the multi-task model when trained using the knowledge distillation technique. The official code implementation is made available in GitHub.
Sound Source Localization for Spatial Mapping of Surgical Actions in Dynamic Scenes
ROCS（罗克辛研究所）；Balgrist University Hospital（巴尔格里斯大学医院）；University of Zurich（苏黎世大学）；Dept. of Orthopedics（骨科部门）；Computer Vision and Geometry Group（计算机视觉与几何组）；ETH Zurich（苏黎世联邦理工学院）
Surgical-VQA: Visual Question Answering in Surgical Scenes using Transformer
National University of Singapore（新加坡国立大学）；Imperial College London（帝国理工学院）；National Institute of Technology, Tiruchirappalli（蒂鲁吉拉伯利国家理工学院）；Chinese University of Hong Kong（香港中文大学）；Shun Hing Institute of Advanced Engineering（信兴高等工程研究所）
SurgTPGS: Semantic 3D Surgical Scene Understanding with Text Promptable Gaussian Splatting
The Chinese University of Hong Kong, Hong Kong SAR, China（香港中文大学）；Shenzhen Research Institute, CUHK, Shenzhen, China（深圳研究学院）；Technical University of Munich, Munich, Germany（慕尼黑技术大学）；University of Strasbourg & IHU Strasbourg, Strasbourg, France（斯特拉斯堡大学及斯特拉斯堡IHU）；University College London, London, United Kingdom（伦敦大学学院）
Endo-NeRF++: Uncertainty-Aware Neural Rendering with Multi-Resolution Hash Encoding for Dynamic Surgical Scene Reconstruction
Finnish Center for Artificial Intelligence (FCAI)（芬兰人工智能中心）；ELLIS Unit Helsinki（欧洲学习与智能系统研究所赫尔辛基分部）；University of Helsinki（赫尔辛基大学）；Institute for Urban Studies (Urbaria)（城市研究所）；Institute of Sustainability Science (HELSUS)（可持续发展科学研究所）
Task-Aware Asynchronous Multi-Task Model with Class Incremental Contrastive Learning for Surgical Scene Understanding
National University of Singapore（新加坡国立大学）；Imperial College London（伦敦帝国理工学院）；Singapore General Hospital（新加坡中央医院）；The Chinese University of Hong Kong（香港中文大学）；Shun Hing Institute of Advanced Engineering（信兴高等工程研究所）
Generalizing Surgical Instruments Segmentation to Unseen Domains with One-to-Many Synthesis
Shun Hing Institute of Advanced Engineering（顺兴高等研究院）；CUHK（香港中文大学）；Wellcome / EPSRC Centre for Interventional and Surgical Sciences（惠康/英国工程与自然科学研究委员会介入与外科科学中心）；UCL（伦敦大学学院）；NUS（新加坡国立大学）
