Purpose: Accurate estimation of the position and orientation (pose) of surgical instruments is crucial for delicate minimally invasive temporal bone surgery. Current techniques lack in accuracy and/or line-of-sight constraints (conventional tracking systems) or expose the patient to prohibitive ionizing radiation (intra-operative CT). A possible solution is to capture the instrument with a c-arm at irregular intervals and recover the pose from the image.
Methods: i3PosNet infers the position and orientation of instruments from images using a pose estimation network. Said framework considers localized patches and outputs pseudo-landmarks. The pose is reconstructed from pseudo-landmarks by geometric considerations.
Results: We show i3PosNet reaches errors less than 0.05mm. It outperforms conventional image registration-based approaches reducing average and maximum errors by at least two thirds. i3PosNet trained on synthetic images generalizes to real x-rays without any further adaptation.
Conclusion: The translation of Deep Learning based methods to surgical applications is difficult, because large representative datasets for training and testing are not available. This work empirically shows sub-millimeter pose estimation trained solely based on synthetic training data.
Deep learning based automatic segmentation of lumbosacral nerves on non-contrast CT for radiographic evaluation: a pilot study
Spinal Pain Research Institute of Tongji University(同济大学脊柱疼痛研究所);Brigham and Women's Hospital, Harvard Medical School(哈佛医学院布莱根妇女医院);School of Data and Computer Science, Sun Yat-sen University(中山大学数据科学与计算机学院);Shanghai Jiao Tong University School of Medicine(上海交通大学医学院);Graduate School of Frontier Sciences, The University of Tokyo(东京大学前沿科学研究生院);Burke Neurological Institute(伯克神经学研究所)
TraSeTR: Track-to-Segment Transformer with Contrastive Query for Instance-level Instrument Segmentation in Robotic Surgery
The Chinese University of Hong Kong(香港中文大学);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University College London(伦敦大学学院)
Real-Time Instrument Segmentation in Robotic Surgery using Auxiliary Supervised Deep Adversarial Learning
NUS Graduate School for Integrative Sciences and Engineering (NGS)(新加坡国立大学综合科学与工程研究生院(NGS));National University of Singapore(新加坡国立大学);University of Moratuwa(莫拉图瓦大学);NIT Trichy(印度国立理工学院蒂鲁吉拉伯利分校)
Learning Where to Look While Tracking Instruments in Robot-assisted Surgery
NUS Graduate School for Integrative Sciences and Engineering (NGS), National University of Singapore(新加坡国立大学NUS综合科学与工程研究生院(NGS));National University of Singapore(新加坡国立大学);University of Michigan-Joint Institute, Shanghai Jiao Tong University(上海交通大学密歇根联合学院)
ST-MTL: Spatio-Temporal Multitask Learning Model to Predict Scanpath While Tracking Instruments in Robotic Surgery
NUS Graduate School for Integrative Sciences and Engineering (NGS), National University of Singapore(新加坡国立大学综合科学与工程研究生院);Dept. of Biomedical Engineering, National University of Singapore(新加坡国立大学生物医学工程系);Dept. of Instrumentation and Control Engineering, NIT Trichy(印度国立理工学院蒂鲁奇拉帕利分校仪器与控制工程系);Dept. of Otolaryngology, Singapore General Hospital(新加坡中央医院耳鼻喉科)
CT synthesis from MR images for orthopedic applications in the lower arm using a conditional generative adversarial network
Image Sciences Institute, University Medical Center Utrecht(乌得勒支大学医学中心影像科学研究所);Department of Orthopedics, University Medical Center Utrecht(乌得勒支大学医学中心骨科);MRIguidance B.V.(MRIguidance有限公司(乌得勒支));University Medical Center Utrecht(乌得勒支大学医学中心)
hSDB-instrument: Instrument Localization Database for Laparoscopic and Robotic Surgeries
hutom;Yonsei University College of Medicine(延世大学医学院)
手术器械的自动定位是了解手术过程并对其进行分析从而为外科医生提供有意义的术中指导或术后手术指标的重要技术。我们引入了一个新的数据集,反映了手术器械的运动学特征,用于手术视频的自动手术器械定位。hSDB(hutom Surgery DataBase)-仪器数据集由24例腹腔镜胆囊切除术和24例机器人胃切除术的仪器定位信息组成。所有仪器的定位信息以用于目标检测的限界框的形式提供。为了处理仪器之间的类别不平衡问题,在Unity中为3D模型建模的合成仪器被包括作为训练数据。此外,对于3D仪器数据,提供了多边形注释以实现工具的实例分割。为了反映所有器械的运动学特征,腹腔镜器械用头部和身体部位注释,机器人器械分别用头部、手腕和身体部位注释。辅助工具(标本袋、针头等)标注数据经常用于外科手术的也包括在内。此外,我们提供了统计信息的hSDB仪器数据集和基线定位性能的目标检测网络训练的MMDetection库和分析结果。
Comparative evaluation of instrument segmentation and tracking methods in minimally invasive surgery
National Center for Tumor Diseases (NCT)(国家肿瘤疾病中心(NCT));Intuitive Surgical(直觉外科公司);Université Grenoble Alpes(格勒诺布尔阿尔卑斯大学);CNRS(法国国家科学研究中心);CHU Grenoble Alpes(格勒诺布尔阿尔卑斯大学医院);Grenoble INP(格勒诺布尔综合理工学院);TIMC-IMAG(TIMC-IMAG实验室);Centre for Medical Image Computing(医学影像计算中心);University College London(伦敦大学学院);Department of General, Visceral and Transplant Surgery, University of Heidelberg(海德堡大学普通、内脏和移植外科系);ARTORG Center, University of Bern(伯尔尼大学ARTORG中心);Computer Aided Medical Procedures, Johns Hopkins University(约翰霍普金斯大学计算机辅助医疗程序);Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology (KIT)(卡尔斯鲁厄理工学院(KIT)人机工程与机器人研究所)
Video-Instrument Synergistic Network for Referring Video Instrument Segmentation in Robotic Surgery
Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Imperial College London(帝国理工学院);Hong Kong University of Science and Technology(香港科技大学);The General Hospital of Southern Theatre Command of PLA(中国人民解放军南部战区总医院);National University of Singapore(新加坡国立大学)
Surgical-DeSAM: Decoupling SAM for Instrument Segmentation in Robotic Surgery
Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS)(Wellcome/EPSRC介入与外科科学中心(WEISS));University College London(伦敦大学学院);Dept. of Computer Science, University College London(伦敦大学学院计算机科学系)
SAM Meets Robotic Surgery: An Empirical Study on Generalization, Robustness and Adaptation
University College London(伦敦大学学院);National University of Singapore(新加坡国立大学);Hubei University of Technology(湖北工业大学);The Chinese University of Hong Kong(香港中文大学)
本文在 EndoVis 2017/2018 上实证评估 SAM 的手术器械零样本分割、扰动鲁棒性与提示表现,并通过 LoRA 微调提出无需提示的 SurgicalSAM。
Video Dataset for Surgical Phase, Keypoint, and Instrument Recognition in Laparoscopic Surgery (PhaKIR)
Regensburg Medical Image Computing (ReMIC), OTH Regensburg(雷根斯堡医学影像计算中心(ReMIC));Regensburg Center of Biomedical Engineering (RCBE), OTH Regensburg(雷根斯堡生物医学工程中心(RCBE));Regensburg University(雷根斯堡大学);Regensburg Center of Health Sciences(雷根斯堡健康科学中心);Technical University of Munich (TUM)(慕尼黑技术大学);TUM University Hospital(慕尼黑技术大学医院);Biomedical Image Analysis Group, Department of Computing, Imperial College London(生物医学影像分析组,计算部门,伦敦帝国学院);Research Group MITI, TUM University Hospital, School of Medicine(MITI研究组,慕尼黑技术大学医院,医学学院);Department of Surgery, TUM University Hospital, School of Medicine(外科部门,慕尼黑技术大学医院,医学学院)
Exploring Deep Learning Methods for Real-Time Surgical Instrument Segmentation in Laparoscopy
SimulaMet(西穆拉大都会实验室);UiT The Arctic University of Norway(挪威北极大学);Oslo Metropolitan University(奥斯陆城市大学);University of Oxford(牛津大学);NIHR Oxford Biomedical Research Centre(英国国家健康研究所牛津生物医学研究中心)
微创手术是一种用于检查腹腔内器官的手术干预,由于其对开放手术的有效性而被广泛使用。由于硬件的改进,如高清晰度摄像机,这一过程已显着改善,新的软件方法已被证明有潜力的计算机辅助程序。然而,在这些外科手术过程中存在改进器械位置的检测和跟踪的挑战和要求。为此,我们评估和比较了一些流行的深度学习方法,这些方法可以用于腹腔镜手术器械的自动分割,这是工具跟踪的重要一步。我们的实验结果表明,与其他最近的深度学习方法相比,双解码器注意力网络(DDANet)产生了更好的结果。DDANet为Robust Medical Instrument Segmentation(ROBUST-MIS)Challenge 2019数据集产生了0.8739的Dice系数和0.8183的平均交叉-联合,实时速度为每秒101.36帧,这对此类程序至关重要。
Deep Learning to Segment Pelvic Bones: Large-scale CT Datasets and Baseline Models
George Mason University(乔治梅森大学);Beijing Electronic Science and Technology Institute(北京电子科技学院);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Beijing Jishuitan Hospital(北京积水潭医院)
SAM Meets Robotic Surgery: An Empirical Study in Robustness Perspective
The Chinese University of Hong Kong(香港中文大学);University College London(伦敦大学学院);National University of Singapore(新加坡国立大学);Hubei University of Technology(湖北工业大学)
Instrument-tissue Interaction Detection Framework for Surgical Video Understanding
National University of Singapore(新加坡国立大学);Research Institute of Trustworthy Autonomous Systems(可信自主系统研究院);Southern University of Science and Technology(南方科技大学);Agency for Science, Technology, and Research(新加坡科技研究局);Osaka University Graduate School of Medicine(大阪大学医学部)
PitVis-2023 Challenge: Workflow Recognition in videos of Endoscopic Pituitary Surgery
University College London(伦敦大学学院);National Hospital for Neurology and Neurosurgery(国立神经学与神经外科医院);Centre for AI and Robotics (CAIR) HKISI, CAS(中国科学院香港创新研究院人工智能与机器人中心);Shanghai Jiao Tong University(上海交通大学)
Privacy-Preserving Synthetic Continual Semantic Segmentation for Robotic Surgery
National University of Singapore(新加坡国立大学);NUSRI Suzhou(新加坡国立大学苏州研究院);University College London(伦敦大学学院);The Chinese University of Hong Kong(香港中文大学);Shun Hing Institute of Advanced Engineering(信兴高等工程研究院)
U-NetPlus: A Modified Encoder-Decoder U-Net Architecture for Semantic and Instance Segmentation of Surgical Instrument
Rochester Institute of Technology(罗切斯特理工学院);Center for Imaging Science(影像科学中心);Biomedical Engineering(生物医学工程)
传统的治疗方法由于有限的视野而限制了外科医生的灵活性控制。随着机器人辅助手术的出现,微创手术的医疗技术发生了范式转变。然而,在手术场景中跟踪手术器械的位置是非常具有挑战性的,并且手术工具的准确检测和识别是至关重要的。手术视频帧中基于深度学习的语义分割有可能促进这一任务。在这项工作中,我们修改了名为U-NetPlus的U-Net架构,通过引入预训练的编码器并重新设计解码器部分,通过使用基于最近邻(NN)插值的上采样操作替换转置卷积操作。为了进一步提高性能,我们还采用了非常快速和灵活的数据增强技术。我们在8 x 225帧机器人手术视频序列上训练了框架,这些视频可通过MICCAI 2017 EndoVis Challenge数据集获得,并在8 x 75帧和2 x 300帧视频上进行了测试。使用我们的U-NetPlus架构,我们报告了90.20%的DICE用于二进制分割,76.26%的DICE用于仪器部件分割,46.07%用于仪器类型(即,所有仪器)分割,优于在这些数据上实施和测试的先前技术的结果。
Imperial College London(伦敦帝国理工学院);Tsinghua University(清华大学);NYU Abu Dhabi(纽约大学阿布扎比分校);Wuhan University(武汉大学);University College London(伦敦大学学院);University of Liverpool(利物浦大学)
Digital Surgery Ltd, Medtronic(美敦力数字外科有限公司);ICube, University of Strasbourg, CNRS, IHU Strasbourg(斯特拉斯堡大学ICube实验室、法国国家科学研究中心、斯特拉斯堡大学医院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Hutom(Hutom公司);Xiamen University(厦门大学);University of Electronic Science and Technology of China(电子科技大学);National University of Singapore(新加坡国立大学);National Institute of Technology, Raipur(赖布尔国家理工学院);King’s College London(伦敦国王学院);University College London(伦敦大学学院)
From Variability To Accuracy: Conditional Bernoulli Diffusion Models with Consensus-Driven Correction for Thin Structure Segmentation
Department of Software Convergence, Seoul Women's University, Seoul, Republic of Korea(首尔女性大学软件融合系);Department of Pediatric Neurosurgery, Craniofacial Reforming and Reconstruction Clinic, Yonsei University College of Medicine, Seoul, Republic of Korea(延世大学医学院小儿神经外科及颅面整复与重建诊所)