Surgical Action Triplet Detection by Mixed Supervised Learning of Instrument-Tissue Interactions
ICube Laboratory（ICube实验室）；University of Strasbourg（斯特拉斯堡大学）；IHU Strasbourg（斯特拉斯堡大学医院研究所）；University Hospital of Strasbourg（斯特拉斯堡大学医院）；CNRS（法国国家科学研究中心）
Saurav Sharma, Chinedu Innocent Nwoye, Didier Mutter, Nicolas Padoy
Surgical action triplets describe instrument-tissue interactions as (instrument, verb, target) combinations, thereby supporting a detailed analysis of surgical scene activities and workflow. This work focuses on surgical action triplet detection, which is challenging but more precise than the traditional triplet recognition task as it consists of joint (1) localization of surgical instruments and (2) recognition of the surgical action triplet associated with every localized instrument. Triplet detection is highly complex due to the lack of spatial triplet annotation. We analyze how the amount of instrument spatial annotations affects triplet detection and observe that accurate instrument localization does not guarantee better triplet detection due to the risk of erroneous associations with the verbs and targets. To solve the two tasks, we propose MCIT-IG, a two-stage network, that stands for Multi-Class Instrument-aware Transformer-Interaction Graph. The MCIT stage of our network models per class embedding of the targets as additional features to reduce the risk of misassociating triplets. Furthermore, the IG stage constructs a bipartite dynamic graph to model the interaction between the instruments and targets, cast as the verbs. We utilize a mixed-supervised learning strategy that combines weak target presence labels for MCIT and pseudo triplet labels for IG to train our network. We observed that complementing minimal instrument spatial annotations with target embeddings results in better triplet detection. We evaluate our model on the CholecT50 dataset and show improved performance on both instrument localization and triplet detection, topping the leaderboard of the CholecTriplet challenge in MICCAI 2022.
University of Zagreb（萨格勒布大学）；University of Twente（特文特大学）；University of Groningen（格罗宁根大学）；Radboud University Medical Center（拉德堡德大学医学中心）；Canisius-Wilhelmina Hospital（卡尼修斯-威廉明娜医院）
Rendezvous in Time: An Attention-based Temporal Fusion approach for Surgical Triplet Recognition
ICube Laboratory（ICube实验室）；University of Strasbourg（斯特拉斯堡大学）；IHU Strasbourg（斯特拉斯堡大学医院研究所）；University Hospital of Strasbourg（斯特拉斯堡大学医院）；CNRS（法国国家科学研究中心）
ProstaTD: Bridging Surgical Triplet from Classification to Fully Supervised Detection
School of Nursing, The Hong Kong Polytechnic University（香港理工大学护理学院）；Nanfang Hospital, Southern Medical University（南方医科大学南芳医院）；Department of Surgery, The Chinese University of Hong Kong（香港中文大学外科部）；South China University of Technology（华南理工大学）；Singapore Management University（新加坡国立管理学院）
SPIRIT: Spatio-temporal Pairwise Relational Modeling of Instrument-Tissue Interactions for Surgical Action Triplet Recognition
University of Strasbourg（斯特拉斯堡大学）；CNRS（法国国家科学研究中心）；INSERM（法国国家健康与医学研究院）；ICube（ICube实验室）；IHU Strasbourg（斯特拉斯堡大学医院研究所）；University of Pavia（帕维亚大学）；University Digestive Health Care Center – Clarunis（克拉鲁尼斯大学消化保健中心）；Università di Roma Tor Vergata（罗马第二大学）；University Hospital of Strasbourg（斯特拉斯堡大学医院）；University Hospital Zurich（苏黎世大学医院）；University Hospital Bern（伯尔尼大学医院）；University of Basel（巴塞尔大学）
Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videos
ICube, University of Strasbourg, CNRS（斯特拉斯堡大学ICube研究所，法国国家科学研究中心）；IHU Strasbourg（斯特拉斯堡大学医院研究所）；University Hospital of Strasbourg（斯特拉斯堡大学医院）；Fondazione Policlinico Universitario Agostino Gemelli IRCCS（阿戈斯蒂诺·杰梅利大学综合医院基金会IRCCS）；IRCAD France（法国消化癌症研究研究所）
fine-CLIP: Enhancing Zero-Shot Fine-Grained Surgical Action Recognition with Vision-Language Models
University of Strasbourg（斯特拉斯堡大学）；CNRS（法国国家科学研究中心）；INSERM（法国国家健康与医学研究院）；ICube（ICube实验室）；IHU Strasbourg（斯特拉斯堡大学医院研究所）；University Hospital of Strasbourg（斯特拉斯堡大学医院）
TRCoRSurg: Temporal-Relational Co-Reasoning for Surgical Video Triplet Recognition
Beihang University（北京航空航天大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Shenzhen University of Advanced Technology（深圳理工大学）
University of Strasbourg（斯特拉斯堡大学）；Fondazione Policlinico Universitario Agostino Gemelli IRCCS（阿戈斯蒂诺·杰梅里基金会天主教大学综合医院）；IHU Strasbourg（斯特拉斯堡大学医院研究所）；University of Basel（巴塞尔大学）
Instrument-tissue Interaction Detection Framework for Surgical Video Understanding
National University of Singapore（新加坡国立大学）；Research Institute of Trustworthy Autonomous Systems（可信自主系统研究院）；Southern University of Science and Technology（南方科技大学）；Agency for Science, Technology, and Research（新加坡科技研究局）；Osaka University Graduate School of Medicine（大阪大学医学部）
CholecTriplet2021: A benchmark challenge for surgical action triplet recognition
University of Strasbourg（斯特拉斯堡大学）；CNRS（法国国家科学研究中心）；IHU Strasbourg（斯特拉斯堡大学医院研究所）；University of Chicago（芝加哥大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Hefei University of Technology（合肥工业大学）；Shanghai Jiao Tong University（上海交通大学）；Argonne National Laboratory（阿贡国家实验室）；Medtronic（美敦力）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
SuPer: A Surgical Perception Framework for Endoscopic Tissue Manipulation with Surgical Robotics
University of California San Diego（加利福尼亚大学圣地亚哥分校）；Zhejiang University（浙江大学）；Alibaba-Zhejiang University Joint Research Institute of Frontier Technologies（阿里巴巴-浙江大学前沿技术联合研究院）
SAGS: Self-Adaptive Alias-Free Gaussian Splatting for Dynamic Surgical Endoscopic Reconstruction
University of Technology Sydney, Australia（悉尼技术大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, China（深圳先进技术研究所）；Shenyang Institute of Automation, Chinese Academy of Sciences, China（沈阳自动化研究所）
Imperial College London（伦敦帝国理工学院）；Tsinghua University（清华大学）；NYU Abu Dhabi（纽约大学阿布扎比分校）；Wuhan University（武汉大学）；University College London（伦敦大学学院）；University of Liverpool（利物浦大学）
Dynamic Interactive Relation Capturing via Scene Graph Learning for Robotic Surgical Report Generation
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；University College London（伦敦大学学院）；The Hong Kong University of Science and Technology（香港科技大学）
SurgicalGaussian: Deformable 3D Gaussians for High-Fidelity Surgical Scene Reconstruction
Center for Digital Media Computing（数字媒体计算中心）；School of Film（电影学院）；School of Informatics（信息学院）；National Institute for Data Science in Health and Medicine（健康与医学数据科学国家研究院）；Xiamen University（厦门大学）；Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan, Ministry of Culture and Tourism（文化和旅游部闽台非物质文化遗产数字化保护与智能处理重点实验室）；Guangdong Provincial Key Laboratory IRADS（广东省IRADS重点实验室）；BNU-HKBU United International College（北京师范大学-香港浸会大学联合国际学院）；The University of Texas at Dallas（德克萨斯大学达拉斯分校）
CholecTriplet2022: Show me a tool and tell me the triplet -- an endoscopic vision challenge for surgical action triplet detection
University of Strasbourg（斯特拉斯堡大学）；CNRS（法国国家科学研究中心）；IHU Strasbourg（斯特拉斯堡大学医院研究所）；Technical University Munich（慕尼黑工业大学）；Riwolink GmbH（Riwolink有限公司）；German Cancer Research Center (DKFZ)（德国癌症研究中心）；Shanghai Jiao Tong University（上海交通大学）；IPCA（葡萄牙卡瓦多和阿韦河理工学院）；University of Minho（米尼奥大学）
Towards Unified Surgical Scene Understanding:Bridging Reasoning and Grounding via MLLMs
Southern University of Science and Technology（南方科技大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究院，中国科学院）；Northwestern University（西北大学）；University of Alberta（阿尔伯塔大学）；Yale University（耶鲁大学）；Nanfang Hospital（南华医院）；Shenzhen University of Advanced Technology（深圳大学先进技术研究院）
Tracking Any Point Methods for Markerless 3D Tissue Tracking in Endoscopic Stereo Images
Institute of Medical Technology and Intelligent Systems, Hamburg University of Technology（医学技术与智能系统研究所，汉堡技术大学）；Department of Otorhinolaryngology, Head and Neck Surgery and Oncology, University Medical Center Hamburg-Eppendorf（耳鼻喉头颈外科与肿瘤学部，汉堡艾本多夫大学医学中心）；Martini-Klinik Prostate Cancer Center, University Medical Center Hamburg-Eppendorf（马尔廷诊所前列腺癌中心，汉堡艾本多夫大学医学中心）；Department of Urology, University Medical Center Hamburg-Eppendorf（泌尿外科，汉堡艾本多夫大学医学中心）
本文提出一种利用2D Tracking Any Point网络结合两个CoTracker模型实现内窥镜立体图像无标记3D组织追踪的新方法，实验验证了其在复杂手术场景中的准确性与潜力。
CurConMix+: A Unified Spatio-Temporal Framework for Hierarchical Surgical Workflow Understanding
Department of Medical Device Management and Research, Samsung Advanced Institute for Health Sciences & Technology (SAIHST), Sungkyunkwan University（医学院设备管理与研究系，三星健康科学与技术研究院（SAIHST），全北大学）；Clinical Robotics and Embodied AI Research Center, Smart Healthcare Research Institute, Research Institute for Future Medicine, Samsung Medical Center（临床机器人与具身人工智能研究中心，智能医疗研究机构，未来医学研究所，三星医疗中心）；Department of Surgery, Samsung Medical Center（外科系，三星医疗中心）
State-Change Learning for Prediction of Future Events in Endoscopic Videos
University of Strasbourg, CNRS, INSERM, ICube, UMR7357, France（斯特拉斯堡大学，法国国家科学研究中心，法国国家医学研究院，ICube，UMR7357，法国）；University Hospital of Strasbourg, France（斯特拉斯堡大学医院，法国）
Unsupervised Surgical Instrument Segmentation via Anchor Generation and Semantic Diffusion
NELVT, Department of Computer Science, Peking University（北京大学计算机科学技术系数字视频编解码技术国家工程实验室）；Deepwise AI Lab（深睿医疗人工智能实验室）；Center on Frontiers of Computing Studies, Peking University（北京大学前沿计算研究中心）；Advanced Institute of Information Technology, Peking University（北京大学信息技术高等研究院）；Peking University Cancer Hospital（北京大学肿瘤医院）
Generalizing Surgical Instruments Segmentation to Unseen Domains with One-to-Many Synthesis
Shun Hing Institute of Advanced Engineering（顺兴高等研究院）；CUHK（香港中文大学）；Wellcome / EPSRC Centre for Interventional and Surgical Sciences（惠康/英国工程与自然科学研究委员会介入与外科科学中心）；UCL（伦敦大学学院）；NUS（新加坡国立大学）
