Three-dimensional (3D) freehand ultrasound (US) reconstruction without a tracker can be advantageous over its two-dimensional or tracked counterparts in many clinical applications. In this paper, we propose to estimate 3D spatial transformation between US frames from both past and future 2D images, using feed-forward and recurrent neural networks (RNNs). With the temporally available frames, a further multi-task learning algorithm is proposed to utilise a large number of auxiliary transformation-predicting tasks between them. Using more than 40,000 US frames acquired from 228 scans on 38 forearms of 19 volunteers in a volunteer study, the hold-out test performance is quantified by frame prediction accuracy, volume reconstruction overlap, accumulated tracking error and final drift, based on ground-truth from an optical tracker. The results show the importance of modelling the temporal-spatially correlated input frames as well as output transformations, with further improvement owing to additional past and/or future frames. The best performing model was associated with predicting transformation between moderately-spaced frames, with an interval of less than ten frames at 20 frames per second (fps). Little benefit was observed by adding frames more than one second away from the predicted transformation, with or without LSTM-based RNNs. Interestingly, with the proposed approach, explicit within-sequence loss that encourages consistency in composing transformations or minimises accumulated error may no longer be required. The implementation code and volunteer data will be made publicly available ensuring reproducibility and further research.
ImplicitCell: Resolution Cell Modeling of Joint Implicit Volume Reconstruction and Pose Refinement in Freehand 3D Ultrasound
School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）；Tongren Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属同仁医院）
Pose-GuideNet: Automatic Scanning Guidance for Fetal Head Ultrasound from Pose Estimation
Institute of Biomedical Engineering, University of Oxford（牛津大学生物医学工程研究所）；Nuffield Department of Women’s & Reproductive Health, University of Oxford（牛津大学纳菲尔德妇女与生殖健康系）
Nonrigid Reconstruction of Freehand Ultrasound without a Tracker
Centre for Medical Image Computing（医学影像计算中心）；Wellcome/EPSRC Centre for Interventional and Surgical Sciences（惠康/工程与物理科学研究理事会介入与外科科学中心）；Department of Medical Physics and Biomedical Engineering（医学物理与生物医学工程系）；University College London（伦敦大学学院）；School of Biomedical Engineering & Imaging Sciences（生物医学工程与影像科学学院）；King’s College London（伦敦国王学院）
TUS-REC2024: A Challenge to Reconstruct 3D Freehand Ultrasound Without External Tracker
organization= UCL Hawkes Institute, Department of Medical Physics；Biomedical Engineering, University College London , city= London , postcode= WC1E 6BT , country= U.K.；organization= Centre for Bioengineering, Digital Environment Research Institute, School of Engineering；Materials Science, Queen Mary University of London , city= London , postcode= E1 4NS , country= U.K.；organization= National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Health Science Center, Shenzhen University , city= Shenzhen , postcode= 518000 , country= China；organization= School of Artificial Intelligence, Shenzhen University , city= Shenzhen , postcode= 518060 , country= China；organization= Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University , city= Shenzhen , postcode= 518000 , country= China；organization= Marshall Laboratory of Biomedical Engineering, Shenzhen University , city= Shenzhen , postcode= 518000 , country= China；organization= Shenzhen RayShape Medical Technology Inc. , city= Shenzhen , postcode= 518000 , country= China；organization= DKFZ (German Cancer Research Center) Heidelberg , country= Germany；organization= University of Cincinnati, Cincinnati, OH, 45219 , country= USA；organization= Tufts University , country= USA；organization= Hong Kong Centre for Cerebro-cardiovascular Health Engineering, Rm 1115-1119, Building 19W, Hong Kong Science Park , city= Hong Kong SAR , country= China；organization= City University of Hong Kong, Kowloon , city= Hong Kong SAR , country= China；organization= Department of Information Convergence Engineering, Pusan National University , city= Yangsan , country= Korea；organization= School of Biomedical Convergence Engineering, Pusan National University , city= Yangsan , country= Korea；organization= Center for Artificial Intelligence Research, Pusan National University , city= Busan , country= Korea；organization= Department of Medical Physics, University of Wisconsin (UW) School of Medicine；organization= Department of Electrical；organization= Department of Biostatistics；Medical Informatics, UW School of Medicine；organization= Department of Computer Sciences, UW–Madison , city= Madison , postcode= WI 53706 , country= USA；organization= School of Biomedical Engineering \& Imaging Sciences, King’s College London , city= London , postcode= WC2R 2LS , country= U.K.
MoNetV2: Enhanced Motion Network for Freehand 3D Ultrasound Reconstruction
National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, Guangdong, China（国家级医学超声关键技术研发实验室、生物医学工程学院、深圳大学医学院、深圳大学、深圳、广东、中国）；Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, Guangdong, China（医学超声图像计算（MUSIC）实验室、深圳大学、深圳、广东、中国）；Shenzhen RayShape Medical Technology Inc.（深圳RayShape医疗科技有限公司）；Cancer Center, Department of Ultrasound Medicine, Zhejiang Provincial People’s Hospital, Affiliated People’s Hospital of Hangzhou Medical College, Hangzhou, Zhejiang, China（肿瘤中心、超声医学科、浙江省人民医院、杭州医学院附属人民医院、杭州、浙江、中国）；Department of Health Management Center, Qilu Hospital, Cheeloo College of Medicine, Shandong University, Jinan, Shandong, China（健康管理中心、齐鲁医院、山东大学齐鲁医学院、济南、山东、中国）
RapidVol: Rapid Reconstruction of 3D Ultrasound Volumes from Sensorless 2D Scans
University of Oxford（牛津大学）；Visual Geometry Group, University of Oxford（牛津大学视觉几何组）；Oxford Machine Learning in NeuroImaging Lab, University of Oxford（牛津大学神经影像机器学习实验室）
A Unified Platform and Quality Assurance Framework for 3D Ultrasound Reconstruction with Robotic, Optical, and Electromagnetic Tracking
School of Computer Science, University of Leeds（利兹大学计算机科学学院）；School of Electronic and Electrical Engineering, University of Leeds（利兹大学电子与电气工程学院）；St James’s University Hospital, Leeds Teaching Hospitals NHS Trust（利兹教学医院，利兹教学医院国家健康服务信托基金）；Leeds Institute of Medical Research, University of Leeds（利兹医学研究所，利兹大学）
