Medical imaging segmentation is a highly active area of research, with deep learning-based methods achieving state-of-the-art results in several benchmarks. However, the lack of standardized tools for training, testing, and evaluating new methods makes the comparison of methods difficult. To address this, we introduce the Medical Imaging Segmentation Toolkit (MIST), a simple, modular, and end-to-end medical imaging segmentation framework designed to facilitate consistent training, testing, and evaluation of deep learning-based medical imaging segmentation methods. MIST standardizes data analysis, preprocessing, and evaluation pipelines, accommodating multiple architectures and loss functions. This standardization ensures reproducible and fair comparisons across different methods. We detail MIST's data format requirements, pipelines, and auxiliary features and demonstrate its efficacy using the BraTS Adult Glioma Post-Treatment Challenge dataset. Our results highlight MIST's ability to produce accurate segmentation masks and its scalability across multiple GPUs, showcasing its potential as a powerful tool for future medical imaging research and development.
Towards Resource-Efficient Streaming of Large-Scale Medical Image Datasets for Deep Learning
University of Maryland School of Medicine（马里兰大学医学院）；University of Maryland Institute for Health Computing（马里兰大学健康计算研究所）；St. Jude Children’s Research Hospital（圣犹日儿童研究医院）；UTHealth Houston（德克萨斯大学健康科学中心休斯顿分校）；Johns Hopkins University School of Medicine（约翰霍普金斯大学医学院）
In convolutional neural network based medical image segmentation, the periphery of foreground regions representing malignant tissues may be disproportionately assigned as belonging to the background class of healthy tissues \cite{attenUnet}\cite{AttenUnet2018}\cite{InterSeg}\cite{UnetFrontNeuro}\cite{LearnActiveContour}. This leads to high false negative detection rates. In this paper, we propose a novel attention mechanism to directly address such high false negative rates, called Paying Attention to Mistakes. Our attention mechanism steers the models towards false positive identification, wh…
Multimodal Learning for Scalable Representation of High-Dimensional Medical Data
Kimia Lab, Dept. of Artificial Intelligence & Informatics, Mayo Clinic, Rochester, MN, USA（Kimia实验室，人工智能与信息学系，梅奥诊所，罗切斯特，MN，美国）；Division of Computational Biology, Dept. of Quantitative Health Sciences, Mayo Clinic, Rochester, MN, USA（计算生物学部门，定量健康科学系，梅奥诊所，罗切斯特，MN，美国）
Raptor: Scalable Train-Free Embeddings for 3D Medical Volumes Leveraging Pretrained 2D Foundation Models
Computer Science Department, University of California, Los Angeles（计算机科学系，加州大学洛杉矶分校）；Department of Computational Medicine, David Geffen School of Medicine, University of California, Los Angeles（计算医学系，大卫·格芬医学院，加州大学洛杉矶分校）；Bioinformatics Interdepartmental Program, University of California, Los Angeles（生物信息学跨学科项目，加州大学洛杉矶分校）；Department of Human Genetics, University of California, Los Angeles（人类遗传学系，加州大学洛杉矶分校）
MedIQA: A Scalable Foundation Model for Prompt-Driven Medical Image Quality Assessment
Faculty of Applied Sciences, Macao Polytechnic University, Macao, China（澳门理工学院）；Department of Engineer Science, University of Oxford, Oxford, UK（牛津大学工程科学系）；Great Bay University, Dongguan, China（东莞大湾大学）；College of Physics and Information Engineering, Fuzhou University, Fuzhou, China（福州大学物理与信息工程学院）；Shanghai Jiao Tong University, Shanghai, China（上海交通大学）；Department of Radiology, Shenzhen People’s Hospital, Shenzhen, China（深圳人民医院放射科）
Warsaw University of Technology（华沙理工大学）；Medical University of Warsaw（华沙医科大学）；University of Warsaw（华沙大学）
The sudden outbreak and uncontrolled spread of COVID-19 disease is one of the most important global problems today. In a short period of time, it has led to the development of many deep neural network models for COVID-19 detection with modules for explainability. In this work, we carry out a systematic analysis of various aspects of proposed models. Our analysis revealed numerous mistakes made at different stages of data acquisition, model development, and explanation construction. In this work, we overview the approaches proposed in the surveyed Machine Learning articles and indicate typical …
Technical University of Munich（慕尼黑工业大学）；Siemens AG（西门子股份公司）
Diagnosing diseases such as leukemia or anemia requires reliable counts of blood cells. Hematologists usually label and count microscopy images of blood cells manually. In many cases, however, cells in different maturity states are difficult to distinguish, and in combination with image noise and subjectivity, humans are prone to make labeling mistakes. This results in labels that are often not reproducible, which can directly affect the diagnoses. We introduce TIMELY, a probabilistic model that combines pseudotime inference methods with inhomogeneous hidden Markov trees, which addresses this …
BioVFM-21M: Benchmarking and Scaling Self-Supervised Vision Foundation Models for Biomedical Image Analysis
Paul C. Lauterbur Research Center for Biomedical Imaging（生物医学成像保罗·劳特伯研究中心）；Shenzhen Institutes of Advanced Technology（深圳先进技术研究院）；Chinese Academy of Sciences（中国科学院）；Pengcheng Laboratory（鹏城实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Faculty of Applied Sciences（应用科学学院）
University of Burgundy（勃艮第大学）；Centre Georges François Leclerc (CGFL)（乔治·弗朗索瓦·勒克莱尔中心）；Institute of Curie（居里研究所）；University Hospital of Dijon（第戎大学医院）
Deep learning has successfully been leveraged for medical image segmentation. It employs convolutional neural networks (CNN) to learn distinctive image features from a defined pixel-wise objective function. However, this approach can lead to less output pixel interdependence producing incomplete and unrealistic segmentation results. In this paper, we present a fully automatic deep learning method for robust medical image segmentation by formulating the segmentation problem as a recurrent framework using two systems. The first one is a forward system of an encoder-decoder CNN that predicts the …
Exploring scalable medical image encoders beyond text supervision
Health Futures, Microsoft Research（微软研究院健康未来部门）；Microsoft Health and Life Sciences（微软健康与生命科学部门）；Department of Radiology, University of Cambridge（剑桥大学放射学系）；Cambridge University Hospitals NHS Foundation Trust（剑桥大学医院NHS基金会信托）；Microsoft Azure AI（微软Azure人工智能部门）
Fairness in Multi-modal Medical Diagnosis with Demonstration Selection
Arizona State University（亚利桑那州立大学）；University of Rochester（罗切斯特大学）；University of Virginia（弗吉尼亚大学）；UCL（伦敦大学学院）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；University of Central Florida（佛罗里达中央大学）
University College London（伦敦大学学院）；Microsoft Research（微软研究院）
Recent years have seen increasing use of supervised learning methods for segmentation tasks. However, the predictive performance of these algorithms depends on the quality of labels. This problem is particularly pertinent in the medical image domain, where both the annotation cost and inter-observer variability are high. In a typical label acquisition process, different human experts provide their estimates of the "true" segmentation labels under the influence of their own biases and competence levels. Treating these noisy labels blindly as the ground truth limits the performance that automati…
MedCLM: Learning to Localize and Reason via a CoT-Curriculum in Medical Vision-Language Models
A.I.MATICS Inc（A.I.MATICS公司）；Boston University（波士顿大学）；University of Southern California（南加州大学）；Heuron（Heuron公司）；MODULABS, Open Neural Networks Research Lab（MODULABS，开放神经网络研究实验室）
PRS-Med: Position Reasoning Segmentation in Medical Imaging
Aalto University（阿尔托大学）；Northwestern University（西北大学）；Technical University of Denmark（丹麦技术大学）；Chongqing University of Posts and Telecommunications（重庆邮电大学）；University of South Dakota（南达科他大学）
CataractSAM-2: A Domain-Adapted Model for Anterior Segment Surgery Segmentation and Scalable Ground-Truth Annotation
Thomas Jefferson High School for Science and Technology（托马斯·杰弗逊高中科学与技术学校）；Mass Eye and Ear, Harvard Medical School（麻省眼耳研究所，哈佛医学院）；Harvard Ophthalmology AI Lab, Schepens Eye Research Institute of Mass Eye and Ear, Harvard Medical School（哈佛眼科学人工智能实验室，麻省眼耳研究所的谢普恩斯眼研究学院，哈佛医学院）
In knowledge bases such as Wikidata, it is possible to assert a large set of properties for entities, ranging from generic ones such as name and place of birth to highly profession-specific or background-specific ones such as doctoral advisor or medical condition. Determining a preference or ranking in this large set is a challenge in tasks such as prioritisation of edits or natural-language generation. Most previous approaches to ranking knowledge base properties are purely data-driven, that is, as we show, mistake frequency for interestingness. In this work, we have developed a human-annotat…
HyPCA-Net: Advancing Multimodal Fusion in Medical Image Analysis
Indian Institute of Technology Ropar（印度理工学院罗帕尔分校）；RoentGen Health（RoentGen健康公司）；Lulea University of Technology（卢勒奥大学）；QUT（昆士兰科技大学）；RMIT University（皇家墨尔本理工大学）；Curtin University（Curtin大学）；Deakin University（德肯大学）
MedKAN: An Advanced Kolmogorov-Arnold Network for Medical Image Classification
School of Computer Science & Software Engineering, Shenzhen University（深圳大学计算机与软件学院）；School of Computer Science, University of Nottingham Ningbo China（宁波诺丁汉大学计算机学院）
Digital medical informatics and images are commonly used in hospitals today,. Because of the interrelatedness of the radiology department and other departments, especially the intensive care unit and emergency department, the transmission and sharing of medical images has become a critical issue. Our research group has developed a Java-based Distributed Object Medical Imaging Model(DOMIM) to facilitate the rapid development and deployment of medical imaging applications in a distributed environment that can be shared and used by related departments and mobile physiciansDOMIM is a unique suite …
Kakao Brain；Seoul National University（首尔大学）；MILA, Université de Montréal（MILA，蒙特利尔大学）
In this paper, a neural architecture search (NAS) framework is proposed for 3D medical image segmentation, to automatically optimize a neural architecture from a large design space. Our NAS framework searches the structure of each layer including neural connectivities and operation types in both of the encoder and decoder. Since optimizing over a large discrete architecture space is difficult due to high-resolution 3D medical images, a novel stochastic sampling algorithm based on a continuous relaxation is also proposed for scalable gradient based optimization. On the 3D medical image segmenta…
University of Cambridge（剑桥大学）；Cancer Research UK Cambridge Centre（英国癌症研究中心剑桥中心）；Cancer Research UK Cambridge Institute（英国癌症研究中心剑桥研究所）；Danube Private University（多瑙河私立大学）；KTH Royal Institute of Technology（KTH皇家理工学院）
Uncertainty quantification in automated image analysis is highly desired in many applications. Typically, machine learning models in classification or segmentation are only developed to provide binary answers; however, quantifying the uncertainty of the models can play a critical role for example in active learning or machine human interaction. Uncertainty quantification is especially difficult when using deep learning-based models, which are the state-of-the-art in many imaging applications. The current uncertainty quantification approaches do not scale well in high-dimensional real-world pro…
Professional applications like telemedicine often require scalable lossless coding of sensitive data. 3-D subband coding has turned out to offer good compression results for dynamic CT data and additionally provides a scalable representation in terms of low- and highpass subbands. To improve the visual quality of the lowpass subband, motion compensation can be incorporated into the lifting structure, but leads to inferior compression results at the same time. Prior work has shown that a denoising filter in the update step can improve the compression ratio. In this paper, we present a new proce…
KU Leuven（鲁汶大学）；Université Claude-Bernard Lyon 1（里昂第一大学）；icometrix；KU Leuven Kulak（鲁汶大学库拉克校区）
Convolutional Neural Networks (CNNs) with U-shaped architectures have dominated medical image segmentation, which is crucial for various clinical purposes. However, the inherent locality of convolution makes CNNs fail to fully exploit global context, essential for better recognition of some structures, e.g., brain lesions. Transformers have recently proven promising performance on vision tasks, including semantic segmentation, mainly due to their capability of modeling long-range dependencies. Nevertheless, the quadratic complexity of attention makes existing Transformer-based models use self-…
Lossless compression of dynamic 2D+t and 3D+t medical data is challenging regarding the huge amount of data, the characteristics of the inherent noise, and the high bit depth. Beyond that, a scalable representation is often required in telemedicine applications. Motion Compensated Temporal Filtering works well for lossless compression of medical volume data and additionally provides temporal, spatial, and quality scalability features. To achieve a high quality lowpass subband, which shall be used as a downscaled representative of the original data, graph-based motion compensation was recently …
