FDIF: Formula-Driven supervised Learning with Implicit Functions for 3D Medical Image Segmentation
Waseda University（早稻田大学）；The University of Osaka（大阪大学）；National Institute of Advanced Industrial Science and Technology（国立研究开发法人产业技术综合研究所）；University of Oxford（牛津大学）
Deep learning-based 3D medical image segmentation methods relies on large-scale labeled datasets, yet acquiring such data is difficult due to privacy constraints and the high cost of expert annotation. Formula-Driven Supervised Learning (FDSL) offers an appealing alternative by generating training data and labels directly from mathematical formulas. However, existing voxel-based approaches are limited in geometric expressiveness and cannot synthesize realistic textures. We introduce Formula-Driven supervised learning with Implicit Functions (FDIF), a framework that enables scalable pre-training without using any real data and medical expert annotations. FDIF introduces an implicit-function representation based on signed distance functions (SDFs), enabling compact modeling of complex geometries while exploiting the surface representation of SDFs to support controllable synthesis of both geometric and intensity textures. Across three medical image segmentation benchmarks (AMOS, ACDC, and KiTS) and three architectures (SwinUNETR, nnUNet ResEnc-L, and nnUNet Primus-M), FDIF consistently improves over a formula-driven method, and achieves performance comparable to self-supervised approaches pre-trained on large-scale real datasets. We further show that FDIF pre-training also benefits 3D classification tasks, highlighting implicit-function-based formula supervision as a promising paradigm for data-free representation learning. Code is available at https://github.com/yamanoko/FDIF.
I-MedSAM: Implicit Medical Image Segmentation with Segment Anything
Peking University（北京大学）；University of Chinese Academy of Sciences（中国科学院大学）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；Beihang University（北京航空航天大学）；Beijing University of Posts and Telecommunications（北京邮电大学）
SimulaMet（西穆拉梅特）；Oslo Metropolitan University（奥斯陆城市大学）；UiT The Arctic University of Norway（挪威北极大学）；University of Oxford（牛津大学）；NIHR Oxford Biomedical Research Centre（英国国家健康研究所牛津生物医学研究中心）
Widely used traditional supervised deep learning methods require a large number of training samples but often fail to generalize on unseen datasets. Therefore, a more general application of any trained model is quite limited for medical imaging for clinical practice. Using separately trained models for each unique lesion category or a unique patient population will require sufficiently large curated datasets, which is not practical to use in a real-world clinical set-up. Few-shot learning approaches can not only minimize the need for an enormous number of reliable ground truth labels that are …
Transformers have shown great success in medical image segmentation. However, transformers may exhibit a limited generalization ability due to the underlying single-scale self-attention (SA) mechanism. In this paper, we address this issue by introducing a Multi-scale hiERarchical vIsion Transformer (MERIT) backbone network, which improves the generalizability of the model by computing SA at multiple scales. We also incorporate an attention-based decoder, namely Cascaded Attention Decoding (CASCADE), for further refinement of multi-stage features generated by MERIT. Finally, we introduce an eff…
Department of Biomedical Engineering, University of Basel（巴塞尔大学生物医学工程系）
Diffusion models have shown impressive performance for generative modelling of images. In this paper, we present a novel semantic segmentation method based on diffusion models. By modifying the training and sampling scheme, we show that diffusion models can perform lesion segmentation of medical images. To generate an image specific segmentation, we train the model on the ground truth segmentation, and use the image as a prior during training and in every step during the sampling process. With the given stochastic sampling process, we can generate a distribution of segmentation masks. This pro…
FlowSDF: Flow Matching for Medical Image Segmentation Using Distance Transforms
Institute of Computer Graphics and Vision（计算机图形与视觉研究所）；Graz University of Technology（格拉茨工业大学）；Photogrammetry and Remote Sensing（摄影测量与遥感研究所）；ETH Zurich（苏黎世联邦理工学院）
Recently, several Bayesian deep learning methods have been proposed for semi-supervised medical image segmentation. Although they have achieved promising results on medical benchmarks, some problems are still existing. Firstly, their overall architectures belong to the discriminative models, and hence, in the early stage of training, they only use labeled data for training, which might make them overfit to the labeled data. Secondly, in fact, they are only partially based on Bayesian deep learning, as their overall architectures are not designed under the Bayesian framework. However, unifying …
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Beijing University of Posts and Telecommunications（北京邮电大学）；Shandong University of Science and Technology（山东科技大学）
Efficient medical image segmentation aims to provide accurate pixel-wise predictions for medical images with a lightweight implementation framework. However, lightweight frameworks generally fail to achieve superior performance and suffer from poor generalizable ability on cross-domain tasks. In this paper, we explore the generalizable knowledge distillation for the efficient segmentation of cross-domain medical images. Considering the domain gaps between different medical datasets, we propose the Model-Specific Alignment Networks (MSAN) to obtain the domain-invariant representations. Meanwhil…
Cross-Modal Conditioned Reconstruction for Language-guided Medical Image Segmentation
College of Information and Electrical Engineering, China Agricultural University（中国农业大学信息与电气工程学院）；Beijing Engineering and Technology Research Center for Internet of Things in Agriculture, China Agricultural University（中国农业大学北京农业物联网工程技术研究中心）；School of Electronic and Computer Engineering, Peking University（北京大学电子与计算机工程学院）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；School of Engineering, The Hong Kong University of Science and Technology（香港科技大学工程学院）；Department of Electrical Engineering, Yale University（耶鲁大学电气工程系）
Learning Semantic Directions for Feature Augmentation in Domain-Generalized Medical Segmentation
Chengdu Institute of Compute Application, Chinese Academy of Sciences, Chengdu, China（成都计算应用研究所，中国科学院，中国）；China Zhenhua Research Institute Co., Ltd., Guiyang, China（中国振华研究院有限公司，贵阳，中国）；School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, China（计算机科学与技术学院，中国科学院大学，北京，中国）
Technical University of Munich（慕尼黑工业大学）；University hospital of Technical University of Munich（慕尼黑工业大学附属医院）；Southern University of Science and Technology（南方科技大学）
In medical imaging, the heterogeneity of multi-centre data impedes the applicability of deep learning-based methods and results in significant performance degradation when applying models in an unseen data domain, e.g. a new centreor a new scanner. In this paper, we propose an unsupervised domain adaptation framework for boosting image segmentation performance across multiple domains without using any manual annotations from the new target domains, but by re-calibrating the networks on few images from the target domain. To achieve this, we enforce architectures to be adaptive to new data by re…
Implicit U-KAN2.0: Dynamic, Efficient and Interpretable Medical Image Segmentation
Department of Applied Mathematics and Theoretical Physics, University of Cambridge, UK（应用数学与理论物理系，剑桥大学）；Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign, USA（计算与数据科学学院，伊利诺伊大学厄巴纳-香槟分校）；Zurich University of Applied Sciences, Switzerland（苏黎世应用科学大学）；Lucerne Cantonal Hospital, Switzerland（卢塞恩州立医院）；Lucerne University of Applied Sciences and Arts, Switzerland（卢塞恩应用科学与艺术大学）；Yau Mathematical Sciences Center, Tsinghua University, China（尤太数学科学中心，清华大学）
Dual Teacher-Student Learning for Semi-supervised Medical Image Segmentation
Center for Applied Mathematics, Tianjin University（应用数学中心，天津大学）；Department of Radiology, Tianjin Hospital of Tianjin University（放射科，天津大学医院）；Department of Hip Trauma, Tianjin Hospital of Tianjin University（骨科创伤科，天津大学医院）
Dual Interaction Network with Cross-Image Attention for Medical Image Segmentation
Department of Information Convergence Engineering, Pusan National University（convergence工程系，釜山国立大学）；School of Computer Engineering, Kyungnam University（计算机工程学院，庆尚大学）；School of Computer Science and Engineering, Pusan National University（计算机科学与工程学院，釜山国立大学）
Towards Voxel Spacing Consistency for Medical Image Segmentation
Institute of Medical Robotics, Shanghai Jiao Tong University（上海交通大学医疗机器人研究院）；Department of Ultrasound, Shanghai Chest Hospital（上海市胸科医院超声科）；CAMPAR, Technical University of Munich（慕尼黑工业大学CAMPAR）
Location information is proven to benefit the deep learning models on capturing the manifold structure of target objects, and accordingly boosts the accuracy of medical image segmentation. However, most existing methods encode the location information in an implicit way, e.g. the distance transform maps, which describe the relative distance from each pixel to the contour boundary, for the network to learn. These implicit approaches do not fully exploit the position information (i.e. absolute location) of targets. In this paper, we propose a novel loss function, namely residual moment (RM) loss…
Data-Efficient Meningioma Segmentation via Implicit Spatiotemporal Mixing and Sim2Real Semantic Injection
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究所，中国科学院）；The First Affiliated Hospital of Shenzhen University（深圳大学第一附属医院）；Health Science Center（健康科学中心）；Shenzhen Second People’s Hospital（深圳第二人民医院）；Shenzhen University（深圳大学）；Department of Radiology（放射科）
Beihang University（北京航空航天大学）；Hefei Innovation Research Institute, Beihang University（北京航空航天大学合肥创新研究院）；Beijing Advanced Innovation Centre for Biomedical Engineering（北京生物医学工程高精尖创新中心）；Beijing Advanced Innovation Centre for Big Data-Based Precision Medicine（北京大数据精准医疗高精尖创新中心）
The success of deep learning methods in medical image segmentation tasks usually requires a large amount of labeled data. However, obtaining reliable annotations is expensive and time-consuming. Semi-supervised learning has attracted much attention in medical image segmentation by taking the advantage of unlabeled data which is much easier to acquire. In this paper, we propose a novel dual-task mutual learning framework for semi-supervised medical image segmentation. Our framework can be formulated as an integration of two individual segmentation networks based on two tasks: learning region-ba…
Cornell University（康奈尔大学）；Weill Cornell Medical College（威尔康奈尔医学院）；University of Pennsylvania（宾夕法尼亚大学）
We introduce Neural Representation of Distribution (NeRD) technique, a module for convolutional neural networks (CNNs) that can estimate the feature distribution by optimizing an underlying function mapping image coordinates to the feature distribution. Using NeRD, we propose an end-to-end deep learning model for medical image segmentation that can compensate the negative impact of feature distribution shifting issue caused by commonly used network operations such as padding and pooling. An implicit function is used to represent the parameter space of the feature distribution by querying the i…
University of Queensland（昆士兰大学）；Australian e-Health Research Centre, CSIRO（澳大利亚联邦科学与工业研究组织澳大利亚电子健康研究中心）
Data scarcity is common in deep learning models for medical image segmentation. Previous works proposed multi-dataset learning, either simultaneously or via transfer learning to expand training sets. However, medical image datasets have diverse-sized images and features, and developing a model simultaneously for multiple datasets is challenging. This work proposes Fabric Image Representation Encoding Network (FIRENet), a universal architecture for simultaneous multi-dataset segmentation and transfer learning involving arbitrary numbers of dataset(s). To handle different-sized image and feature…
Interpretable Probabilistic Medical Image Segmentation via Gaussian Process with Explicit Modelling of Annotation Bias and Variability
University College London（伦敦大学学院）；Queen Mary University of London（伦敦玛丽女王大学）；University of Oxford（牛津大学）；University College London Hospitals NHS Trust（伦敦大学学院医院NHS信托基金会）；Imperial College Healthcare（帝国理工学院医疗保健）
Convolutional Neural Networks (CNNs) work very well for supervised learning problems when the training dataset is representative of the variations expected to be encountered at test time. In medical image segmentation, this premise is violated when there is a mismatch between training and test images in terms of their acquisition details, such as the scanner model or the protocol. Remarkable performance degradation of CNNs in this scenario is well documented in the literature. To address this problem, we design the segmentation CNN as a concatenation of two sub-networks: a relatively shallow i…
Consistent View Alignment Improves Foundation Models for 3D Medical Image Segmentation
Department of Applied Mathematics, Technical Medical Centre, University of Twente（代尔夫特理工大学应用数学系）；Digital Technology and Innovation, Siemens Healthineers, Erlangen, Germany（西门子医疗创新部，埃尔朗根，德国）
End-to-End Learning of Multi-Organ Implicit Surfaces from 3D Medical Imaging Data
University of Strasbourg, CNRS, INSERM, ICube, UMR7357, Strasbourg, France Institute of Image-Guided Surgery, IHU Strasbourg, Strasbourg, France Department of Diagnostic Radiology, McGill University, Montreal, Canada Augmented Intelligence \& Precision Health Laboratory (AIPHL), McGill University Health Centre Research Institute, Montreal, Canada
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Weill Cornell Medicine（威尔康奈尔医学院）
Deep neural networks (DNNs) have rapidly become a \textit{de facto} choice for medical image understanding tasks. However, DNNs are notoriously fragile to the class imbalance in image classification. We further point out that such imbalance fragility can be amplified when it comes to more sophisticated tasks such as pathology localization, as imbalances in such problems can have highly complex and often implicit forms of presence. For example, different pathology can have different sizes or colors (w.r.t.the background), different underlying demographic distributions, and in general different …
CitAI Research Centre（CitAI研究中心）；City, University of London（伦敦城市大学）；BioMedIA（生物医学影像分析组）；Imperial College（帝国理工学院）
U-Net has been the go-to architecture for medical image segmentation tasks, however computational challenges arise when extending the U-Net architecture to 3D images. We propose the Implicit U-Net architecture that adapts the efficient Implicit Representation paradigm to supervised image segmentation tasks. By combining a convolutional feature extractor with an implicit localization network, our implicit U-Net has 40% less parameters than the equivalent U-Net. Moreover, we propose training and inference procedures to capitalize sparse predictions. When comparing to an equivalent fully convolut…
RibPull: Implicit Occupancy Fields and Medial Axis Extraction for CT Ribcage Scans
Electrical and Computer Engineering Department, University of California, Santa Cruz（加州大学圣克ruz分校电子与计算机工程系）；Inria, Univ. Rennes, CNRS, IRISA, M2S, Rennes（Inria，里尔大学，CNRS，IRISA，M2S，里尔）；LIRIS - CNRS - Université Claude Bernard Lyon 1（LIRIS-CNRS-里尔1大学）；Computer Science Engineering Department, University of California, Santa Cruz（加州大学圣克ruz分校计算机科学工程系）
Beyond Pixels: Medical Image Quality Assessment with Implicit Neural Representations
Computer Engineering Department, Istanbul Technical University（伊斯坦布尔技术大学计算机工程系）；Department of Applied Mathematics, Technical Medical Centre, University of Twente（代尔夫特理工大学应用数学系）
Implicit Neural Representation in Medical Imaging: A Comparative Survey
Iran University of Science and Technology（伊朗科技大学）；South Dakota State University（南达科他州立大学）；RWTH Aachen University（亚琛工业大学）；University of Regensburg（雷根斯堡大学）；Fraunhofer Institute for Digital Medicine MEVIS（弗劳恩霍夫数字医学MEVIS研究所）
Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）
In this paper, we consider the problem of disease diagnosis. Unlike the conventional learning paradigm that treats labels independently, we propose a knowledge-enhanced framework, that enables training visual representation with the guidance of medical domain knowledge. In particular, we make the following contributions: First, to explicitly incorporate experts' knowledge, we propose to learn a neural representation for the medical knowledge graph via contrastive learning, implicitly establishing relations between different medical concepts. Second, while training the visual encoder, we keep t…
SINTEF（辛泰夫研究院）；Oslo University Hospital（奥斯陆大学医院）；The Intervention Centre（介入中心）
We propose a novel approach to image segmentation based on combining implicit spline representations with deep convolutional neural networks. This is done by predicting the control points of a bivariate spline function whose zero-set represents the segmentation boundary. We adapt several existing neural network architectures and design novel loss functions that are tailored towards providing implicit spline curve approximations. The method is evaluated on a congenital heart disease computed tomography medical imaging dataset. Experiments are carried out by measuring performance in various stan…
Cornell University（康奈尔大学）；Cornell Tech（康奈尔科技学院）；Massachusetts Institute of Technology（麻省理工学院）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
The convolutional neural network (CNN) is one of the most commonly used architectures for computer vision tasks. The key building block of a CNN is the convolutional kernel that aggregates information from the pixel neighborhood and shares weights across all pixels. A standard CNN's capacity, and thus its performance, is directly related to the number of learnable kernel weights, which is determined by the number of channels and the kernel size (support). In this paper, we present the \textit{hyper-convolution}, a novel building block that implicitly encodes the convolutional kernel using spat…
Huawei Cloud Computing Technologies Company Limited（华为云计算技术有限公司）；Tianjin University（天津大学）
Machine learning based medical image analysis highly depends on datasets. Biases in the dataset can be learned by the model and degrade the generalizability of the applications. There are studies on debiased models. However, scientists and practitioners are difficult to identify implicit biases in the datasets, which causes lack of reliable unbias test datasets to valid models. To tackle this issue, we first define the data intrinsic bias attribute, and then propose a novel bias identification framework for medical image datasets. The framework contains two major components, KlotskiNet and Bia…
The University of Texas Arlington（德克萨斯大学阿灵顿分校）；RIKEN（理化学研究所）；University of Tokyo（东京大学）；National Cancer Center Research Institute（国立癌症中心研究所）；National Library of Medicine - National Institutes of Health（美国国立卫生研究院国家医学图书馆）；Temple University（天普大学）
Medical visual question answering (VQA) aims to answer clinically relevant questions regarding input medical images. This technique has the potential to improve the efficiency of medical professionals while relieving the burden on the public health system, particularly in resource-poor countries. Existing medical VQA methods tend to encode medical images and learn the correspondence between visual features and questions without exploiting the spatial, semantic, or medical knowledge behind them. This is partially because of the small size of the current medical VQA dataset, which often includes…
City, University of London（伦敦大学城市学院）；Imperial College（帝国理工学院）
We propose a novel unsupervised out-of-distribution detection method for medical images based on implicit fields image representations. In our approach, an auto-decoder feed-forward neural network learns the distribution of healthy images in the form of a mapping between spatial coordinates and probabilities over a proxy for tissue types. At inference time, the learnt distribution is used to retrieve, from a given test image, a restoration, i.e. an image maximally consistent with the input one but belonging to the healthy distribution. Anomalies are localized using the voxel-wise probability p…
In the past ten years, with the help of deep learning, especially the rapid development of deep neural networks, medical image analysis has made remarkable progress. However, how to effectively use the relational information between various tissues or organs in medical images is still a very challenging problem, and it has not been fully studied. In this thesis, we propose two novel solutions to this problem based on deep relational learning. First, we propose a context-aware fully convolutional network that effectively models implicit relation information between features to perform medical i…
