Segmentation of organs of interest in 3D medical images is necessary for accurate diagnosis and longitudinal studies. Though recent advances using deep learning have shown success for many segmentation tasks, large datasets are required for high performance and the annotation process is both time consuming and labor intensive. In this paper, we propose a 3D few shot segmentation framework for accurate organ segmentation using limited training samples of the target organ annotation. To achieve this, a U-Net like network is designed to predict segmentation by learning the relationship between 2D slices of support data and a query image, including a bidirectional gated recurrent unit (GRU) that learns consistency of encoded features between adjacent slices. Also, we introduce a transfer learning method to adapt the characteristics of the target image and organ by updating the model before testing with arbitrary support and query data sampled from the support data. We evaluate our proposed model using three 3D CT datasets with annotations of different organs. Our model yielded significantly improved performance over state-of-the-art few shot segmentation models and was comparable to a fully supervised model trained with more target training data.
BiCLIP: Bidirectional and Consistent Language-Image Processing for Robust Medical Image Segmentation
Department of Computer Engineering, University of Kurdistan, Iran(伊朗库尔德大学计算机工程系);Artificial Intelligence and Innovation Centre, University of Kurdistan, Erbil, Iraq(伊拉克埃尔比尔库尔德大学人工智能与创新中心);College of Information Technology, United Arab Emirates University, UAE(阿联酋大学信息科技学院)
Bidirectional Uncertainty-Aware Region Learning for Semi-Supervised Medical Image Segmentation
Key Laboratory of Intelligent Computing \& Signal Processing (ICSP), Ministry of Education, School of Artificial Intelligence, Anhui Uniersity, Hefei, 230601, China;Anhui Provincial Key Laboratory of Multimodal Cognitive Computation, School of Computer Science;Technology, Anhui University, Hefei, 230601, China;Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, 230026, China
Background Matters: A Cross-view Bidirectional Modeling Framework for Semi-supervised Medical Image Segmentation
State Key Laboratory for Novel Software Technology, Nanjing University(新型软件技术国家重点实验室,南京大学);College of Physics and Information Engineering, Fuzhou University(福州大学物理与信息工程学院);National Institute of Healthcare Data Science, Nanjing University(健康数据科学国家研究院,南京大学);Nanjing Drum Tower Hospital(南京鼓楼医院)
BiSegMamba: Efficient Bidirectional Tri-Oriented Mamba for 3D Medical Image Segmentation
School of Computer Science and Engineering, Beihang University(北航计算机科学与工程学院);State Key Laboratory of Virtual Reality Technology and Systems, Beihang University(北航虚拟现实技术与系统国家重点实验室)
Minimally invasive surgery mainly consists of a series of sub-tasks, which can be decomposed into basic gestures or contexts. As a prerequisite of autonomic operation, surgical gesture recognition can assist motion planning and decision-making, and build up context-aware knowledge to improve the surgical robot control quality. In this work, we aim to develop an effective surgical gesture recognition approach with an explainable feature extraction process. A Bidirectional Multi-Layer independently RNN (BML-indRNN) model is proposed in this paper, while spatial feature extraction is implemented …
Nankai University(南开大学);Civil Aviation General Hospital(民航总医院);Chinese PLA General Hospital(中国人民解放军总医院)
Breast lesions segmentation is an important step of computer-aided diagnosis system, and it has attracted much attention. However, accurate segmentation of malignant breast lesions is a challenging task due to the effects of heterogeneous structure and similar intensity distributions. In this paper, a novel bidirectional aware guidance network (BAGNet) is proposed to segment the malignant lesion from breast ultrasound images. Specifically, the bidirectional aware guidance network is used to capture the context between global (low-level) and local (high-level) features from the input coarse sal…
Focal Modulation and Bidirectional Feature Fusion Network for Medical Image Segmentation
Department of Computing, Abasyn University Islamabad Campus (AUIC)(阿巴斯扬大学伊斯兰堡校区计算机系);School of Computer Science and Informatics, De Montfort University(德蒙福特大学计算机科学与信息学学院);Naif Arab University for Security Sciences(纳夫阿拉伯安全科学大学);School of Computer Science and Engineering, University of New South Wales(新南威尔士大学计算机科学与工程学院);Center of Excellence in Precision Medicine and Digital Health, Chulalongkorn University(朱拉隆功大学精准医学与数字健康卓越中心);School of Biomedical Engineering, UNSW Sydney(新南威尔士大学悉尼分校生物医学工程学院)
Bi-CRCL: Bidirectional Conservative-Radical Complementary Learning with Pre-trained Foundation Models for Class-incremental Medical Image Analysis
Department of Biomedical Engineering, The Chinese University of Hong Kong(生物医学工程系,香港中文大学);Department of Radiation Oncology, Columbia University Irving Medical Center and Data Science Institute, Columbia University(放射肿瘤学系,哥伦比亚大学伊万杰琳医学中心及数据科学研究院,哥伦比亚大学);Department of Electrical and Electronic Engineering and School of Biomedical Engineering, The University of Hong Kong(电气电子工程系和生物医学工程学院,香港大学);Department of Computer Science, City University of Hong Kong(计算机科学系,城市大学);Department of Artificial Intelligence, Westlake University(人工智能系,西湖大学)
The Chinese University of Hong Kong(香港中文大学);Imsight Medical Technology Co. Ltd.(视见医疗科技有限公司);The Hong Kong Polytechnic University(香港理工大学)
Unsupervised domain adaptation has increasingly gained interest in medical image computing, aiming to tackle the performance degradation of deep neural networks when being deployed to unseen data with heterogeneous characteristics. In this work, we present a novel unsupervised domain adaptation framework, named as Synergistic Image and Feature Alignment (SIFA), to effectively adapt a segmentation network to an unlabeled target domain. Our proposed SIFA conducts synergistic alignment of domains from both image and feature perspectives. In particular, we simultaneously transform the appearance o…
Small Lesions-aware Bidirectional Multimodal Multiscale Fusion Network for Lung Disease Classification
Xidian University(西安电子科技大学);Hangzhou Dianzi University(杭州电子科技大学);Zhejiang College of Security Technology, School of Artificial Intelligence(浙江安全技术学院人工智能学院);Shenzhen Polytechnic University(深圳职业技术学院);Shenzhen University(深圳大学);Shenzhen Research Institute of Big Data(深圳大数据研究院)
Indian Institute of Technology Madras(印度理工学院马德拉斯分校)
Magnetic Resonance (MR) Imaging and Computed Tomography (CT) are the primary diagnostic imaging modalities quite frequently used for surgical planning and analysis. A general problem with medical imaging is that the acquisition process is quite expensive and time-consuming. Deep learning techniques like generative adversarial networks (GANs) can help us to leverage the possibility of an image to image translation between multiple imaging modalities, which in turn helps in saving time and cost. These techniques will help to conduct surgical planning under CT with the feedback of MRI information…
TiBiX: Leveraging Temporal Information for Bidirectional X-ray and Report Generation
Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Sheikh Shakhbout Medical City(谢赫·沙赫布特医疗城);Big Data Institute, University of Oxford(牛津大学大数据研究所)
Bangladesh University of Engineering and Technology(孟加拉国工程技术大学)
In the advent of a digital health revolution, vast amounts of clinical data are being generated, stored and processed on a daily basis. This has made the storage and retrieval of large volumes of health-care data, especially, high-resolution medical images, particularly challenging. Effective image compression for medical images thus plays a vital role in today's healthcare information system, particularly in teleradiology. In this work, an X-ray image compression method based on a Convolutional Recurrent Neural Networks RNN-Conv is presented. The proposed architecture can provide variable com…
Imperial College London(帝国理工学院);Technische Universität München(慕尼黑工业大学);FAU Erlangen–Nürnberg(埃尔朗根-纽伦堡大学);National Institutes of Health(美国国立卫生研究院)
Chest radiographs are one of the most common diagnostic modalities in clinical routine. It can be done cheaply, requires minimal equipment, and the image can be diagnosed by every radiologists. However, the number of chest radiographs obtained on a daily basis can easily overwhelm the available clinical capacities. We propose RATCHET: RAdiological Text Captioning for Human Examined Thoraces. RATCHET is a CNN-RNN-based medical transformer that is trained end-to-end. It is capable of extracting image features from chest radiographs, and generates medically accurate text reports that fit seamless…
AGA: An adaptive group alignment framework for structured medical cross-modal representation learning
School of Computing and Artificial Intelligence(计算机与人工智能学院);Southwest Jiaotong University(西南交通大学);Department of Gastroenterology(消化内科部);The Third People’s Hospital of Chengdu(成都第三人民医院)
Feinberg School of Medicine(范伯格医学院);Northwestern University(西北大学)
Joint image-text embedding extracted from medical images and associated contextual reports is the bedrock for most biomedical vision-and-language (V+L) tasks, including medical visual question answering, clinical image-text retrieval, clinical report auto-generation. In this study, we adopt four pre-trained V+L models: LXMERT, VisualBERT, UNIER and PixelBERT to learn multimodal representation from MIMIC-CXR radiographs and associated reports. The extrinsic evaluation on OpenI dataset shows that in comparison to the pioneering CNN-RNN model, the joint embedding learned by pre-trained V+L models…
SliceMamba with Neural Architecture Search for Medical Image Segmentation
School of Computer Science, Beijing University of Technology(北京工业大学计算机科学学院);Xiaomi Inc(小米公司);Department of Radiology, Capital Medical University Affiliated Beijing Friendship Hospital(首都医科大学附属北京友谊医院放射科);New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所模式识别新实验室)
This work tackles the problem of generating a medical report for multi-image panels. We apply our solution to the Renal Direct Immunofluorescence (RDIF) assay which requires a pathologist to generate a report based on observations across the eight different WSI in concert with existing clinical features. To this end, we propose a novel attention-based multi-modal generative recurrent neural network (RNN) architecture capable of dynamically sampling image data concurrently across the RDIF panel. The proposed methodology incorporates text from the clinical notes of the requesting physician to re…
Institute for Interdisciplinary Information Sciences(交叉信息研究院);Tsinghua University(清华大学)
Videos have become ubiquitous on the Internet. And video analysis can provide lots of information for detecting and recognizing objects as well as help people understand human actions and interactions with the real world. However, facing data as huge as TB level, effective methods should be applied. Recurrent neural network (RNN) architecture has wildly been used on many sequential learning problems such as Language Model, Time-Series Analysis, etc. In this paper, we propose some variations of RNN such as stacked bidirectional LSTM/GRU network with attention mechanism to categorize large-scale…
East China Normal University(华东师范大学);Tongji University(同济大学)
The encoder-decoder networks are commonly used in medical image segmentation due to their remarkable performance in hierarchical feature fusion. However, the expanding path for feature decoding and spatial recovery does not consider the long-term dependency when fusing feature maps from different layers, and the universal encoder-decoder network does not make full use of the multi-modality information to improve the network robustness especially for segmenting medical MRI. In this paper, we propose a novel feature fusion unit called Recurrent Decoding Cell (RDC) which leverages convolutional R…
University of Waterloo(滑铁卢大学);Sunnybrook Research Institute(桑尼布鲁克研究所)
We present Recurrence with Correlation Network (RWCNet), a medical image registration network with multi-scale features and a cost volume layer. We demonstrate that these architectural features improve medical image registration accuracy in two image registration datasets prepared for the MICCAI 2022 Learn2Reg Workshop Challenge. On the large-displacement National Lung Screening Test (NLST) dataset, RWCNet is able to achieve a total registration error (TRE) of 2.11mm between corresponding keypoints without instance fine-tuning. On the OASIS brain MRI dataset, RWCNet is able to achieve an avera…
NVIDIA Corporation(英伟达公司);Seoul National University(首尔大学)
Synthesizing medical images, such as PET, is a challenging task due to the fact that the intensity range is much wider and denser than those in photographs and digital renderings and are often heavily biased toward zero. Above all, intensity values in PET have absolute significance, and are used to compute parameters that are reproducible across the population. Yet, usually much manual adjustment has to be made in pre-/post- processing when synthesizing PET images, because its intensity ranges can vary a lot, e.g., between -100 to 1000 in floating point values. To overcome these challenges, we…