In medical image processing, accurate diagnosis is of paramount importance. Leveraging machine learning techniques, particularly top-rank learning, shows significant promise by focusing on the most crucial instances. However, challenges arise from noisy labels and class-ambiguous instances, which can severely hinder the top-rank objective, as they may be erroneously placed among the top-ranked instances. To address these, we propose a novel approach that enhances toprank learning by integrating a rejection module. Cooptimized with the top-rank loss, this module identifies and mitigates the impact of outliers that hinder training effectiveness. The rejection module functions as an additional branch, assessing instances based on a rejection function that measures their deviation from the norm. Through experimental validation on a medical dataset, our methodology demonstrates its efficacy in detecting and mitigating outliers, improving the reliability and accuracy of medical image diagnoses.
the University of Hong Kong（香港大学）；China Agricultural University（中国农业大学）；Deepwise Healthcare（深睿医疗）
During clinical practice, radiologists often use attributes, e.g. morphological and appearance characteristics of a lesion, to aid disease diagnosis. Effectively modeling attributes as well as all relationships involving attributes could boost the generalization ability and verifiability of medical image diagnosis algorithms. In this paper, we introduce a hybrid neuro-probabilistic reasoning algorithm for verifiable attribute-based medical image diagnosis. There are two parallel branches in our hybrid algorithm, a Bayesian network branch performing probabilistic causal relationship reasoning a…
Helmholtz-Zentrum Dresden-Rossendorf（亥姆霍兹德累斯顿罗森多夫研究中心）；University Hospital Carl Gustav Carus（卡尔·古斯塔夫·卡鲁斯大学医院）
The medical image fusion combines two or more modalities into a single view while medical image translation synthesizes new images and assists in data augmentation. Together, these methods help in faster diagnosis of high grade malignant gliomas. However, they might be untrustworthy due to which neurosurgeons demand a robust visualisation tool to verify the reliability of the fusion and translation results before they make pre-operative surgical decisions. In this paper, we propose a novel approach to compute a confidence heat map between the source-target image pair by estimating the informat…
Imperial College London（帝国理工学院）；Technische Universität München（慕尼黑工业大学）；FAU Erlangen–Nürnberg（埃尔朗根-纽伦堡大学）；National Institutes of Health（美国国立卫生研究院）
Chest radiographs are one of the most common diagnostic modalities in clinical routine. It can be done cheaply, requires minimal equipment, and the image can be diagnosed by every radiologists. However, the number of chest radiographs obtained on a daily basis can easily overwhelm the available clinical capacities. We propose RATCHET: RAdiological Text Captioning for Human Examined Thoraces. RATCHET is a CNN-RNN-based medical transformer that is trained end-to-end. It is capable of extracting image features from chest radiographs, and generates medically accurate text reports that fit seamless…
SMFusion: Semantic-Preserving Fusion of Multimodal Medical Images for Enhanced Clinical Diagnosis
College of Information and Intelligence, Hunan Agricultural University（湖南农业大学信息与智能学院）；College of Artificial Intelligence, Nankai University（南开大学人工智能学院）
The inability to interpret the model prediction in semantically and visually meaningful ways is a well-known shortcoming of most existing computer-aided diagnosis methods. In this paper, we propose MDNet to establish a direct multimodal mapping between medical images and diagnostic reports that can read images, generate diagnostic reports, retrieve images by symptom descriptions, and visualize attention, to provide justifications of the network diagnosis process. MDNet includes an image model and a language model. The image model is proposed to enhance multi-scale feature ensembles and utiliza…
Ghent University Global Campus（根特大学全球校区）；Ghent University（根特大学）
Glaucoma is a major eye disease, leading to vision loss in the absence of proper medical treatment. Current diagnosis of glaucoma is performed by ophthalmologists who are often analyzing several types of medical images generated by different types of medical equipment. Capturing and analyzing these medical images is labor-intensive and expensive. In this paper, we present a novel computational approach towards glaucoma diagnosis and localization, only making use of eye fundus images that are analyzed by state-of-the-art deep learning techniques. Specifically, our approach leverages Convolution…
Federated Distillation for Medical Image Classification: Towards Trustworthy Computer-Aided Diagnosis
School of Information and Communication Engineering, Hainan University（海南大学信息与通信工程学院）；College of Mathematical Sciences, BoHai University（渤海大学数学科学学院）；Australian AI Institute, University of Technology Sydney（悉尼科技大学澳大利亚人工智能研究所）；School of Electronic Science and Technology, Hainan University（海南大学电子科学与技术学院）
Isfahan University of Technology（伊斯法罕理工大学）；University of Michigan（密歇根大学）
Automating classification and segmentation process of abnormal regions in different body organs has a crucial role in most of medical imaging applications such as funduscopy, endoscopy, and dermoscopy. Detecting multiple abnormalities in each type of images is necessary for better and more accurate diagnosis procedure and medical decisions. In recent years portable medical imaging devices such as capsule endoscopy and digital dermatoscope have been introduced and made the diagnosis procedure easier and more efficient. However, these portable devices have constrained power resources and limited…
Graduate School at Shenzhen, Tsinghua University（清华大学深圳研究生院）；Computer Science and Engineering, University of South Carolina（南卡罗来纳大学计算机科学与工程系）
The implementation of medical AI has always been a problem. The effect of traditional perceptual AI algorithm in medical image processing needs to be improved. Here we propose a method of knowledge AI, which is a combination of perceptual AI and clinical knowledge and experience. Based on this method, the geometric information mining of medical images can represent the experience and information and evaluate the quality of medical images.
Key Patches Are All You Need: A Multiple Instance Learning Framework For Robust Medical Diagnosis
Institute for Systems and Robotics（系统与机器人研究所）；Instituto Superior Técnico（里斯本高等理工学院）；Institute of Computing（计算研究所）；University of Campinas（坎皮纳斯大学）；Lisbon ELLIS Unit（里斯本ELLIS单元）
Mining Gaze for Contrastive Learning toward Computer-Assisted Diagnosis
School of Biomedical Engineering, Shanghai Jiao Tong University（上海交通大学生物医学工程学院）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能医疗科技有限公司）；Shanghai Clinical Research and Trial Center（上海临床研究与试验中心）
A Clinical-oriented Multi-level Contrastive Learning Method for Disease Diagnosis in Low-quality Medical Images
Northeastern University（东北大学）；Key Laboratory of Intelligent Computing in Medical Image of Ministry of Education（医学图像智能计算教育部重点实验室）；National Frontiers Science Center for Industrial Intelligence and Systems Optimization（工业智能与系统优化全国前沿科学中心）
Deep neural networks (DNN) have achieved unprecedented performance in computer-vision tasks almost ubiquitously in business, technology, and science. While substantial efforts are made to engineer highly accurate architectures and provide usable model explanations, most state-of-the-art approaches are first designed for natural vision and then translated to the medical domain. This dissertation seeks to address this gap by proposing novel architectures that integrate the domain-specific constraints of medical imaging into the DNN model and explanation design.
NeRD: Neuro-Symbolic Rule Distillation for Efficient Ontology-Grounded Chain-of-Thought in Medical Image Diagnosis
Department of Data Science & AI, Faculty of Information Technology, Monash University（莫纳什大学信息技术学院数据科学与人工智能系）；AIM for Health Lab, Faculty of Information Technology, Monash University（莫纳什大学信息技术学院AIM健康实验室）；Faculty of Engineering, Monash University（莫纳什大学工程学院）；Faculty of Medicine, The Chinese University of Hong Kong（香港中文大学医学院）；School of Computing Technologies, RMIT University（皇家墨尔本理工大学计算技术学院）
Large language models (LLMs) have recently demonstrated their potential in clinical applications, providing valuable medical knowledge and advice. For example, a large dialog LLM like ChatGPT has successfully passed part of the US medical licensing exam. However, LLMs currently have difficulty processing images, making it challenging to interpret information from medical images, which are rich in information that supports clinical decisions. On the other hand, computer-aided diagnosis (CAD) networks for medical images have seen significant success in the medical field by using advanced deep-le…
University of Central Florida（中佛罗里达大学）；Northwestern University（西北大学）；Mayo Clinic（梅奥诊所）；Sheikh Shakhbout Medical City（谢赫·沙赫布特医疗城）
Current data augmentation techniques and transformations are well suited for improving the size and quality of natural image datasets but are not yet optimized for medical imaging. We hypothesize that sub-optimal data augmentations can easily distort or occlude medical images, leading to false positives or negatives during patient diagnosis, prediction, or therapy/surgery evaluation. In our experimental results, we found that utilizing commonly used intensity-based data augmentation distorts the MRI scans and leads to texture information loss, thus negatively affecting the overall performance …
MeLo: Low-rank Adaptation is Better than Fine-tuning for Medical Image Diagnosis
ShanghaiTech University（上海科技大学）；Shanghai Jiao Tong University（上海交通大学）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能医疗科技有限公司）；Shanghai Clinical Research and Trial Center（上海临床研究与试验中心）
Systematic Integration of Attention Modules into CNNs for Accurate and Generalizable Medical Image Diagnosis
Department of Computer Science and Artificial Intelligence（计算机科学与人工智能系）；Division of Electronics and Electrical Engineering（电子与电气工程系）；Dongguk University（东国大学）
Xidian University（西安电子科技大学）；Robarts Research Institute（罗巴茨研究所）；Western University（韦仕敦大学）
Recently, diagnosis, therapy and monitoring of human diseases involve a variety of imaging modalities, such as magnetic resonance imaging(MRI), computed tomography(CT), Ultrasound(US) and Positron-emission tomography(PET) as well as a variety of modern optical techniques. Over the past two decade, it has been recognized that advanced image processing techniques provide valuable information to physicians for diagnosis, image guided therapy and surgery, and monitoring of the treated organ to the therapy. Many researchers and companies have invested significant efforts in the developments of adva…
Learning 3D Medical Image Models From Brain Functional Connectivity Network Supervision For Mental Disorder Diagnosis
University of Science and Technology of China（中国科学技术大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）
South China University of Technology（华南理工大学）；Pazhou Lab（琶洲实验室）；Tencent AI Lab（腾讯人工智能实验室）
Deep learning based medical image diagnosis has shown great potential in clinical medicine. However, it often suffers two major difficulties in real-world applications: 1) only limited labels are available for model training, due to expensive annotation costs over medical images; 2) labeled images may contain considerable label noise (e.g., mislabeling labels) due to diagnostic difficulties of diseases. To address these, we seek to exploit rich labeled data from relevant domains to help the learning in the target task via {Unsupervised Domain Adaptation} (UDA). Unlike most UDA methods that rel…
Shahid Beheshti University（沙希德·贝赫什提大学）；Shahid Beheshti University of Medical Sciences（沙希德·贝赫什提医科大学）；University of Leeds（利兹大学）；Royal Adelaide Hospital（阿德莱德皇家医院）；The University of Adelaide（阿德莱德大学）；Tehran University of Medical Sciences（德黑兰医科大学）
Coronavirus disease (COVID-19) is an infectious disease caused by a newly discovered coronavirus. The disease presents with symptoms such as shortness of breath, fever, dry cough, and chronic fatigue, amongst others. Sometimes the symptoms of the disease increase so much they lead to the death of the patients. The disease may be asymptomatic in some patients in the early stages, which can lead to increased transmission of the disease to others. Many studies have tried to use medical imaging for early diagnosis of COVID-19. This study attempts to review papers on automatic methods for medical i…
A ChatGPT Aided Explainable Framework for Zero-Shot Medical Image Diagnosis
Zhejiang University-University of Illinois at Urbana-Champaign Institute（浙江大学伊利诺伊大学厄巴纳香槟校区联合学院）；Zhejiang University（浙江大学）；National University of Singapore（新加坡国立大学）；Angelalign Inc.（时代天使公司）
Beijing Institute of Technology（北京理工大学）；Xuanwu Hospital, Capital Medical University（首都医科大学宣武医院）；National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College（中国医学科学院北京协和医学院肿瘤医院（国家癌症中心/国家肿瘤临床医学研究中心））
Current Computer-Aided Diagnosis (CAD) methods mainly depend on medical images. The clinical information, which usually needs to be considered in practical clinical diagnosis, has not been fully employed in CAD. In this paper, we propose a novel deep learning-based method for fusing Magnetic Resonance Imaging (MRI)/Computed Tomography (CT) images and clinical information for diagnostic tasks. Two paths of neural layers are performed to extract image features and clinical features, respectively, and at the same time clinical features are employed as the attention to guide the extraction of imag…
Concept Complement Bottleneck Model for Interpretable Medical Image Diagnosis
Hong Kong University of Science and Technology（香港科技大学）；Department of Computer Science and Engineering（计算机科学与工程系）；Department of Chemical and Biological Engineering（化学与生物工程系）；Division of Life Science（生命科学部）
Random Token Fusion for Multi-View Medical Diagnosis
KTH Royal Institute of Technology（皇家理工学院）；Science for Life Laboratory（生命科学实验室）；Karolinska Institutet（卡罗林斯卡学院）；Karolinska University Hospital（卡罗林斯卡大学医院）
Optimization of Convolutional Neural Network Hyperparameter for Medical Image Diagnosis using Metaheuristic Algorithms: A short Recent Review (2019-2022)
Universiti Sains Malaysia（马来西亚理科大学）；King Fahd University of Petroleum and Minerals（法赫德国王石油与矿业大学）；University of Information Technology and Communications（信息技术与通信大学）
Microsoft Research（微软研究院）；TuSimple（图森未来）；Baidu Research（百度研究院）；Tencent Medical AI Labs（腾讯医疗AI实验室）；PingAn Tech（平安科技）
Medical imaging is an essential tool in many areas of medical applications, used for both diagnosis and treatment. However, reading medical images and making diagnosis or treatment recommendations require specially trained medical specialists. The current practice of reading medical images is labor-intensive, time-consuming, costly, and error-prone. It would be more desirable to have a computer-aided system that can automatically make diagnosis and treatment recommendations. Recent advances in deep learning enable us to rethink the ways of clinician diagnosis based on medical images. In this t…
COVID-19 Computer-aided Diagnosis through AI-assisted CT Imaging Analysis: Deploying a Medical AI System
University of Ioannina（约阿尼纳大学）；AIandMe SMPC（AIandMe SMPC公司）；National Technical University of Athens（雅典国立技术大学）；Queen Mary University of London（伦敦大学玛丽皇后学院）
Huazhong University of Science and Technology（华中科技大学）；Guizhou Normal University（贵州师范大学）；Hubei University（湖北大学）
Mapping X-ray images, radiology reports, and other medical data as binary codes in the common space, which can assist clinicians to retrieve pathology-related data from heterogeneous modalities (i.e., hashing-based cross-modal medical data retrieval), provides a new view to promot computeraided diagnosis. Nevertheless, there remains a barrier to boost medical retrieval accuracy: how to reveal the ambiguous semantics of medical data without the distraction of superfluous information. To circumvent this drawback, we propose Deep Supervised Information Bottleneck Hashing (DSIBH), which effectivel…
National and Kapodistrian University of Athens（雅典国立卡波迪斯特里亚大学）
Medical Informatics and the application of modern signal processing in the assistance of the diagnostic process in medical imaging is one of the more recent and active research areas today. This thesis addresses a variety of issues related to the general problem of medical image analysis, specifically in mammography, and presents a series of algorithms and design approaches for all the intermediate levels of a modern system for computer-aided diagnosis (CAD). The diagnostic problem is analyzed with a systematic approach, first defining the imaging characteristics and features that are relevant…
Fairness in Multi-modal Medical Diagnosis with Demonstration Selection
Arizona State University（亚利桑那州立大学）；University of Rochester（罗切斯特大学）；University of Virginia（弗吉尼亚大学）；UCL（伦敦大学学院）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；University of Central Florida（佛罗里达中央大学）
Tencent Jarvis Lab（腾讯Jarvis实验室）；Institute of Computing Technology Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；The First Affiliated Hospital of Anhui Medical University（安徽医科大学第一附属医院）；University of Science and Technology of China（中国科学技术大学）
Fast screening and diagnosis are critical in COVID-19 patient treatment. In addition to the gold standard RT-PCR, radiological imaging like X-ray and CT also works as an important means in patient screening and follow-up. However, due to the excessive number of patients, writing reports becomes a heavy burden for radiologists. To reduce the workload of radiologists, we propose DeltaNet to generate medical reports automatically. Different from typical image captioning approaches that generate reports with an encoder and a decoder, DeltaNet applies a conditional generation process. In particular…
School of Electronics Engineering and Computer Science Peking University（北京大学电子工程与计算机科学学院）；Institute of Sports Medicine Peking University Third Hospital（北京大学第三医院运动医学研究所）
The process of determining which disease or condition explains a person's symptoms and signs can be very complicated and may be inaccurate in some cases. The general belief is that diagnosing diseases relies on doctors' keen intuition, rich experience and professional equipment. In this work, we employ ideas from recent advances in plantar pressure research and from the powerful capacity of the convolutional neural network for learning representations. Here, we propose a model using convolutional neural network based on plantar pressure for medical diagnosis. Our model learns a network that ma…
George Washington University（乔治华盛顿大学）；University of Waterloo（滑铁卢大学）；University of Toronto（多伦多大学）
This paper constructs a novel intelligent medical diagnosis system, which can realize automatic communication and breast cancer pathological image recognition. This system contains two main parts, including a pre-training chatbot called M-Chatbot and an improved neural network model of EfficientNetV2-S named EfficientNetV2-SA, in which the activation function in top layers is replaced by ACON-C. Using information retrieval mechanism, M-Chatbot instructs patients to send breast pathological image to EfficientNetV2-SA network, and then the classifier trained by transfer learning will return the …
See-in-Pairs: Reference Image-Guided Comparative Vision-Language Models for Medical Diagnosis
Electrical and Computer Engineering Department, The University of British Columbia（不列颠哥伦比亚大学电气与计算机工程系）；Vector Institute（向量研究所）；ETH Zurich（苏黎世联邦理工学院）；Renmin University of China（中国人民大学）；National University of Singapore（新加坡国立大学）
