A Self-Guided Framework for Radiology Report Generation
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University of Chinese Academy of Sciences(中国科学院大学);Shenzhen University General Hospital(深圳大学总医院)
Jun Li, Shibo Li, Ying Hu, Huiren Tao
备注:
11 pages, 3 figures, accepted by Medical Image Computing and Computer Assisted Intervention 2022(MICCAI 2022)
Automatic radiology report generation is essential to computer-aided diagnosis. Through the success of image captioning, medical report generation has been achievable. However, the lack of annotated disease labels is still the bottleneck of this area. In addition, the image-text data bias problem and complex sentences make it more difficult to generate accurate reports. To address these gaps, we pre-sent a self-guided framework (SGF), a suite of unsupervised and supervised deep learning methods to mimic the process of human learning and writing. In detail, our framework obtains the domain knowledge from medical reports with-out extra disease labels and guides itself to extract fined-grain visual features as-sociated with the text. Moreover, SGF successfully improves the accuracy and length of medical report generation by incorporating a similarity comparison mechanism that imitates the process of human self-improvement through compar-ative practice. Extensive experiments demonstrate the utility of our SGF in the majority of cases, showing its superior performance over state-of-the-art meth-ods. Our results highlight the capacity of the proposed framework to distinguish fined-grained visual details between words and verify its advantage in generating medical reports.
Trust It or Not: Confidence-Guided Automatic Radiology Report Generation
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Duke University(杜克大学);The Chinese University of Hong Kong(香港中文大学);Brigham and Women’s Hospital, Harvard Medical School(布莱根妇女医院,哈佛医学院);Lenovo Research(联想研究院);Lenovo Ltd.(联想集团有限公司);University of North Carolina at Charlotte(北卡罗来纳大学夏洛特分校)
MAIRA-Seg: Enhancing Radiology Report Generation with Segmentation-Aware Multimodal Large Language Models
Microsoft Health Futures(微软健康未来);Microsoft Research India(微软研究院印度分院);University of Cambridge(剑桥大学);Cambridge University Hospitals NHS Foundation Trust(剑桥大学医院国民保健信托基金)
Semantically Informed Salient Regions Guided Radiology Report Generation
School of Artificial Intelligence, Beijing University of Posts and Telecommunications(人工智能学院,北京邮电大学);Department of Radiology, Peking University Third Hospital(北京大学第三医院放射科)
CCS: Clinical Consensus Selection for Radiology Report Generation
School of Computing Science, University of Glasgow(格拉斯哥大学计算机科学学院);School of Electrical and Computer Engineering, University of Sydney(悉尼大学电气与计算机工程学院);Language Technology Lab, University of Cambridge(剑桥大学语言技术实验室)
RADAR: Enhancing Radiology Report Generation with Supplementary Knowledge Injection
Department of Computing, The Hong Kong Polytechnic University(香港理工大学计算机系);Research Institute of Trustworthy Autonomous Systems and Department of Computer Science and Engineering, Southern University of Science and Technology(南方科技大学可信自主系统研究院和计算机科学与工程系);School of Computer Science, University of Nottingham Ningbo China(宁波大学计算机学院)
Department of Electronic Engineering, Tsinghua University(清华大学电子工程系);College of AI, Tsinghua University(清华大学人工智能学院);Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心);Beijing Electronic Digital & Intelligence(北京电子数字与智能)
Location-based Radiology Report-Guided Semi-supervised Learning for Prostate Cancer Detection
National Cancer Institute(美国国家癌症研究所);National Institutes of Health(美国国立卫生研究院);Clinical Center, National Institutes of Health(美国国立卫生研究院临床中心);Center for Cancer Research, National Cancer Institute(美国国家癌症研究所癌症研究中心)
ICON: Improving Inter-Report Consistency in Radiology Report Generation via Lesion-aware Mixup Augmentation
The Hong Kong Polytechnic University(香港理工大学);Southern University of Science and Technology(南方科技大学);Research Institute of Trustworthy Autonomous Systems(可信自主系统研究院)
SGRNet: Spatially Guided Radiology Network for Structured Radiological Reporting of Head and Neck Cancer
University of Strasbourg(斯特拉斯堡大学);IHU Strasbourg(斯特拉斯堡大学医院研究所);Indian Institute of Technology (IIT) Madras(印度马德拉斯理工学院);All India Institute of Medical Sciences(全印度医学科学院)
Radiology Report Generation with a Learned Knowledge Base and Multi-modal Alignment
Institute of Computing Technology, CAS(中国科学院计算技术研究所);University of Science and Technology of China(中国科学技术大学);Tencent Medical AI Lab(腾讯医疗人工智能实验室);University of Chinese Academy of Sciences(中国科学院大学)
Microsoft Research Health Futures(微软研究院健康未来部门);Microsoft Research India(微软研究院印度分部);Microsoft Azure AI(微软Azure人工智能部门);Microsoft Health and Life Sciences(微软健康与生命科学部门);Department of Radiology, Addenbrooke’s Hospital, Cambridge University Hospitals(剑桥大学医院艾登布鲁克医院放射科)
LaB-RAG: Label Boosted Retrieval Augmented Generation for Radiology Report Generation
Department of Computer Science(计算机科学系);Center for Translational Data Science(转化数据科学中心);Medical Scientist Training Program(医学科学家培训计划);Section of Biomedical Data Science, Department of Medicine(医学部生物医学数据科学部门)
Detecting Spurious Correlations with Sanity Tests for Artificial Intelligence Guided Radiology Systems
Memorial Sloan Kettering Cancer Center(纪念斯隆凯特琳癌症中心);Rochester Institute of Technology(罗切斯特理工学院);Institute of Research and Medical Care (IRCCS) SDN(研究与医疗护理研究所(IRCCS)SDN);University of Cagliari(卡利亚里大学)
Joint Modeling of Chest Radiographs and Radiology Reports for Pulmonary Edema Assessment
Massachusetts Institute of Technology(麻省理工学院);Philips Research North America(飞利浦北美研究院);Beth Israel Deaconess Medical Center, Harvard Medical School(哈佛医学院贝斯以色列女执事医疗中心)
Automated Labelling using an Attention model for Radiology reports of MRI scans (ALARM)
School of Biomedical Engineering, King’s College London(伦敦国王学院生物医学工程学院);King’s College Hospital(国王学院医院);Institute of Psychiatry, Psychology & Neuroscience, King’s College London(伦敦国王学院精神病学、心理学与神经科学研究所);Centre for Medical Image Computing, Dementia Research, University College London(伦敦大学学院医学影像计算中心(痴呆研究))
Direct Preference Optimization for Suppressing Hallucinated Prior Exams in Radiology Report Generation
Harvard University(哈佛大学);Jawaharlal Institute of Postgraduate Medical Education & Research(贾瓦哈拉尔 postgraduate 医学教育与研究学院);Johns Hopkins University(约翰斯·霍普金斯大学)
VisualCheXbert: Addressing the Discrepancy Between Radiology Report Labels and Image Labels
Stanford University(斯坦福大学);VinBrain
从自由文本放射学报告中自动提取医学状况对于监督计算机视觉模型解释医学图像至关重要。在这项工作中,我们表明,放射科医生标记报告显着不同意放射科医生标记相应的胸部X射线图像,这降低了质量的报告标签作为图像标签的代理。我们开发和评估的方法,以产生标签的放射学报告,有更好的协议与放射科医生标记图像。我们的最佳性能方法称为VisualCheXbert,它使用生物医学预训练的BERT模型直接从放射学报告映射到图像标签,并通过训练用于从胸部X射线图像中检测医疗状况的计算机视觉模型确定监督信号。我们发现,VisualCheXbert优于使用现有放射学报告标签的方法,平均F1得分为0.14(95%CI 0.12,0.17)。我们还发现,VisualCheXpert更好地同意放射科医生标记胸部X射线图像比放射科医生标记相应的放射学报告之间的平均F1分数在0.12(95% CI 0.09,0.15)和0.21(95% CI 0.18,0.24)之间的几种医疗条件。
BTReport: A Framework for Brain Tumor Radiology Report Generation with Clinically Relevant Features
University of Washington(华盛顿大学);University of Washington School of Medicine(华盛顿大学医学院);Microsoft Health AI(微软健康人工智能);Johns Hopkins School of Medicine(约翰霍普金斯医学院)